## Flashing router firmware through Serial Port: CFE bootloader (usually Broadcom) based routers

Here’s a summary of learnings from dd-wrt’s serial recovery instructions:

1. Use a UART controller that signals at 3.3V (e.g. FTDI TTL-232R-3V3) to talk to the board. Regular serial RS-232 ports requires a voltage level shifter converting the signal to swing between 0V to 3.3V.
2. You only need 3 pins: Tx, Rx and Ground. There’s voltage contention if you plug in the Vcc from TTL-232R-3V3 (It’s the USB’s 5V despite the signaling is 3.3V) to the 3.3V supply of the router. You don’t need the Vcc pin. It didn’t harm anything or do anything when I connected the Vcc pin.
3. Stick with all serial port defaults and only set the baud to 115200 (default is 9600) and turn off flow control (default is xon/xoff). I use Putty for terminal.
4. The terminal serves as the monitor for the computer on the router that shows a text console. Broadcom uses CFE bootloader (others use U-Boot with busybox).
5. CFE bootloader defaults to 192.168.1.1 with subnet mask 255.255.255.0 (aka /24). Set up the network interface to have a static IP on the same subnet to talk to the board.
6. Good habit: nvram erase
7. The flash program relies on TFTP protocol to receive the firmware file. So get your TFTP client ready. Microsoft included a TFTP client/server since Windows 7 but usually disabled (turn it on in Windows OptionalFeatures.exe).
8. TFTP is a simple push(put)/pull(get) design. You can either “push a file on your computer” or “get a file as filename”. You’d want to specify -i switch (binary image transfer) with Windows tftp.exe.
9. So type this command at the command prompt but don not press enter until your router is ready to grab the file: tftp -i 192.168.1.1 put {path to whatever TRX firmware file}
10. Go back to the serial terminal and tell the router to accept a TFTP push (in a window of a few seconds before it time out) and flash the memory region flash1.trx with this command: flash -ctheader : flash1.trx
11. Immediately initiate the TFTP push from your computer (Windows command line example in Step #9 above)
12. Wait for a couple of hours! The terminal might tell you that it has received the file completely, but it won’t show anything when it’s writing to the flash! It’s a painfully slow process with no feedback. Just be patient!

Some observations

• FreshTomato firmware absolutely won’t tell you on the screen after it has done flashing (Merlin-WRT does). Just turn the router back on after a couple of hours.
• Merlin firmware repeats (exposes) the raw passwords to the serial port!
• FreshTomato firmware boots to a linux prompt on the serial port

4 total views,  1 views today

## Asus-wrt Merlin Firmware DDNS update interval hack

The “WAN – DDNS” page only allows users to set the DDNS updater to check as frequently as every 30 minutes. My DDNS provider does not have an update frequency limit, so I’d like to have the update client check for every 1 minute. The setting is called “Verify every”:

Attempting to set it to every 1 minute gives this error message:

I searched for the “WAN-DDNS” config webpage file (Advanced_ASUSDDNS_Content.asp) in the firmware source code, and found that it’s under /www folder in the router’s linux root.

Since “Verify every” is such generic words, and Github does not support exact phrase match in search (I use “in:file” specifier in the search box), I pick “WAN IP and hostname verification” (the closest setting which I expect the code to be in the proximity of the one corresponding to “Verify every”) so it has more unique keywords. The first jump:

Since it’s just a dictionary files, we search for the associated internal variable name
DDNS_verification_enable” which points to the this line in Advanced_ASUSDDNS_Content.asp:

Since this name appeared nowhere else, I traced the “id” attribute above, which is “check_ddns_field” and I see a Javascript (.js) file that process the data from the web page forms:

The variable check_ddns_field appears in the if-else-if branches of change_ddns_settings(), so one of the few next few variables after it is likely to correspond to “Verify with”.

The variable name showed up in 4 branches of if-elseif-else switches (switching DDNS service providers), which ddns_regular_period comes right after

Searching for the class member (or struct field)

Bingo. Here’s the entry value range check code. I’ll change the “30” minutes to “1” minute to enable checking at 1 minute intervals (which I think it’s reasonably responsive for testing and general use).

I’d prefer to check if the input range check is there out of feasibility (i.e. what is the smallest increment) or it’s just set to prevent people from getting banned by the DDNS provider for checking too frequently. I looked into the last occurrence of ddns_regular_period and found this:

Which means the web forms is updating NVRAM (environmental) variable of the same name ddns_regular_period, which appears to be called only in watchdog.c:

And Dang! The code enforces if the ddns_regular_period (on NVRAM) is set to be less than the original 30 minute minimum (invalid condition), it’d be set to the default 60 minutes (1 hr).

It’s actually sloppy coding because the defaults are specified in struct fields in defaults.c:

yet that 60 minutes is hard-coded in watchdog.c. That means if I don’t catch it and only changed the default in one place, the behavior will not be what I expected given the right conditions. This is an example of why software feature expansion are likely to break things. If you have solid code, bugs on updates are likely to happen.

I was curious why it says (period*2)

and suspected the ddns_check_count is incremented in 30 second (half-minute) interval. Since it’s watchdog.c, my natural guess is that the watchdog checks every 30 seconds for these event hooks. Turns out the notes (comments) in the code has “30 seconds periods” noted everywhere.

I searched a little bit more about linux watchdogs and found this useful webpage which explained how it works. I didn’t see /dev/watchdog in my router’s rootfs (root file system) so I assumed it’s a hardware watchdog (embedded linux, so duh).

I was about to dig up the hardware manual for the chipset for my router, but I search for they string HW_RTC_WATCHDOG  first and it showed up in linux kernel code (duh):

Note that the HW_RTC_WATCHDOG is a register in this code base, not the number of seconds from Christian’s Blog. i.e. they are completely different things, but it provided a good keyword lead for me to start digging.

The code seems to be the same for various kernel version so I picked any one of them to understand the behavior. First occurrence is in wdt_enable():

The other places are suspend/resume, so I’ll ignore those for now. Note that wdt_enable() is a static function, so only need to search within the same file. The only active place that calls it is wdt_ping():

So there are only 2 things I’ll need to find out: heartbeat and WDOG_COUNTER_RATE:

…. unfinished

https://bitsum.com/firmware_mod_kit.htm

While it’s a lot of useful learning about Embedded Linux and hunting down source code, for the meantime, given that namecheap does not care if you blindly update every minute, it’s easier to just set up a cron job that runs at every N minutes using curl/wget.

dd-wrt has a place for you to enter the cron scripts with the web interface, but you might need to log into the router using SSH and register the cron job yourself:

The core command is called ‘cru‘, which typing it in the command prompt will show you the very simple usage:



Cron Utility
add:    cru a <unique id> <"min hour day month week command">
delete: cru d <unique id>
list:   cru l

<unique id> is just a tag that you make up to name your task. Again the one-liner command needs to be direct absolute path. My ‘curl‘ program is located in /usr/sbin, so the command is:

cru a ncddns * * * * * /usr/sbin/curl "https://dynamicdns.park-your-domain.com/update?host={subdomain or @ for root}&domain={registered domain name}&password={DDNS-specific password generated by namecheap's domain administration page under Advanced DNS}"

The “* * * * *” refers to run at every “minute, hour, date of month, month, date of week”, in other words, run at every minute in every waking moment. The wild card * means ALL-OF.

Cron job registration through CRU is not persistent, so to make it survive reboots, add the above cru command as a line to /jffs/scripts/services-start script. It should be executable by default, if not, make sure you set it to be executable or it won’t run.

3 total views

## Namecheap Dynamic DNS setup in dd-wrt

Namecheap support page explained the process of configuring your dd-wrt router firmware to use Namecheap’s REST (HTTP URL) update interface to dynamically update the IP of your (sub-)domain. The instruction works, but there are few items which doesn’t quite make sense to me as a programmer, and I did a few experiments to figured that it’s bogus and developed a few insights about what’s necessary and why they do it.

Their instructions looks like this:

and specific verbal instructions are:

• DDNS Service: Custom
• DYNDNS Server: dynamicdns.park-your-domain.com – the name of the server should not be changed
• Username: yourdomain.com – replace it with your domain name
• Password: Dynamic DNS password for your domain (Domain List >> click on the Manage button next to the domain >> the Advanced DNS tab >> Dynamic DNS)
• Hostname: Your subdomain (@ for yourdomain.comwww for www.yourdomain.com, etc.)

I stroke out Username and Password fields because they are not used!

If you look at the URL, namecheap’s instructions are asking you to re-enter the domain and the password key-value pair AGAIN, which means Username and Password fields are not used.

My programmer instinct immediate screams the updater is assuming certain REST API syntax that are not properly substituted so they need to be entered manually, exposing the password without the benefit of the masks (forget about keeping the password top secret, router firmware guys aren’t top security engineers. Just re-generate one in Namecheap’s admin interface if it gets compromised).

I checked by entering bogus Username and Password fields (the web’s GUI/forms checks if they are blank, so you can’t get away with not entering). It worked as expected. This means the two fields are dummies with Namecheap’s instructions.

Based on the fact that Namecheap’s instructions being unable to substitute Username and Password fields and the host key must be put at the end for Hostname field to substitute correctly, I can safely speculate that the one who wrote this couldn’t find out what the syntax for the variables are, and exploit that the last parameter hostname gets attached at the end in the absence of substitution variables in the URL syntax.

Apparent people are doing something stupid like this because nobody in the chain remember to document the substitution variable names! It’s not in dd-wrt’s user interface (should have that printed the ‘usage’ info next to the URL box) and neither it’s in INADYN’s github readmes!

I decided to dig deeper and go after the dynamic DNS updater package in question. dd-wrt is using inadyn package to do the dynamic DNS update, as “INADYN” is shown in “DDNS status” box gives it away (confirmed by dd-wrt’s docs):

The service itself is called ddns though.

I ended up reading the /examples folder on the repository and found this:

Bingo! Here it is:

generic.c is a plugin also shows the above table as well

Since namecheap’s dynamic DNS service do not mandate how frequently you can update nor they charge per update, it’s easiest and most reliable to just blindly update the IP every N minutes instead of checking against a local cache to see if the external IP has really changed before updating at each poll interval

This user interface does not have the option to set the updater to run every 1 minute, so why bother since it’s just a simple program that creates a simple URL and do a curl/wget? At the end of the day, I decided to just do a cron job:

PATH=/sbin:/usr/sbin:/bin:/usr/bin
* * * * * root curl "https://dynamicdns.park-your-domain.com/update?host={subdomain or @}&domain={domain name bought}&password={generated by namecheap's account management page}"

There are 3 things that you will need to know:

• Paths is from a clean slate. Need to define it first
• * * * * * means every minute. Specify numbers/range for each time unit (minute, hour, day of month, month, day of week) if desired. Asterisk means ON EVERY.
• Need to specify the user after the time syntax and before the actual command

3 total views

## dd-wrt gotchas

dd-wrt is very a powerful firmware compared to ASUS Merlin, but the UI leaves a lot to be desired. It’s very close to editing a config file and there’s little help to what each setting. The developers of dd-wrt didn’t invest time in designing the web administration interface and used the most basic primitive HTML forms so there are no tooltip that explains the features and the interaction between different settings.

There are also some confusing (nonsensical) UI design that are a lot less work to the developer but confused users to no end. Here are the examples I’ve found so far:

• Enabling remote admin through SSH (for embedded linux command prompt) is a two step process out of the box. You’ll need to first enable SSHd from Services -> Secure Shell before enabling SSH Management from Administration -> Management (otherwise it’s greyed out)
• The router username (user modifiable) for dd-wrt applies to web UI only. SSH’s username remains root. They share the same password though (so login and password are decoupled in dd-wrt, they are effectively two passwords in practice except they don’t put asterisk over the username as you type). ASUS Merlin firmware’s login is consistent across both web page and SSH
• Cron jobs is from a bare environment which means you need to manually define the paths and specify the user in the cron job syntax. e.g.
PATH=/sbin:/usr/sbin:/bin:/usr/bin* * * * * root {command_to_execute}

3 total views

## Remote Desktop INTO Linux GUI (xrdp)

To serve Linux Desktop just like other Windows computer through Windows Remote Desktop (formerly Terminal Services), so far I have found xrdp (xorgxrdp). VNCs, NX (NoMachine)/TeamViewer does not count because they share the screen of an existing session, instead of creating a new one for you.

Xrdp does not follow the use pattern as Microsoft’s RDP. When you log in to a Xrdp host (server) through a RDP client, you go into an intermediary (welcome interface) called sesman (Session Manager), which is a multi-protocol remote graphical session client (think of it as a very rudimentary Remmina).

The two session modules we are interested in here is

• Xorg (libxup.so): Xorgxrdp is the MS-RDP-like mode that starts a new X session without first attaching to a screen.
• Xvnc (libvnc.so): basically a VNC client. You start a VNC server (like X11vnc) with a display/screen (can be started in any X session you logged in, or the local user screen if you set the VNC server as a service) and connect to it in this RDP intermediary (welcome interface) without installing VNC client software.

In Windows, RDP do not distinguish between local and remote users and sessions with the same login account will take over other existing sessions. If you want each session to start fresh and leave other sessions alone, disable this in Group Policy Object editor under Computer Configuration > Administrative Templates > Windows Components > Remote Desktop Services > Remote Desktop Session Host > Connections -> "Restrict Remote Desktop Services user to a single Remote Desktop Services session“.

I am usually fine with this arrangement as well, but often prefer to connect to my remote sessions work in the background leaving the local user alone (i.e. if I want things to show up on the local monitor screen, I’ll use VNC instead). I’d also like to resume my remote sessions if I log in from another computer instead of starting from scratch with each new RDP connection. Turns out given a bunch of quirks of xrdp, this is much easier to do so than reproducing MS-RDP’s default behavior.

First of all, out of the box, the same remote user cannot overtake locally logged-in desktops nor be simultaneously logged in! It’s either one way or the other! I got bumped out immediately after logging in through sesman, or if I logged in remotely first, I get bumped out when I try to log in locally.

Found somebody suggested that certain desktop environment might have added code to prevent the second session from opening. And this blog suggested you edit the windows manager launch script

sudo nano /etc/xrdp/startwm.sh

unset DBUS_SESSION_BUS_ADDRESS
unset XDG_RUNTIME_DIR

OR

export \$(dbus-launch)

RIGHT BEFORE the last lines which checks and calls the Xsession

test -x /etc/X11/Xsession && exec /etc/X11/Xsessionexec /bin/sh /etc/X11/Xsession

This only solves the part of simultaneous local & remote logons

In the newer version as of writing, the default behavior is that locally logged in sessions are independent of remotely logged in sessions, yet the remotely logged in sessions resumes by default (if you log in as the same user). Turns out this is what I preferred as the local sessions should be reached with VNC instead and I’d prefer my remote sessions happen at the background without showing it on the local screen.

Since the VNC server only has a password (not using system’s active directory or user management system), there is no user name. I went to [Xvnc] section of xrdp.ini and replaced username=ask with username=na. The port number -1 no longer applies as we aren’t emulating RDP with VNC anymore (where sesman creates a new VNC server instance if not previously done). Given that I’m running VNC as a service at default port 5900, I also changed it to port=5900.

Session/Module “vnc-any” uses the same libvnc.so as Xvnc before it, and they are pretty much the same thing except it exposes ip:port entry so you can use it as a gateway to connect to VNC servers hosted on other machines (can be used to connect to the VNC server on the current machine you just connected to through RDP if you stick with 127.0.0.1:5900). It’s more like a convenience thing that hosts the VNC client software that you can RDP into (so you don’t need to install a VNC client from where you are).

There is also a RDP client module/session called ‘neutrinordp-any’, which basically uses the linux machine you just connected to as a gateway to visit another machine hosting RDP. It’s rarely useful and it doesn’t work out of the box when I tried it (does nothing after I press OK despite entering all the info correctly). So I removed it from my xrdp.ini

There’s also a minor annoyance that if you connect remotely, “Authentication Required…” message box will show up on start since remote user is a little more restrictive than local users. This can be solved by creating this file with nano

sudo nano /etc/polkit-1/localauthority/50-local.d/46-allow-update-repo.pkla

and paste the contents there and save it:

[Allow Package Management all Users]
Identity=unix-user:*
Action=org.freedesktop.packagekit.system-sources-refresh
ResultAny=yes
ResultInactive=yes
ResultActive=yes

8 total views

## Run VNC server before logging to Linux GUI

I installed X11vnc and to my dismay, there isn’t a easy option that automatically configures VNC as a service like most Windows VNC software does (so you can VNC into a computer before you login as a user graphically and launch the X11vnc executable).

I had to manually create a service and I ran into a few problems as the instructions on StackExchange and other forums are missing critical pieces.

In here, I will use X11vnc server on Ubuntu Cinnamon (systemd) as an example. Instead of blindly pasting code here without context, I’ll sketch out the ideas here:

1. Establish a password in a password file stored in common areas such as /etc/x11vnc.pwd instead of using the user-specific home folder default ~/.vnc/passwd
2. Create a service (such as systemd) pointing to x11vnc program with the right parameters, which includes the path to the password file stored in common areas
3. Start the service

It’s worth nothing that the X11server connection is unencrypted. I tried the -ssl options but my RealVNC clients complained about their version

First of all, x11vnc -storepasswd creates the encrypted password file at the current home folder where you run the code. You are going to call the said password file with x11vnc -rbfauth {path to password file} parameter when launching the X11vnc server program.

One way to do it is to copy the created password to a system-specific configuration folder instead of user’s home folder:

sudo cp ~/.vnc/passwd /etc/x11vnc.pwd

Alternatively (which I do not recommend), is to specify the password AND the password-file path directly with optional specifiers of the -storepasswd parameter.

# Directly create the password file without a prompt that hides the password entry
x11vnc -storepasswd my_pASSword /etc/x11vnc.pwd
# Clean up your terminal command history since you've exposed the password visually
history -c

Unfortunately, if you want to specify the path to the password-file, you have to specify type the plain text password in the command line, which you should do it when nobody’s watching and clear the history immediately afterwards. If you are in a public place, just do it the old way and copy the password file over

The core part of setting (doing the data-entry) for registering a service is the figuring out the command line parameters executing x11vnc program. At minimal, you’ll need

• -rfbauth specifies where the password file is (or you can directly specify the password with -passwd, which I do not recommended)
• auth: authentication means (prefers –auth guess, but you can specify where your .Xauthority file is)
• -display: 0 connects to the X11 server display, which is usually 0
• -create is the missing link! you must absolutely use this tell the VNC server to make a Xvfb (X virtual framebuffer) session if no display session is found (which is the case when you are running X11vnc as a service before logging in the a Desktop Environment like Cinnamon)

You’ll typically want this for a constant-on VNC server

• -forever: x11server instances are by default (-once) killed after the client disconnects. -forever option keeps it there

My personal preferences

• -shared: I might have a few computer VNC’ing into the linux computer and I don’t want to make sure I remember to close the ones I’m not using.
• -noxdamage: XDamage is a system that only updates the changed parts of the screen. Don’t need it when bandwidth isn’t super tight.
• -repeat: allow hold and repeat keystrokes just like what we are used to. By default it’s set to -norepeat to avoid stuck key scenarios.

For debugging (useful! that’s how I figured out the missing part that I have to use -create to make a dummy screen when using x11vnc as a service):

• -o {output log file}: typically -o /var/log/x11vnc.log
• -loop: if the program crashes for any reason, it’ll try to auto-restart for robustness. Might not need it if you use -forever

So the core command needed is:

x11vnc -repeat -noxdamage -create -display :0 -auth guess -rfbauth /etc/x11vnc.pwd -shared -forever

Now after we’ve decided the exact launch command, we will have to create the service entry. In systemd Linux, it’s done by writing a service configuration file in text format very much like Windows INI files under /etc/systemd/system and the filename MUST end with suffix “.service

In short, create /etc/systemd/system/x11vnc.service. Basic file without logging is like this:

[Unit]
Description=VNC Server for X11
Requires=display-manager.service
# The two below are for performance (make sure everything is ready before launching)
After=network-online.target
Wants=network-online.target

[Service]
ExecStart=/usr/bin/x11vnc -repeat -noxdamage -create -display :0 -auth guess -rfbauth /etc/x11vnc.pwd -shared -forever
# The 3 lines below are option, but for robustness
ExecStop=/usr/bin/x11vnc -R stop
Restart=on-failure
RestartSec=2

# For automatically creating symlinks with "sudo systemctl enable" only
[Install]
# Start at runlevel 5 (multi-user)
WantedBy=multi-user.target

This is the minimum skeleton that does the same less robustness against the unexpected:

[Unit]
Description=VNC Server for X11
Requires=display-manager.service

[Service]
ExecStart=/usr/bin/x11vnc -repeat -noxdamage -create -display :0 -auth guess -rfbauth /etc/x11vnc.pwd -shared -forever

[Install]
WantedBy=multi-user.target

The default permissions 644 (everybody reads but only root can write is standard for services. 640, denying unknown people the read access is also acceptable if you are paranoid) should be correct if you use sudo creating the file in the /etc/systemd/system folder.

There are some older tutorials using the (/usr)/lib/systemd/system folder, which are now reserved for automatic entry by programs instead of manual service entry like what we are doing now. Technically either way works, but follow the convention so people know where to find the entries.

After that enable the service file you’ve created (the “.service” suffix is optional when calling), preferably do a daemon-reload to make sure edits in the service file is reflected. If you don’t want to wait until the next book, you can start it with systemctl

sudo systemctl enable x11vnc
sudo systemctl start x11vnc

This kind of stuff in Linux is bitchworthy. It’s 2021 now. How come users need to mess with defining their custom services for such a common VNC use case (start before logging in graphically)? Never have to deal with this kind of shit in Windows with VNCs: they always expect users has the computer to themselves and always offer the option to set up the service automatically!

12 total views

## XFX TS Series 550W Power Supply – Made In China – Bulging Capacitor because it was installed backwards

I opened up my ATX Power Supply as I had it for quite a few years but it has been stowed away and used intermittently until I use it a lot more in my office computer in recent years. I just don’t trust any power supplies Made in China, even from a reputable brand as a couple of decades of working with computers tells me that they are bound to break after a few years, and very often it is the capacitor that rotted and the rest are collateral damages. Lo and behold there is one:

After I took the capacitor out, I noticed something odd: the polarity marker on the circuit board is the reverse of how the capacitor was installed! Holy smokes! I just want to verify if the PCB markings is right or the installer was right, so I installed wires to the capacitor to lift it up so I can connect the multimeter leads across it to measure the voltage polarity. This picture also shows the PCB’s capacitor orientation marking:

And the multimeter reads -5V following the original orientation of the capacitor before I took it out. This means the polarity was reversed!! No wonder the capacitor bulged. I was lucky that it didn’t blow up after a few years of use! Probably it was rated 16V yet only -5V was passed to it so the electrolytic capacitor rotted slowly.

To give XFX Force credit, they didn’t slap the power supply together with the cheapest white label components from the gutters. It uses proper Nichicon and Hitachi capacitor, so it might be the reason that reversed capacitor lasted so many years.

It’s the workmanship in China. If you go with a Red Chinese (Yellow-Soviets) brand, they might use junk components, but don’t think you are safe with foreign companies that has a solid process and design. The cheap labor in China who doesn’t give a crap can still manage to fuck it up. So trust nothing

ElectroBOOM!

15 total views

## Image Remote Disks with Norton Ghost

Symantec Ghost has been my favorite tool since high school as the user interface is minimalistic (runs fast) yet intuitive. It pretty much has every single feature (use case) you can imagine organized in a sensible way (unlike the fucking linux man pages that drown you with 4 dozens of command switches not logically organized so you have to skim through the entire thing to find out what is relevant).

The software is well made in general so we can get a lot mileage out of old versions. I recently had to clone a drive over the network yet I don’t want to share the image file. My initial plan is to have the remote computer I plan to image the disk attached to it run as slave (in Master-Slave mode Peer-To_Peer over TCP mode), but there are a few hurdles:

• The documentation didn’t say which port is used. I have to use TCPview to figure it out. It’s Port 6668.
• Turns out slave mode does not support restoring from a image file located from the (puppet-)master. In other words, the when you connect to the slave session, the file dialog box of “From Image” only shows the files on the slave side! WTF!

It’s strange that you can clone a raw drive / partition from master session to slave session, but you cannot choose image file as a source in place of the source drive. I tried the command line before and no avail. After some web searching I realized that I’m not insane. It was the way Ghost is:

The rules inferred from this table means:

• image files ALWAYS stay at the slave session
• direct drive/partition copies is always master pushing data to slave.
• slave drives are never cloned (read)
• master cannot read its own files to find image files
• master can only select remote (slave) image files

First of all, direct drive-to-drive copy are bidirectional. The above list is not entirely accurate, so I stroke through the conclusion derived from the incorrect assumptions above. Y:

The rules for image files do not make much sense to me. Just can’t come up with a good excuse for it. The session have full access to both storage from both sides, and ghost command line’s logic is to make image files fungible with direct drives/partitions. It doesn’t discourage accidental overwrites or prevent one side’s data from being siphoned. All they did is to tease the user by not allowing them to read files/images from the master computer where the user interaction is.

The first instinct is to restore the GHO image I want to push to the server onto a disk and do the direct clone. This is logically fungible with creating a VHD, mount it, restore the GHO image to the mounted drive, then use direct ‘virtual disk’-to-disk clone to restore the remote (slave) disk. Luckily, newer Ghost has tools to simplify these steps. We’ll need this 3 pieces of clues to figure it out:

1. Virtual machine disk image files such as VHD can be used as source or destination
2. There’s a command switch to mount virtual machine disk image files internally WITHIN the ghost session (no side effects: windows won’t see it. Won’t persist between ghost sessions)
3. GHO files are not directly mountable as a virtual disk even internally within ghost session

So the complicated process can be shorten to converting GHO to VHD and then internally mount the VHD as a direct drive through command switch launching Ghost. Use DEMO.gho as an example:

REM Convert DEMO.gho to DEMO.vhd
ghost -clone,mode=restore,src=DEMO.gho,dst=DEMO.vhd

REM Launch Ghost with DEMO.vhd internally mapped as a (direct) logical drive
ghost -ad=DEMO.vhd

I ran into some obscure error messages like “ABORT: 11030, Invalid destination drive” when trying to specify the full absolute path. So instead of fussing with the syntax that breaks the code, I added ghost to my Windows %PATH% environmental variable and run ghost directly at the folder where the files are. I suspect it can be fixed with /translate command switch to make sure the drive letter is not ambigious whether it’s local or remote, but that’s something for later if I have a project that require scripting this reliably.

My cliff notes here.

Run Ghost as slave mode

ghost -tcps

Do this at Ghost master computer

REM Convert DEMO.gho to DEMO.vhd
ghost -clone,mode=restore,src=DEMO.gho,dst=DEMO.vhd

REM Launch Ghost with DEMO.vhd internally mapped as a (direct) logical drive
ghost -ad=DEMO.vhd -tcpm:{IP address of the slave computer}

Remember to open port 6668 at the Ghost slave computer.

Appendix

Technically, it’s possible to restore from an image file located AT THE SLAVE side, but it’d be a stupid idea. Initially I thought Ghost would be smart enough to directly use the image file locally on the slave session to clone the drive locally. However, given the speed and my observation with TCPview, this is not the case. It’s doing the stupid thing of crawling the contents of image file from the slave machine in chunks and send it back to the slave!

59 total views

## rsync/Deltacopy gotchas (especially Windows transfers)

Deltacopy is a GUI wrapper around rsync, a feature-packed tool to copy files locally AND remotely, AND differentially (automatically figure out the parts that are different and resend. Excellent for repair) through hash comparisons. For non-programmers, hash is a unique ID computed for a chunk of data that are expected to change wildly even at the slightest data/file change/corruption).

Deltacopy is very useful if you just want to do the basic stuff and not know the rsync syntax and switch combinations off the top of your head. It also provides a windows port of rsync based on Cygwin (a tiny Linux runtime environment for windows). This is the only free alternative to cwRsync, a paid Windows port of rsync.

rsync is a Swiss Army Knife that can also work from one local path to another. Deltacopy is intended for remote file transfer.

Deltacopy server is basically this:

rsync --daemon 

However in Windows, since it’s cygwin, it’s looking for linux’s /etc/rsyncd.conf by default if you do not specify the config file through --config switch.

Deltacopy client basically help you generate the command to transfer files. Most of the features are done through right-click (context) menu, not toolbar or pull-down menus, which might confuse some people. You set up your tasks as Profile, which can be scheduled (the bottom panel) or executed immediately by right clicking on the profile:

Run is pushing file to the server, Restore is pulling files from the server. Run Now and Restore are for executing the command (aka task) immediately. You can peek into what it generated by right-clicking on the profile and choose “Display Run/Restore Command”. First time users might not be able to find it since the only place to access it is through context menus.

There are some tricky parts (gotchas) for specifying the files/folders to copy. First of all, even though you use Add Folder/Add Files button for entries

Basically you can make a (source, destination) pair by modifying the selection and target path. It’s just passed onto rsync command verbatim. The target path is relative to the virtual directory set on he server (see Deltacopy Server’s directory)

The destination path which is endowed with the branch folder name (one-level). In other words, if your source is C:/foo/bar, Deltacopy by default set the destination to /bar instead of /. This is probably to avoid the temptation of lumping all contents in the same remote destination root. If you just want to simply lay the files at the root virtual folder at the destination (my most common use case), you’ll have to edit and clear out the (relative) destination path.

As for the source, the author of rsync chose to do it the logical (more conservative) way but not intuitive way: by default it reconstruct the source folder’s FULL path structure at the destination! For example if you intend to copy everything under C:\foo over, the destination will create {destination root}\foo in the process and put everything under it instead of directly at {destination root}. The design choice was supposed to prevent accidental overwrites as multiple source subfolders try to write over each other with the same names at the destination.

Luckily, there’s a way around it! See man pages for -R –relative: Put a dot (.) at the place where the relative path starts! For example, the source is C:\foo\bar\baz and you do not want /foo to be created at the destination and want it to start with /bar instead. You should enter C:\foo\.\bar\baz as source. Everything the left of the dot (refers to self-folder) are stripped from the destination path structure.

ACL support for Windows sucks because rsync lives on cygwin, which has POSIX (unix/linux) type of permissions/ACL.

https://unix.stackexchange.com/questions/547275/how-do-i-use-rsync-to-reliably-transfer-permissions-acls-when-copying-from-ntf

In my opinion, the best way to go about it is to not transfer ACLs from the source and follow the preexisting ACLs at the destination. I’d also leave the groups and owners alone (inherit at destination) as well I might not be on the same active directory (or workgroup user management) as the destination computer so accounts with the same name might not be actually the same accounts.

--no-p --no-g --no-o

–no-{command} is the complement prefix that does the opposite of the -{command}, so the above means skipping -p (perms/permissions), -g (group), -o (owner) and make sure it has full permissions for everybody.

Sometimes a remote path can be mistaken as a relative local path with the hostname/IP address as the folder name if there’s no username. Start it with rsync:// as the URL scheme and the syntax is like ftp:// as far as username is concerned.

Deltacopy protects the source and destination paths with double quotes (“). It’s a good practice that we should do it even with direct rsync calls.

33 total views

## Tomato OpenVPN client assigned for specific computers

Setting Redirect Internet traffic to “Policy Rules” opens a table where you can specify which computer goes through VPN and which ones uses direct connection. Leave the destination IP unspecified and it’ll pick the 0.0.0.0 as intended

However, there’s a logical trap when you blindly follow instructions setting “Accept DNS configuration” to “Exclusive” as given by most instructions assuming all computers go on the network through VPN. Setting it as “Exclusive” means even the computer not intending to use VPN will still need to go through your VPN provider’s DNS! For slow VPN connection, this will be painfully slow for ALL computers! Set it to “Relaxed” instead.

36 total views