Run VNC server before logging to Linux GUI

I installed X11vnc and to my dismay, there isn’t a easy option that automatically configures VNC as a service like most Windows VNC software does (so you can VNC into a computer before you login as a user graphically and launch the X11vnc executable).

I had to manually create a service and I ran into a few problems as the instructions on StackExchange and other forums are missing critical pieces.

In here, I will use X11vnc server on Ubuntu Cinnamon (systemd) as an example. Instead of blindly pasting code here without context, I’ll sketch out the ideas here:

  1. Establish a password in a password file stored in common areas such as /etc/x11vnc.pwd instead of using the user-specific home folder default ~/.vnc/passwd
  2. Create a service (such as systemd) pointing to x11vnc program with the right parameters, which includes the path to the password file stored in common areas
  3. Start the service

It’s worth nothing that the X11server connection is unencrypted. I tried the -ssl options but my RealVNC clients complained about their version


First of all, x11vnc -storepasswd creates the encrypted password file at the current home folder where you run the code. You are going to call the said password file with x11vnc -rbfauth {path to password file} parameter when launching the X11vnc server program.

One way to do it is to copy the created password to a system-specific configuration folder instead of user’s home folder:

sudo cp ~/.vnc/passwd /etc/x11vnc.pwd

Alternatively (which I do not recommend), is to specify the password AND the password-file path directly with optional specifiers of the -storepasswd parameter.

# Directly create the password file without a prompt that hides the password entry
x11vnc -storepasswd my_pASSword /etc/x11vnc.pwd
# Clean up your terminal command history since you've exposed the password visually
history -c

Unfortunately, if you want to specify the path to the password-file, you have to specify type the plain text password in the command line, which you should do it when nobody’s watching and clear the history immediately afterwards. If you are in a public place, just do it the old way and copy the password file over


The core part of setting (doing the data-entry) for registering a service is the figuring out the command line parameters executing x11vnc program. At minimal, you’ll need

  • -rfbauth specifies where the password file is (or you can directly specify the password with -passwd, which I do not recommended)
  • auth: authentication means (prefers –auth guess, but you can specify where your .Xauthority file is)
  • -display: 0 connects to the X11 server display, which is usually 0
  • -create is the missing link! you must absolutely use this tell the VNC server to make a Xvfb (X virtual framebuffer) session if no display session is found (which is the case when you are running X11vnc as a service before logging in the a Desktop Environment like Cinnamon)

You’ll typically want this for a constant-on VNC server

  • -forever: x11server instances are by default (-once) killed after the client disconnects. -forever option keeps it there

My personal preferences

  • -shared: I might have a few computer VNC’ing into the linux computer and I don’t want to make sure I remember to close the ones I’m not using.
  • -noxdamage: XDamage is a system that only updates the changed parts of the screen. Don’t need it when bandwidth isn’t super tight.
  • -repeat: allow hold and repeat keystrokes just like what we are used to. By default it’s set to -norepeat to avoid stuck key scenarios.

For debugging (useful! that’s how I figured out the missing part that I have to use -create to make a dummy screen when using x11vnc as a service):

  • -o {output log file}: typically -o /var/log/x11vnc.log
  • -loop: if the program crashes for any reason, it’ll try to auto-restart for robustness. Might not need it if you use -forever

So the core command needed is:

x11vnc -repeat -noxdamage -create -display :0 -auth guess -rfbauth /etc/x11vnc.pwd -shared -forever

Now after we’ve decided the exact launch command, we will have to create the service entry. In systemd Linux, it’s done by writing a service configuration file in text format very much like Windows INI files under /etc/systemd/system and the filename MUST end with suffix “.service

In short, create /etc/systemd/system/x11vnc.service. Basic file without logging is like this:

[Unit]
Description=VNC Server for X11
Requires=display-manager.service
# The two below are for performance (make sure everything is ready before launching)
After=network-online.target
Wants=network-online.target

[Service]
ExecStart=/usr/bin/x11vnc -repeat -noxdamage -create -display :0 -auth guess -rfbauth /etc/x11vnc.pwd -shared -forever
# The 3 lines below are option, but for robustness
ExecStop=/usr/bin/x11vnc -R stop
Restart=on-failure
RestartSec=2

# For automatically creating symlinks with "sudo systemctl enable" only
[Install]
# Start at runlevel 5 (multi-user)
WantedBy=multi-user.target

This is the minimum skeleton that does the same less robustness against the unexpected:

[Unit]
Description=VNC Server for X11
Requires=display-manager.service

[Service]
ExecStart=/usr/bin/x11vnc -repeat -noxdamage -create -display :0 -auth guess -rfbauth /etc/x11vnc.pwd -shared -forever

[Install]
WantedBy=multi-user.target

The default permissions 644 (everybody reads but only root can write is standard for services. 640, denying unknown people the read access is also acceptable if you are paranoid) should be correct if you use sudo creating the file in the /etc/systemd/system folder.

There are some older tutorials using the (/usr)/lib/systemd/system folder, which are now reserved for automatic entry by programs instead of manual service entry like what we are doing now. Technically either way works, but follow the convention so people know where to find the entries.


After that enable the service file you’ve created (the “.service” suffix is optional when calling), preferably do a daemon-reload to make sure edits in the service file is reflected. If you don’t want to wait until the next book, you can start it with systemctl

sudo systemctl enable x11vnc
sudo systemctl daemon-reload
sudo systemctl start x11vnc

This kind of stuff in Linux is bitchworthy. It’s 2021 now. How come users need to mess with defining their custom services for such a common VNC use case (start before logging in graphically)? Never have to deal with this kind of shit in Windows with VNCs: they always expect users has the computer to themselves and always offer the option to set up the service automatically!

Loading

XFX TS Series 550W Power Supply – Made In China – Bulging Capacitor because it was installed backwards

I opened up my ATX Power Supply as I had it for quite a few years but it has been stowed away and used intermittently until I use it a lot more in my office computer in recent years. I just don’t trust any power supplies Made in China, even from a reputable brand as a couple of decades of working with computers tells me that they are bound to break after a few years, and very often it is the capacitor that rotted and the rest are collateral damages. Lo and behold there is one:

After I took the capacitor out, I noticed something odd: the polarity marker on the circuit board is the reverse of how the capacitor was installed! Holy smokes! I just want to verify if the PCB markings is right or the installer was right, so I installed wires to the capacitor to lift it up so I can connect the multimeter leads across it to measure the voltage polarity. This picture also shows the PCB’s capacitor orientation marking:

And the multimeter reads -5V following the original orientation of the capacitor before I took it out. This means the polarity was reversed!! No wonder the capacitor bulged. I was lucky that it didn’t blow up after a few years of use! Probably it was rated 16V yet only -5V was passed to it so the electrolytic capacitor rotted slowly.

To give XFX Force credit, they didn’t slap the power supply together with the cheapest white label components from the gutters. It uses proper Nichicon and Hitachi capacitor, so it might be the reason that reversed capacitor lasted so many years.

It’s the workmanship in China. If you go with a Red Chinese (Yellow-Soviets) brand, they might use junk components, but don’t think you are safe with foreign companies that has a solid process and design. The cheap labor in China who doesn’t give a crap can still manage to fuck it up. So trust nothing

ElectroBOOM!

Loading

Image Remote Disks with Norton Ghost

Symantec Ghost has been my favorite tool since high school as the user interface is minimalistic (runs fast) yet intuitive. It pretty much has every single feature (use case) you can imagine organized in a sensible way (unlike the fucking linux man pages that drown you with 4 dozens of command switches not logically organized so you have to skim through the entire thing to find out what is relevant).

The software is well made in general so we can get a lot mileage out of old versions. I recently had to clone a drive over the network yet I don’t want to share the image file. My initial plan is to have the remote computer I plan to image the disk attached to it run as slave (in Master-Slave mode Peer-To_Peer over TCP mode), but there are a few hurdles:

  • The documentation didn’t say which port is used. I have to use TCPview to figure it out. It’s Port 6668.
  • Turns out slave mode does not support restoring from a image file located from the (puppet-)master. In other words, the when you connect to the slave session, the file dialog box of “From Image” only shows the files on the slave side! WTF!

It’s strange that you can clone a raw drive / partition from master session to slave session, but you cannot choose image file as a source in place of the source drive. I tried the command line before and no avail. After some web searching I realized that I’m not insane. It was the way Ghost is:

The rules inferred from this table means:

  • image files ALWAYS stay at the slave session
  • direct drive/partition copies is always master pushing data to slave.
  • slave drives are never cloned (read)
  • master cannot read its own files to find image files
  • master can only select remote (slave) image files

First of all, direct drive-to-drive copy are bidirectional. The above list is not entirely accurate, so I stroke through the conclusion derived from the incorrect assumptions above. Y:

The rules for image files do not make much sense to me. Just can’t come up with a good excuse for it. The session have full access to both storage from both sides, and ghost command line’s logic is to make image files fungible with direct drives/partitions. It doesn’t discourage accidental overwrites or prevent one side’s data from being siphoned. All they did is to tease the user by not allowing them to read files/images from the master computer where the user interaction is.


The first instinct is to restore the GHO image I want to push to the server onto a disk and do the direct clone. This is logically fungible with creating a VHD, mount it, restore the GHO image to the mounted drive, then use direct ‘virtual disk’-to-disk clone to restore the remote (slave) disk. Luckily, newer Ghost has tools to simplify these steps. We’ll need this 3 pieces of clues to figure it out:

  1. Virtual machine disk image files such as VHD can be used as source or destination
  2. There’s a command switch to mount virtual machine disk image files internally WITHIN the ghost session (no side effects: windows won’t see it. Won’t persist between ghost sessions)
  3. GHO files are not directly mountable as a virtual disk even internally within ghost session

So the complicated process can be shorten to converting GHO to VHD and then internally mount the VHD as a direct drive through command switch launching Ghost. Use DEMO.gho as an example:

REM Convert DEMO.gho to DEMO.vhd
ghost -clone,mode=restore,src=DEMO.gho,dst=DEMO.vhd

REM Launch Ghost with DEMO.vhd internally mapped as a (direct) logical drive
ghost -ad=DEMO.vhd

I ran into some obscure error messages like “ABORT: 11030, Invalid destination drive” when trying to specify the full absolute path. So instead of fussing with the syntax that breaks the code, I added ghost to my Windows %PATH% environmental variable and run ghost directly at the folder where the files are. I suspect it can be fixed with /translate command switch to make sure the drive letter is not ambigious whether it’s local or remote, but that’s something for later if I have a project that require scripting this reliably.


My cliff notes here.

Run Ghost as slave mode

ghost -tcps

Do this at Ghost master computer

REM Convert DEMO.gho to DEMO.vhd
ghost -clone,mode=restore,src=DEMO.gho,dst=DEMO.vhd

REM Launch Ghost with DEMO.vhd internally mapped as a (direct) logical drive
ghost -ad=DEMO.vhd -tcpm:{IP address of the slave computer}

Remember to open port 6668 at the Ghost slave computer.


Appendix

Technically, it’s possible to restore from an image file located AT THE SLAVE side, but it’d be a stupid idea. Initially I thought Ghost would be smart enough to directly use the image file locally on the slave session to clone the drive locally. However, given the speed and my observation with TCPview, this is not the case. It’s doing the stupid thing of crawling the contents of image file from the slave machine in chunks and send it back to the slave!

Loading

rsync/Deltacopy gotchas (especially Windows transfers)

Deltacopy is a GUI wrapper around rsync, a feature-packed tool to copy files locally AND remotely, AND differentially (automatically figure out the parts that are different and resend. Excellent for repair) through hash comparisons. For non-programmers, hash is a unique ID computed for a chunk of data that are expected to change wildly even at the slightest data/file change/corruption).

Deltacopy is very useful if you just want to do the basic stuff and not know the rsync syntax and switch combinations off the top of your head. It also provides a windows port of rsync based on Cygwin (a tiny Linux runtime environment for windows). This is the only free alternative to cwRsync, a paid Windows port of rsync.

rsync is a Swiss Army Knife that can also work from one local path to another. Deltacopy is intended for remote file transfer.

Deltacopy server is basically this:

rsync --daemon 

However in Windows, since it’s cygwin, it’s looking for linux’s /etc/rsyncd.conf by default if you do not specify the config file through --config switch.

Deltacopy client basically help you generate the command to transfer files. Most of the features are done through right-click (context) menu, not toolbar or pull-down menus, which might confuse some people. You set up your tasks as Profile, which can be scheduled (the bottom panel) or executed immediately by right clicking on the profile:

Run is pushing file to the server, Restore is pulling files from the server. Run Now and Restore are for executing the command (aka task) immediately. You can peek into what it generated by right-clicking on the profile and choose “Display Run/Restore Command”. First time users might not be able to find it since the only place to access it is through context menus.

There are some tricky parts (gotchas) for specifying the files/folders to copy. First of all, even though you use Add Folder/Add Files button for entries

Basically you can make a (source, destination) pair by modifying the selection and target path. It’s just passed onto rsync command verbatim. The target path is relative to the virtual directory set on he server (see Deltacopy Server’s directory)

The destination path which is endowed with the branch folder name (one-level). In other words, if your source is C:/foo/bar, Deltacopy by default set the destination to /bar instead of /. This is probably to avoid the temptation of lumping all contents in the same remote destination root. If you just want to simply lay the files at the root virtual folder at the destination (my most common use case), you’ll have to edit and clear out the (relative) destination path.

As for the source, the author of rsync chose to do it the logical (more conservative) way but not intuitive way: by default it reconstruct the source folder’s FULL path structure at the destination! For example if you intend to copy everything under C:\foo over, the destination will create {destination root}\foo in the process and put everything under it instead of directly at {destination root}. The design choice was supposed to prevent accidental overwrites as multiple source subfolders try to write over each other with the same names at the destination.

Luckily, there’s a way around it! See man pages for -R –relative: Put a dot (.) at the place where the relative path starts! For example, the source is C:\foo\bar\baz and you do not want /foo to be created at the destination and want it to start with /bar instead. You should enter C:\foo\.\bar\baz as source. Everything the left of the dot (refers to self-folder) are stripped from the destination path structure.


ACL support for Windows sucks because rsync lives on cygwin, which has POSIX (unix/linux) type of permissions/ACL.

https://unix.stackexchange.com/questions/547275/how-do-i-use-rsync-to-reliably-transfer-permissions-acls-when-copying-from-ntf

In my opinion, the best way to go about it is to not transfer ACLs from the source and follow the preexisting ACLs at the destination. I’d also leave the groups and owners alone (inherit at destination) as well I might not be on the same active directory (or workgroup user management) as the destination computer so accounts with the same name might not be actually the same accounts.

--no-p --no-g --no-o

–no-{command} is the complement prefix that does the opposite of the -{command}, so the above means skipping -p (perms/permissions), -g (group), -o (owner) and make sure it has full permissions for everybody.


Sometimes a remote path can be mistaken as a relative local path with the hostname/IP address as the folder name if there’s no username. Start it with rsync:// as the URL scheme and the syntax is like ftp:// as far as username is concerned.

Deltacopy protects the source and destination paths with double quotes (“). It’s a good practice that we should do it even with direct rsync calls.

Loading

Tomato OpenVPN client assigned for specific computers

Setting Redirect Internet traffic to “Policy Rules” opens a table where you can specify which computer goes through VPN and which ones uses direct connection. Leave the destination IP unspecified and it’ll pick the 0.0.0.0 as intended

However, there’s a logical trap when you blindly follow instructions setting “Accept DNS configuration” to “Exclusive” as given by most instructions assuming all computers go on the network through VPN. Setting it as “Exclusive” means even the computer not intending to use VPN will still need to go through your VPN provider’s DNS! For slow VPN connection, this will be painfully slow for ALL computers! Set it to “Relaxed” instead.

Loading