Exploiting Short-Circuit Evaluation for conditional execution

TLDR:

T && X is equivalent to "if( T) then run X"
T || X is equivalent to "if(!T) then run X"

I don’t exploit this too much in C/C++ because it’s hard to read (and therefore hard to keep track of it to make sure it’s bug free) and most often I’m interested in the output value so I have to watch out for the side effects. However this is common in Bash scripts

Domination Property

In languages that expressions evaluates to a value, sometimes if-statements can be replaced by short-circuit evaluation because short-circuit evaluation exploits the domination property of AND and OR logic operations:

\newcommand{\Hquad}{\hspace{0.5em}}
\begin{alignat*}{2}
0 \Hquad & \mathrm{AND} &  \Hquad X  =  0 \\
1 \Hquad & \mathrm{OR} & \Hquad X =  1
\end{alignat*}

When you FIRST run into the dominant value for the binary operation (0 for AND) and (1 for OR), evaluate no further (i.e. skip the rest) because rest won’t change the overall result away from the dominant value.

So in this use case (emulating if-then statements), what the latter expression X evaluates to or what the combined logic value is irrelevant. We are merely tricking the short-circuit mechanism to trip (short) to NOT evaluate based on what the earlier expression turned out. Action is the ‘norm’. Conditional inaction is the essence of this idiom.

The dual of domination property is idempotent, which is easier to reason because if pre-condition (say T) forces overall expression to boil down to the expression we want to conditionally execute (say X), we are stuck evaluating X if condition T is met.

\newcommand{\Hquad}{\hspace{0.5em}}
\begin{alignat*}{2}
1 \Hquad & \mathrm{AND} &  \Hquad X  =  X \\
0 \Hquad & \mathrm{OR} & \Hquad X =  X
\end{alignat*}

These 2 possibilities (domination and idempotency) partitions to space (choices) of possibilities (i.e. cover all possible combinations), in other words there are no other scenarios than described. So the precondition T decides whether you run X or not, which is the equivalent of an if-then statement.

Operator (function) view of logic domination [Functional programming perspective]

By grouping the first value and the binary logic operator with a pair of parenthesis, in dominance view

  • (0 AND) is also called the ‘clear’ operator
  • (1 OR) is also called the ‘set’ operator

but this view is not too interesting for our case because we are not interested in what the conditional expression and the overall expression evaluates to, which is signified by ‘clear’ and ‘set’.

On the other hand, (1 AND) and (0 OR) are pass-through (idempotent) operators which passes the evaluation to the latter expression X.

\newcommand{\Hquad}{\hspace{0.5em}}
\begin{alignat*}{2}
(1 \Hquad & \mathrm{AND}) \Hquad & \circ &  \Hquad X  =  X \\
(0 \Hquad & \mathrm{OR}) \Hquad & \circ & \Hquad X =  X
\end{alignat*}

This reads

  • (1 AND): pass-through (evaluate latter expression) if (earlier expression is) TRUE
  • (0 OR): pass-through (evaluate latter expression) if (earlier expression is) FALSE

Let’s call T the condition to test (the ‘if’-condition). The expression to run remain X.

\newcommand{\Hquad}{\hspace{0.5em}}
\begin{alignat}{2}
(T \Hquad \mathrm{AND}) X & = \overline{f_T}(X) \\
(T \Hquad \mathrm{OR}) X & = f_T(X)
\end{alignat}

where

  • \overline{f_T} reads “Run if T is false”
  • f_T reads “Run if T is true”.

Loading

SoftEther VPN Server Firewall instructions

The firewall rules in MerlinWRT just quit working so the table I entered doesn’t do anything. Seems like other people had the same problem too. So if you just use the WebUI, it’s either turning the firewall all on or all off.

There’s a twist in configuring the firewall with iptables in routers: the instruction you got on the Internet often append the entry to the end of the table in the section (they use the -A switch) and they didn’t explicitly tell you that’s what they are doing and what the implications are.

Turns out when you enable the firewall in the router, it enters a table of DROP ALL lines, which is supposed to be a catch all after all the exceptions you spelled out! You are supposed to enter the exceptions BEFORE the default entries set by the router, not append after, so the packet you want to accept gets fished out before it reaches the catch all (drop all) entries entered by the router automatically when you turn on firewall. So the solution is to replace the -A (append) switch with -I (insert) switch

As for putting SoftEther VPN on MerlinWRT, here’s the commands to type in SSH to add the firewall rules to open the port, which is what the broken WebUI interface was supposed to do:

iptables -I INPUT -p tcp -m multiport --dport 443,992,5555,8888 -j ACCEPT

This line opens SoftEther’s standard ports 443,992,5555,8888, all TCP only.

Don’t even think of using iptables-save as the router do not have persistent storage other than /jffs or USB. I suspect iptables-save merely writes to RAM disk so the changes will be lost on next boot.

You’ll need to put this line in /jffs/scripts/firewall-start script instead. If this is a new file, make sure you chmod +x /jffs/scripts/firewall-start to make the file executable.

Well it’s not too bad of a design after-all since it’s a mess to fish out the iptable lines you want to delete, so keeping your changes in a file to be loaded with the firewall on boot is a smart move.

If all your devices downstream that you forward incoming traffic to (aka servers) has firewalls, the built-in firewall mainly protect the router itself, so if it gets too frustrating, it might not be too big of a deal to turn the router’s firewall off.

There’s also another weird behavior that if the port is firewall blocked, the server admin program intermittently still connect but it connects to a blank state server (blank config). WTF!

More bonus: open ports for UDP acceleration

Setup L2TP/IPsec VPN Server on SoftEther VPN Server – SoftEther VPN Project

I believe UDP port 500 is IKE and UDP port 4500 is IPSec NAT transverasal, which I think it’s used by L2TP/IPsec if you turn it on.

If you use OpenVPN server part of it, open UDP port 1194:

But according to this forum discussion, it simply look up the source code for UDP acceleration to determine the UDP port range they are using:

So I’ll add this line to /jffs/scripts/firewall-start to open UDP ports 500,4500,40000:44999 as well

iptables -I INPUT -p udp -m multiport --dport 500,4500,40000:44999 -j ACCEPT

Loading

Quirks of SoftEther VPN Server that came with DD-WRT

DD-WRT came with SoftEther VPN and it looked pretty scary because it shows no user interface and a box for you to enter a config file!

Turns out SoftEther VPN’s Remote Admin interface is basically a tool that takes all user settings, generate a config file and upload it to the server behind the scenes!

However out of the box the DD-WRT did not specify which config file to tie to (or open with by default) so the config received from the Server Manager only stays in memory and is not written anywhere! WTF!

So every time I reboot, the settings are totally lost and I have to re-enter it from scratch. Linux mentality again! People did the hardest fun work showing off how smart they are yet their software doesn’t gain the mainstream adoption because nobody ties up the loose ends so the 95% excellent work got sabotaged by the 5% loose ends that are not tied!

Here’s the tutorial on how to fix this nonsense: DD-WRT :: View topic – SoftEther Working after reboots *Special Way with JFFS2* but it the path was stale, so I’m writing this tutorial to explain the reasoning behind it instead of just parroting the cookbook instructions.

Concepts to know

vpn_server.config is the core file the defines the server.

SoftEther VPN Server Manager (The remote admin program) is basically a tool to generate the config text string from the UI and pass it to the server’s memory (not file yet until the server stops and flushes its config state out)

Config file’s file access/update mechanism

SoftEther’s explicitly stated that their config file (vpn_server.config) handling mechanism does not flush the current state in use to the file until the server stops. So it has the following behavioral implications:

  • Changes made by the admin program is reflected immediately despite no file is updated
  • If you change the config file while the server is running, the changes will be lost/overwritten as the server flushes the data on RAM to disk
  • If you abruptly power off the server, the changes made while the server is running (through the remote admin program) is lost as it doesn’t have a chance to flush the updated state out.
  • If you read the config file while the server is running, you are not getting the changes that are currently done through the Manager (Remote Admin) program.
  • In summary the config file is stale while the server is running.
  • When loading the config file to the server by any means (command or GUI), the server parses the entire file and immediately scan and act on the new state defined by the config file (eager execution), not waiting for the next turn the specific state is accessed (lazy execution).

Why this nonsensical starting from blank slate on boot crap!?

Note that the above describes a WRITE-ONLY mechanism. It says nothing about READs. I suspect what causes the dd-wrt developer to overlook the obvious is that the program by default (without the config filename passed as an argument) do not read from the default config file it writes to!

So unless intervened by specifying a config file location that will be read on launch then flushed out (write) to on server stopping, the server always starts in a blank state (because there’s no config file to read and SoftEther do not define a default location), yet the server always write out to the default file on close.

This is why every time you reboot the router, it’s going to read from nothing (starting the server in a blank slate) and writes to a config file (on proper shutdown) that the server is never going to read on the next launch! WTF!

This is a very odd design choice. SoftEther VPN is a hell lot more polished than most linux mess I’ve seen, but this loose end requires the developer downstream to put proper wrappers/indirections to make it intuitively work out of the box, but apparently this odd user trap just slips right through.

Default location of the config file

SoftEther by default reads and writes the vpn_server.config file which controls everything in a JFFS folder. So you are better off identifying it by doing a path search:

find /jffs -name vpn_server.config

So for my case the core file path is

/jffs/var/softethervpn/vpn_server.config

DD-WRT’s text entry box for config string

The text box in DD-WRT’s UI for SoftEther VPN is hardwired to /tmp/vpn_server.config, which is freaking used by nowhere unless the user points to it. WTF?! This is very unpolished and wastes people a lot more time than it saves. At least drop people a hint with a text note saying this is not done yet and the rail connects to nowhere!

Messy solution

  1. The server always start blank on boot/reboot. No ifs-and-buts
  2. Use a one-liner command to load the config file into the the VPN server’s volatile config memory on start.
  3. Give the text edit box on DD-WRT productive life by symbolic linking

This is the dd-wrt startup command/script given in the forum (the path is stale) so I’ll update it below and deconstruct what it does

Step 1: Load the config file into the running server’s memory on start

vpncmd is a prompt based command line user interface like diskpart, but there’s a shortcut to log into the server and execute a command like ConfigSet in one line instead of starting vpncmd first then type the syntax, which is the /CMD switch:

vpncmd localhost:5555 /SERVER /PASSWORD: /CMD ConfigSet //jffs//var//softethervpn//vpn_server.config

Note that the program uses ‘//’ for the path names to prevent the ‘/’ symbol from being misinterpreted as a command switch.

The path I have is the default path mentioned above. Technically you can load any config file anywhere, but if you want to read the config file SoftEther VPN server flushes out (the most updated state after your changes through the Remote Admin interface), stick with loading the default path. This is likely what most people wanted.

/SERVER merely means the remote admin interface is going to administer a SoftEther VPN Server/Bridge, not Client, or VPN tools mode. Yes, you can puppeteer the Client setup managing connections from elsewhere with the all-in-one vpncmd tool, so the distinction is necessary.

/PASSWORD: should be left empty as the SoftEther VPN server ALWAYS start in a blank state (with a blank password) until you explicitly tells the server program to load a config file into its memory. The server starts blank is the reason we had to go through this drivel in the first place. If you had a set password, you already had a config file loaded.

Step 2 (Optional): Activate the text edit box at the DD-WRT interface

Just use a symbolic link to point the hard-wired /tmp/vpn_server.config to where your default config file path is. This is a mere convenience for you to view what just loaded because when the server is running, this file is guaranteed stale. If you want to read the updated copy or make changes, you have to Disable the SoftEther VPN server using the radio button, wait a few second and refresh so

ln -s /jffs/var/softethervpn/vpn_server.config /tmp/vpn_server.config

Remember to delete /tmp/vpn_server.config first before creating the symbolic link because the node you want it to redirect should not be occupied by an existing file. Obviously back up the content if you want to use it elsewhere.

Step 1 + Step 2 on DD-WRT’s interface

vpncmd localhost:5555 /SERVER /PASSWORD: /CMD ConfigSet //jffs//var//softethervpn//vpn_server.config
ln -s /jffs/var/softethervpn/vpn_server.config /tmp/vpn_server.config

Step 3: Add the TAP adapter to the LAN Bridge

This part is described in my other article for a different router firmware platform (Merlin WRT), but the same idea translates here: you need a TAP adapter to put SoftEther VPN Server on a Router, despite SoftEther UI has the option to create the TAP adapter, it doesn’t have the feature to add that newly created TAP adapter to the LAN bridge and therefore renders the feature useless out of the box the way it came with DD-WRT!

The good part about DD-WRT is that the tun kernel module is already loaded so SoftEther VPN server can freely make the TAP adapter (for Merlin you need to modprobe tun first).

Since the TAP adapter only appears when SoftEther’s VPN is running WITH with config file that defines the TAP adapter, you have to make sure the command to add the TAP to the LAN bridge happens a little right after the TAP is created by SoftEther VPN server, namely when the server just started the config file you know that contains the TAP adapter (and you know the said TAP adapter’s name).

If you call your TAP adapter tap0 in SoftEther’s config, it’s called tap_tap0 in Linux when it exist. Most often the LAN bridge is called br0, so if you have these common default names, the command to add the TAP interface to the LAN bridge is

brctl addif br0 tap_tap0

Of course replace the names accordingly. You can check the names by ifconfig or ip link show, or use whichever tool you know that lists all network device and adapters.

Since I noticed that DD-WRT and the VPN service /opt/etc/init.d files merely enable/disable the incoming connections and the admin service remain on (which means the config is not flushed out to the file) despite I ‘stopped’ the service. I agree with the original author that startup script is the right place to put the command to load the configuration (and therefore “add TAP interface to bridge” should immediately follow it after it finishes creating the TAP adapter).

As with what I learned from doing the same on Merlin WRT, add a few seconds (I used 3 seconds) of delay right after the config defining the TAP adapter is loaded so the OS got enough time to finish creating it before you try to add that TAP adapter to the bridge or else it’d fail silently.

Do it all in one page:

vpncmd localhost:5555 /SERVER /PASSWORD: /CMD ConfigSet //jffs//var//softethervpn//vpn_server.config
ln -s /jffs/var/softethervpn/vpn_server.config /tmp/vpn_server.config
sleep 3
brctl addif br0 tap_tap0

Loading

Installing SoftEther VPN Server in Asus Merlin WRT

I’m experimenting with SoftEther as the user experience/interface is a lot more polished than WireGuard and OpenVPN and it has wide platform coverages. The Windows administration interfaces are sensibly designed (organized conceptually), unlike the Linux software culture that basically pretend to have a user interface yet it’s just a step away from editing the raw config file. SoftEther’s documentation also has nice graphical illustrations and it’s use cases oriented. The best part of the docs is that they are short and to the point.

Here I have a use case that’s not as quite as common for SoftEther’s users, so I might as well do a quick write up so if I run into this again in the future, I don’t have to do the research again.

Use case: router doubling as VPN server

I made a diagram based on my current understanding of how ethernet router works, and what needs to be done to have the router dual as SoftEther VPN server.

No guarantee that it’s the correct model and the lingo, and I’d appreciate comments to help me improve as I’m still learning (I was in algorithms and non-networking software development, more on the DSP side + light embedded systems, so this area is new to me).

  • The part shaded in blue is the new part we are building.
  • Installing SoftEther from Entware is just the square block.
  • You need to modprobe tun to let the SoftEther Server Admin software create the TAP port (made up Ethernet card).
  • At the same time the TAP port is created, say tap0, what SoftEther ‘bridges‘ is NOT the LAN, but the orange link in the diagram that goes to the virtual hub (which belongs to a VPN server instance). This lingo confusion wasted me days.
  • The ‘bridge’ on SoftEther’s side tells the incoming VPN connection which ‘Ethernet card’ (turns out to the the TAP interface) on the host computer should act on its behalf.
  • I felt like something is odd that SoftEther did not ask me what local network should the TAP interface go into so I suspect the TAP is just sitting there not talking to anybody, and it turned out to be the reason why my incoming VPN connection succeed but I’m not getting DHCP assignments.
  • I bit the bullet and understand how a Linux router work as if it were a computer with 5 Ethernet cards and one important piece of the puzzle is that the 4 LAN ports aren’t directly talking to the the WAN, but instead they form a bridge (software switch) which the bridge represents them and talk to the processed WAN traffic.
  • So the missing link is the double-line on the diagram where I add the TAP interface to the LAN bridge, namely brctl addif br0 tap_tap0. Linux adds a tap_ prefix to tap interfaces so it’s tap_tap0 for tap0 in SoftEther.
  • One more non-obvious thing here is that you also need to register the brctl a few seconds (using sleep delay) right after the SoftEther VPN Server service starts and nowhere else. The TAP has to exist before you put it on the LAN bridge and the TAP is programmed correctly to be as short-lived as needed, which is very responsible.

What to watch out for this use case

SoftEther’s interface does support creating a TAP adapter, but it provides scary warnings as this is an unusual settings.

TAP depends on the TUN module being loaded first, but Merlin-WRT’s firmware do not load this out of the box.

Some other websites tells you to install packages ip-full (for ip command) and OpenVPN (for the TAP) adapter, but it’s not necessary in some newer releases of Merlin-WRT. It’s all there, just waiting for you to modprobe tun (load TUN/TAP kernel drivers) before you can create TAP adapters.

If you don’t have the TUN module loaded first, the newly created bridge will show ‘Error’ with no explanation, which is confusing.

I figured out this is the missing part that causes the Error status by learning how TAP interface are created on Linux and speculated the Windows remote server admin interface (Server Manager) calls this under the hood:

ip tuntap add dev {YOUR TAP DEVICE NAME GOES HERE} mode tap

and tried to imitate the call and researched the error messages.

The next hard part is that the TAP adapter created by SoftEther’s is not tied to anything in the router when freshly created by “Local Bridge Setting”! It’s like you just freshly added an extra network card into a computer with the drivers set up, it doesn’t interact with anything on your network before you plug a cable in the right port!

As the last step you will need to SSH into the router to put in brctl addif br0 tap_tap0.

Obvious preparations

  1. Prepare USB storage (format it with amtm) to host Entware if not already done
  2. install Entware (amtm has an installer for it)
  3. install softethervpn5-server through Entware (opkg install softethervpn5-server)

Enable TUN/TAP drivers

By default TUN/TAP kernel module is by default not loaded, so we somehow need to add modprobe tun to startup scripts.

Out of the box the router is read-only so you cannot get it to remember the startup scripts unless you turn on /jffs, a small (like 64MB) onboard non-volatile memory to store user data such as startup scripts.

After you turn on /jffs, you will see tapping points to the startup process provided by executable files located in /jffs/scripts:

If you haven’t installed anything that has written to services-start (the earliest point), you can install spdmerlin (from amtm), a tool that provides a customized router admin page that creates a dashboard with all admin goodies and it will create services-start and make it executable if it’s not already there for you to tap in the modprobe tun line.

If you want to do this yourself, make sure you spell ‘services’ with the plural ‘s’ (the pre-existing ‘service-event’ which the ‘service’ is singular might tempt you to imitate it, which is incorrect) and chmod +x services-start to make the script executable.

I use nano to sneak modprobe tun into /jffs/scripts/services-start (I also tried init-start and it works too since modprobe is very early kernel stuff). Do whatever that’s convenient for you as long as you can sneak modprobe tun in:

I recommend rebooting right away then run lsmod | grep tun to make sure the module is indeed loaded. If you can’t spare a reboot (which is like 5 minutes), you can simply run modprobe tun at the terminal right away and hope the startup script remembers to do it on the next reboot

Use SoftEther Server Manager to remotely configure the softethervpn5-server installed on the router

The server program on the router did not ask for a password, yet SoftEther asks for it. This UI design is actually a little confusing. Turns out you enter an empty password on the first access/run and the user interface will ask you to create a proper password (just like some routers’ admin pages do).

The first time you set it up, you will be greeted by a Wizard which I cannot find again. This wizard is equivalent to ‘Create a Virtual Hub’ -> [‘Manage Virtual Hub’ -> Add Users] -> Local Bridge Setting. However, you want to skip the last step (create bridge) in the wizard because the wizard version caters basic users and they don’t offer the option to make a TAP adapter for the bridge.

By exiting the wizard at the bridge creation step, you’ve created the ‘Virtual Hub’ (which SoftEther sees ‘Virtual Hub’ as an instance of VPN server which you can run in parallel. Confusing lingo for beginners, but it might be sensible with the logic of the architecture). Click on the ‘Local Bridge Setting’ to finish the step that was not done by the Wizard

Bridging is a matter of hardware, so it’s universal across all Virtual Hub (or VPN server instances). This is why it’s at the top level outside your VPN server instance (Virtual Hub) configs.

Softether is trying to be helpful but we know what we doing something unusual here (using the router itself as a computer that plugs into the router). Don’t get scared by the warning and just click Yes to continue

If you remember to start the TUN kernel module, the Status should turn from Error to Operating in a split second. If it stays in the Error, go back and check if you have TUN running properly.

Oversimplified view of LAN bridges

From an end-user perspective, a bridge can be thought of in terms of switches despite the order of evolution is the other way round.

You can think of a bridge as a switch where the computer that hosts it gets a free ride on the switch without the extra physical switch NIC port, physical ethernet cable, and physical device/computer NIC port. I called it ‘implied’ in the diagram on top of this post.

Say for example with 1 ethernet adapter computer A connects to the upstream (say Internet and home network managed by a router above) and let’s call it NIC-A. Then we install an extra network card/interface called NIC-B that’s for serving other devices.

By creating a bridge BR0 formed by NIC-A and NIC-B, you created an illusion of NIC-A behaving as two network cards NIC-A and NIC-B with 2 cables connected to the upstream LAN directly despite only 1 card (NIC-A) is physically there. So what NIC-A did in the bridge BR0 is it act as a software switch which it gets a free ride (implied) and the downstream Computer B rides on the switch that’s served by Computer A.

Add the TAP interface to the existing LAN bridge

brctl is the command line interface for managing bridges

addif adds an interface to the bridge. Here’s the manual page for the syntax:

In this forum post user miscell reversed the order and typed the interface first, which you’ll get a ioctl error complaining that you’re trying to write to an unwritable file.

However, since we are in a router, these changes won’t stick on reboot so we need to put this somewhere. Turns out it’s a colossal pain in the butt to figure this out because the tap0 adapter is correctly programmed to exist only when the SoftEther VPN server service is running and disappear when the service stops. In other words, the TAP adapters created and managed by SoftEther is ephemeral.

Since the TAP does not exist before the SoftEther VPN Server service (S05vpnserver) starts and vanished when the service stops, the ONLY place you should attach the bridging operation is within the start) section of /opt/etc/init.d/S05vpnserver, right after the core service completely finished starting so the TAP is fully created. I monitored the output of ifconfig and realize I need a few seconds of delay before adding the TAP interface to the bridge because the TAP bridge has to exist first. Add the highlighted line to the right place

with the chunk repeated here if your LAN bridge is called br0 and the TAP is called tap0 in SoftEther:

sleep 3
brctl addif br0 tap_tap0

I also tested it and it seems like the bridge association is removed when the TAP adapter was cleaned up when the service is stopped, so I didn’t bother to add the brctl delif in the stop) section.

/opt is actually points to your entware folder (I choose not to show the raw path because it contains my usb partition label which you’ll have to substitute your own) so the data is not volatile and it’s living in your USB entware storage. Basically the SoftEther VPN server registry lives in Entware’s /init.d as S05vpnserver.

Double check the naming on your router with say ifconfig instead of trusting the tap_ prefix which might not be universal across routers. Also check if your router’s LAN bridge is indeed named br0 and replace the interface names accordingly. You can also adapt this to other routers as long as you know where to sneak in the startup scripts


Bonus: Firewall instructions

The firewall rules in MerlinWRT just quit working so the table I entered doesn’t do anything when I turn the firewall on. It doesn’t seem like it’s placing firewall exceptions the way I intended.

There’s also another weird behavior that if the port is firewall blocked, the server admin program intermittently still connect but it connects to a blank state server (blank config). WTF!

You won’t run into these problems if the firewall is turned off, but if you want to keep the firewall on, here’s the SoftEther VPN Server Firewall instructions.

Suggestion to SoftEther: Add a LAN bridging UI to the TAP option

Since this is an unusual concept, I copied the diagram from 3.6 Local Bridges – SoftEther VPN Project and overlay it to illustrated the ‘Local (VPN) Bridge’ has nothing to do with your LAN bridge which is necessary for the TAP adapter to do anything useful.

Right now there’s too little help on this topic which SoftEther considers it as advanced. Turns out putting SoftEther on a router isn’t too uncommon of a thing to ask for once people find out that it’s not impossible.

It’d save us who want to put SoftEther on a Linux router a lot of grief if SoftEther has an extra UI section in the dialog with a pulldown menu that states what bridge it can optionally join:

This is better done inside SoftEther instead of outside it because the users do not have to anticipate the names of the TAP adapters administrators create in the UI. Don’t worry about this extra option of adding it to a LAN bridge could confuse new users, as the lack of such option is way more confusing because there’s a TAP adapter created just to not connect to anything and it shoves new users to a dead end!

In the worst case you can throw a dialog box when users choose a non-blank item from the bridge list saying that this is for advanced users and make sure you know what you are doing (it’d be helpful to remind this could be used for installing SoftEther server on a Linux router).

Extras (feel free to skip it): First-time Wizard

Under no circumstance you should pick one of the LAN ports like /eth0 to bridge. This made no sense (btw /eth0 is usually the WAN interface) and I tried it just out of curiosity and it bricked the router by boot loop (luckily there’s self-recovery to fresh state after a few crashes).

The wizard isn’t that useful as soon as you notice the so called ‘VPN server (instance)’ is called ‘Virtual Hub’ and the buttons on the screen make intuitive sense that requires little explanation.

Loading

Getting sound to work Ubuntu VM guests for Hyper-V

It’s frustrating that there are no simple packages provided by Microsoft or Ubuntu to get something as basic as sound working on Hyper-V. It’s nuts that we still have to deal with these kind of integration bullshit in 2023 when people are claiming that Linux is useable! At the time of writing, there are still way too many rough edges on Linux that are clearly not the users’ fault!

To understand why things are the way they are. We have to first understand that Hyper-V talks to the VM under RDP under the hood which has many advantages that the RDP provides. RDP is way less sluggish for the video and sound quality than the damned X11 server, which people spent little time to provide for Windows. MobaXterm is the only decent free X11 server for Windows in terms of user experience but it’s not the first one that populate search results.

In other words, we’ll need to set up Linux to stream the apps through RDP, not X11 to take advantage of Hyper-V manager’s interface (specifically vmconnect.exe under the hood). In Linux, this area is not maturely developed and the author of xrdp is not keen to make polished packages so most of the time we get pointed to the struggle of compiling the source code.

Given RDP is not native to Linux, xrdp did not rebuild the guts of X11 that maps natively to RDP. Instead it launches a bare minimal RDP session with nothing on it other than a basic client (sesman) that you can use it as a VNC client or X11 client that streams to the bare minimum RDP, so there’s an extra layer of indirection. You are not running RDP natively. That RDP session is for you to call X11.

Since X11 does not natively stream audio, you need to a sound server (over network) system that streams the audio to the bare minimal RDP layer of xrdp, then have the RDP layer stream/relay to RDP with pulseaudio-module-xrdp (which is a kernel package that you need to build from scratch at the time of writing since there’s no packages for it).

By the way, if you get this xrdp+pulseaudio ordeal working, you also get (x)RDP working for the linux even if you don’t use vmconnect to connect to it, because it’s RDP under the hood anyway.

I followed the messy (and often broken) instructions from multiple sources to build and install pulseaudio-module-xrdp, but it turned out this wasn’t enough. There’s no sound and the anticipated sound device didn’t show up in Ubuntu and all I see is a dummy sound driver.


Turns out there are many pieces of the puzzle scattered through different blogs and the blog either has typos, missing a key component, or the URLs changed so it’s broken. Here’s an overview of what you need to do

  • Replace Pipeware completely with PulseAudio on Ubuntu
  • RDP Enhanced Session is required for sound. This is true for Windows Guests as well. Linux doesn’t have RDP so you’ll need xrdp first before you even talk about enhanced session. Vsocket is how Enhanced Session talk. On Linux side we configure xrdp to talk on vsocket instead of a raw rdp protocol over Port 3389. On Windows we enable Enhanced Session (if not already) and enable hv-socket (with it the Windows side of vsocket).
  • Since xrdp ‘cheats’ by redirecting X11 instead of implementing RDP from the core, you’ll need to relay the pulseaudio from the Ubuntu itself to the RDP layer which is done by pulseaudio-module-xrdp. Unfortunately it’s does not come with xrdp and there’s no package so you have to build it yourself the install your compiled product. Remember by default source code repo is disabled so you need to enable it first before following the any build instructions.

Get rid of Pipeware Completely & Install PulseAudio

Griffon’s IT library provided the insight that we need to take out Pipeware (the competitior of PulseAudio) completely and replace it with PulseAudio. After that I got it working. Here’s a path to his tutorial:

XRDP – Bring back xRDP sound redirection on Ubuntu 22.10 – Griffon’s IT Library (c-nergy.be)

His tutorial included the script he made to install a more recent xrdp he built but the link is now broken. So what I did turned out to be necessary after all.

His tutorial basically stop/disabled/masked the hell out of Pipewire so it’s dead and deader and make sure the users cannot install it later and displace PulseAudio. It’s a lot of gymnastics because systemctl disable and mask do not take wildcards so you have to find out each service/daemon named pipewire. I’ll take this shortcut instead:

sudo apt purge pipewire

A tool to check if there’s audio server running (optional) is

pactl info

House keeping and install both pulseaudio and xrdp if not already done. pavucontrol is for controlling the volume which is often needed:

sudo apt update
sudo apt install pulseaudio pavucontrol xrdp

Then enable pulse audio and start it immediately (the –now switch follows enable with start) without rebooting:

systemctl --user enable --now pulseaudio.service pulseaudio.socket

Configure Ubuntu and xrdp for Enhanced Session (nothing to do with PulseAudio itself)

I got the clue from this blog: How to install Ubuntu 20.04 on Hyper-V with enhanced session | by Francesco Tonini | Medium, but some of the details changed as time moves on so I’ll document it here for the state of the art in 2023.

Part 1 [DONE BY DEFAULT NOW]: Change xrdp to receive vsocket instead of raw RDP

TLDR: No actionable item here. Included here for educational purposes only as it was a required step before.

Enhanced Session uses vsocket instead of raw Port 3389 for connection which xrdp.ini defaults to out of the box. Griffon’s IT library gave the instruction to replace the port=3389 to port=vsock://-1:3389 in xrdp.ini. However it’s already taken care of in the new install.sh script for linux-vm-tools:

The more recent xrdp already defaulted to use_vsock=false. Discussion: Cannot establish RDP connection to Ubuntu VM made with Hyper-V Quick Create · Issue #1260 · neutrinolabs/xrdp (github.com)

Part 2: install linux-vm-tools (think of it as vsocket driver)

linux-vm-tools is “Hyper-V Linux Guest VM Enhancements” developed by Microsoft which dropped support for it and picked up (forked) by hinara to support Ubuntu 22.04. Hinara’s version is more updated.

So the first thing you do is to download (can use wget, curl, aria whatever you like as the downloader) the install.sh of the latest/appropriate version (right now it’s Ubuntu 22.04):

wget https://raw.githubusercontent.com/Hinara/linux-vm-tools/ubuntu20-04/ubuntu/22.04/install.sh

This blog (Enable Sound Output Hyper-V (ubuntuforums.org))’s answer is a little outdated as involves a hack to rename 20.04 to 22.04 and keep the same script.

Note that the folder name at the top level says 20.04 (branch name) but at the lower level says 22.04 (folder name). So be prepared the path might change in the future if the author figured it’s better to use a consistent branch name later. The safest bet is to go to the Github page and discover the latest version then click raw to get the direct link to use with wget (or any file downloader).

Downloads by default are not executable for your safety, so enable the execute attribute with chmod +x:

sudo chmod +x install.sh

and of course execute the install.sh after you checked it’s all kosher:

sudo ./install.sh

The last 2 lines of the script tells you to RUN IT AGAIN AFTER REBOOT. It’s easy to overlook given the text doesn’t stand out after the user got bombarded with lots of verbose info. Make sure you follow it!

Configure Hyper-V for Enhanced Session (Host)

On Windows side, you’ll need to enable the Windows version of vsocket to communicate with the vsocket. There’s no curly braces when you type in {your VM’s name}:

Set-VM {your VM's name} -EnhancedSessionTransportType HvSocket

The chaos trying to build pulseaudio-module-xrdp

Deploy a Linux VM on Hyper-V with Sound 20.04 Edition – techbloggingfool.com‘s scripts do not use the default folder choices by the build script so the changes in the upstream broke his folder scheme by adding a ‘+dfsg1‘ to the folder name. Instead of fixing it, I followed the official instructions on the github page but take advantage of the one very useful piece he provided: enable source code repo:

He has command line scripts that enables the source code too. Given my confidence about linux developer coordinating with people downstream about the naming schemes changes, I’ll stick with the GUI which gives a consistent interface.

After source repos are allowed, get the build tools (official instructions for Ubuntu):

sudo apt install build-essential dpkg-dev libpulse-dev git autoconf libtool

Grab the pulseaudio-modle-xrdp source code and go into that folder

git clone https://github.com/neutrinolabs/pulseaudio-module-xrdp.git
cd pulseaudio-module-xrdp

Run the install_pulseaudio_sources_apt-wrapper.sh under the /scripts folder:

./scripts/install_pulseaudio_sources_apt_wrapper.sh

(I tried the non ‘_wrapper’ version next to it. It works too, but it doesn’t relieve you from the rest of the build and install process). This is taken straight from Build on Debian or Ubuntu · neutrinolabs/pulseaudio-module-xrdp Wiki (github.com):

./bootstrap && ./configure PULSE_DIR=$HOME/pulseaudio.src
make

PULSE_DIR=$HOME/pulseaudio.src because this is the default in the wrapper script above if no arguments are specified. Stick with the defaults (working at home folder) as it’s not wise to trust people to coordinate their naming scheme consistently. The defaults are likely tested more thoroughly.

And the last step is to install your hard work (README · neutrinolabs/pulseaudio-module-xrdp Wiki (github.com)):

sudo make install

which you can OPTIONALLY check your work to see if the kernel modules are indeed installed:

ls $(pkg-config --variable=modlibexecdir libpulse) | grep xrdp

[Optional Cleanup] If you don’t have anything named pulseaudio that you’d like to keep, you can clean up by removing the files using wildcard and the install.sh

sudo rm -rf ~/pulseaudio*
sudo rm ~/install.sh

Summary

Here’s a consolidated ‘script’ to show the complexity. I do not recommend copying and pasting it as the dependencies might change and it might break.

# Change audio server in Ubuntu
sudo apt -y update
sudo apt -y purge pipewire
sudo apt -y install pulseaudio pavucontrol xrdp
systemctl --user enable --now pulseaudio.service pulseaudio.socket

# [Optional] check if PulseAudio was installed correctly
pactl info

# Build pulseaudio-module-xrdp and install the kernel modules
# Stick with official instructions which assumes home folder
cd ~
sudo apt -y install build-essential dpkg-dev libpulse-dev git autoconf libtool
git clone https://github.com/neutrinolabs/pulseaudio-module-xrdp.git
cd pulseaudio-module-xrdp
./scripts/install_pulseaudio_sources_apt_wrapper.sh
./bootstrap && ./configure PULSE_DIR=$HOME/pulseaudio.src
make
sudo make install

# [Optional] check if pulseaudio-module-xrdp was installed correctly
ls $(pkg-config --variable=modlibexecdir libpulse) | grep xrdp

# Install linux-vm-tools which enables Enhanced Session
# Give linux-vm-tools its own folder to avoid confusion
mkdir -p ~/linux-vm-tools && cd $_
wget https://raw.githubusercontent.com/Hinara/linux-vm-tools/ubuntu20-04/ubuntu/22.04/install.sh
sudo chmod +x install.sh
sudo ./install.sh

# The last 2 lines of screen output of install.sh tells you to reboot and run this again
# This is automated below by making a icon in Gnome desktop's autostart folder
# that will self-destruct after first launch
cat > ~/.config/autostart/startonce.desktop <<EOF
[Desktop Entry]
Type=Application
Name=startonce.desktop
Exec=gnome-terminal -- sh -c '~/linux-vm-tools/install.sh && rm -rf ~/pulseaudio-module-xrdp ~/pulseaudio.src ~/linux-vm-tools ~/.config/autostart/startonce.desktop && init 0'
EOF

sudo reboot

I also noticed a quirk that the first reboot after everything’s installed might be a little too fast. In this case restart again once more. So instead I just have the desktop shortcut shutdown your VM after it’s done and have you manually start it again so it’d work right the first time.

I’ve created a Github repo for my own convenience, but feel free to use it however way you like, but I’m not responsible for any damages it might cause. Better read through the code with the help of this blog page and understand what it does and decide if you want to try it for your setup. I tried to make it robust, but it’s designed to be installed on freshly install Ubuntu guest VMs.

https://github.com/wonghoi/enhanced_session_linux/

Loading