dd-wrt web admin UI WTF!? Inconspicuous invalid config combinations crashes your router out of the box.

I was puzzled by why my dd-wrt router behave erratically each time I change the “Start IP Address” in DHCP leases to an upper range and I just figured out why.

I hate the user interface of dd-wrt with a passion, but it’s the only open source firmware for one of my routers that signed Broadcom’s close source NDA to get its fucking driver SDK so I’m stuck with it:

Ugly as fuck

In the bad old days people think it’s a good idea to make 4 little edit boxes for IP addresses than checking if the input conforms to the IP address format with dots. But it cannot detect ‘.’ keypress and jump to the next box (use Tab instead). e.g.

Inconsistent state possible

Start IP address, which is dependent on Local IP Address, is not updated/reflected until you press “Save”. This means every time you make a change, you need to hit “Save” immediately so other dependent settings will make sense before you start editing them.

Features are arranged/grouped like config files

This is bare minimum effort on UI dev, which is not much better than going to linux prompt and edit the config files.

With config files, at least we’d be more careful and try to understand what each key-value pair mean and their relationship map. This lousy web admin UI interface gives a false impression that non-developers knows what they are doing, so it turns into a puzzle that we’ll have to google the answer for every fucking basic application.

Using the web UI instead of editing the config files feels like programming in assembly as an improvement over programming in raw machine code. It’s begrudgingly painful.

One example is the grouping for wireless radio. For most considerate web admin interface, the SSID are grouped logically with your WiFi password, but in dd-wrt, you set SSID in “Basic Settings” and the WiFi password under “Wirelesss Security”. Make sense for the programmer to decouple the radio from the access control (group by features), but it’s not application/use case oriented (group by radio interface), thus it frustrates users.

Non-intuitive (less common) presentation

As described above, out of developer’s convenience, dd-wrt’s web admin UI just do everything that makes beginners’ life miserable or just throw them off. e.g. Windows users are used to specifying the subnet mask in quad-dotted notation like 255.255.255.0, not the CIDR notation like /24:

Confusing names

The names are often too terse that creates confusion with similar named features in a lot of places.
e.g. “Wireless GUI Access” does not mean the welcome page for your Guest network, but whether the wireless client have access to the Router Administration‘s Web UI!

It’s probably a few minutes of extra thought to call it “Allow admin web UI: Yes/No”

Another example is AP Isolation, which is under Advanced settings tab for each radio:

Is this isolating APs in a mesh or isolating clients connected to the AP from each other? Turns out it’s the latter! Just say “(For this AP), allow connected wireless clients to talk to each other on the same network: Yes/No”. I think it’s a common use scenario that the regular users should be aware of and shouldn’t be stowed away with obscure radio/PHY-level tweaks (settings aimed for hackers).

Overloaded names

The admin web UI is littered with overloaded names/terms which means something completely different depending on context (like the settings 2 lines above). For example:

Image:Access point.jpg
Access Point (AP) mode
The SSID here is the SSID of the Access Point
(Wifi host. AP station accepting wirelesss clients)
Client/Repeater[Bridge] Mode
(all involves the dd-wrt router connecting to certain AP station/SSID)
The SSID here is the SSID where the Client/Repeater-Bridge/Client-Bridge is attempting to connect to that contains the uplink/WAN!

WTF?! The dialog box looks exactly the same despite the wifi section is acting like a completely different device! What the fuck is “Network Configuration: Unbridged” for a Client Bridge?! HELP!

Unchecked invalid combinations that crashes!

This is the most frustrating behavior and should be considered a bug. I wasted days resetting my router over and over and it keeps hanging randomly after I change the DHCP server’s starting IP address. This is the default out of the box settings:

Default DHCP settings. End IP Address is 192.168.1.253

End IP Address = Start IP Address + Maximum DHCP Users – 1. They probably chose this number to reserve 192.168.1.254 for static IP (like some admin page of other devices). 255 is broadcast IP so of course it shouldn’t be assigned

Moving the the “Start IP Address” up without adjusting “Maximum DHCP Users” accordingly will make your router behave erratically because the DHCP will try to lease IP out of range!

This will corrupt your router’s memory!

The End IP Address is displayed on Status -> LAN -> Dynamic Host Configuration Protocol -> DHCP Status. And here’s the WTF moment:

The End IP Address is now in a different subnet from the Start IP Address!
I’m using /24 so 192.168.1.X is different from 192.168.2.X.
From the web page, the attribute name is “dhcp_num”
This is the code that shows the derived ‘End IP Address” shown above

I don’t know how it is coded, but if this number is computed in a low level way, chances are it’ll write garbage to the memory (for example if the number is used as an array index). I think normally it’s checked in any not so shitty user interface so the invalid state/condition won’t propagate down the code and hang the router. But in my case it did. If I just reboot the router without resetting to the defaults, it’d just hang again after a few interactions (like moving between a few pages or applying a setting).

In any case, this check must be done at user level as even if the low level code say, quietly sets a valid default value when an invalid range was ‘entered’, it’d only surprise the user and make it even harder to troubleshoot. This UI bug is even less excusable as it’s more natural to have users enter the (Start, End) instead of (Start, # of slots). Probably takes half an hour more in coding to layout the UI code to enter ranges (add 4 boxes for quad-dotted notation for End IP and check them instead of just 1 box for # of DHCP leases), but making the user to do mental gymnastics and punish them by if they did it wrong is just outright terrible.

FreshTomato has quite a bit of learning curve, but at least it try to do something that’s sensible for users for common scenarios instead of sticking strictly to how the code/config files are written at developer’s level.

I seriously thought of tossing my router that only have dd-wrt as the only fully functional open source 3rd party firmware and find one that works with FreshTomato/OpenWRT/Merlin or the like. DD-WRT is powerful but the UI suck big time, not because the features overwhelm less tech savvy users, but it’s purely unnecessary torture and pain even you know why it’s done that way.

 36 total views

RS-232 Stop Bits in Agilent Instruments

Turns out Agilent instruments do not use the same defaults for the RS-232 in their instruments.

54600 series uses 1 stop bits (most common):

RS-232 modules used in old 54600 series
54620/54640 series (newer 54600 series)

However other bench instruments such as power supplies (E3640 series 663X series) and 33120A arbitrary waveform generator uses 2 stop bits (fixed regardless of parity), which is usually NOT THE DEFAULT for most terminal clients:

E3640 series and 33120A’s RS-232 configuration.
Parity only trade away one data bit, so it does not affect stop bit
663X series powers supplies’ programming manual aren’t explicit about that except in code example

 43 total views

RS-232 motherboard header mapping (DB9)

The pin ordering for RS-232 (DB9) pin is sequential is row-wise (the long side is a row) while the IDC-10 (ribbon cable) header is column-wise (zig-zag pattern).

This might be a little confusing because geometrically, they are in-place on both sides (you can overlay the pins of DB9 on top of IDC-10 and they align perfectly, except pin 6-9 was lowered by half a notch on the DB9 side). I am writing this post so nobody waste their time separating the wires in a ribbon just to find out the DB9 was designed so it aligns with the flat ribbon cable perfectly.

Here’s a great tutorial building your own RS-232 cable which I took the images from to illustrate the point. Please pay them a visit to show some love: https://developer.toradex.com/knowledge-base/assembling-serial-idc-to-db9-cable

If you split the table on the right in half (cut after pin 5) and place pins 6 (DSR) ~ 9 (RI) on the right, you’ll see it align with the IDC10

1DCD (Data Carrier Detect)
Check if connection dropped
DSR (Data Set Ready)
DTR-DSR Handshaking
6
2RxD (Receive Data)
RTS (Request to Send)
RTS-CTS Flow control
7
3TxD (Transmit Data)
CTS (Clear to Send)
RTS-CTS Flow control
8
4DTR (Data Terminal Ready
DTR-DSR Handshaking
RI (Ring Indicator)
For phone rings
9
5GND (Ground pin)– (Not connected)10
Rearranged DB9M RS-232 to align with IDC10

Cable-tester.com has a clearer annotated picture that matches the physical mapping above:

http://www.cable-tester.com/rs232-pin-out/

Note that the tutorial itself has Tx(D) and Rx(D) reserved it was building a null modem cable and they skipped all the handshake lines. I’m doing a straight cable (which should be done for internal board header cables where the DB9 socket is male, hence DB9M).

The DTE/DCE might be confusing. Hope these properties can help people make sense out of it (so you can figure it out in your head confidently instead of randomly trying null modem adapters till it work)

  • DTE device (colloquially ‘computer’) has the pattern as shown in the pictures above (receive pins above/before the transmit pins), which is usually the computer end and the port/socket is male. Think of it as the ‘driver/master’ (though it’s arbitrary)
  • DCE device (colloquially ‘modem’) reverses all the sends and receives of the DTE. Can think of it as the ‘receiver/slave’ (though it’s arbitrary). It’s usually the modem and the port/socket is female.
  • For DTE-DTE (like data transfer between two PCs), the send lines on one side should go to the receive lines on the other side. A null modem cable that swaps the send pins with receive pins. You can think of it as making one side DCE. Given that the topology is symmetric, it’s up to the software set up to decide which side is the initiator/client (master) and which side is the reactor/server (slave)
  • The handshaking (optional) and flow control (optional) lines also have their initiator/reactor roles reversed with null modem cable.
http://www.lm-technologies.com/wp-content/downloads/wireless%20adapters/LM048/Application%20Notes/DTR-DSR-RTS-CTS_Difference.pdf

Here’s an image for the above mapping:

Puertos E/S COM
https://itigic.com/input-and-output-ports-on-pc-types-uses-and-characteristics/

It turns out the direct geometric mapping (IDC male pins match the relative locations of the DB9 male pins) mentioned above is the less common type of motherboard header configuration. IDC ribbon crimp-on DB9 headers like this:

ฅนบ้ายอ: [34+] Db9 Male And Female Connector Pinout

has to follow the above geometric layout since the pins cannot be remapped (so it has to follow the ribbon order). The soldered version looks like this:

https://store.cwc-group.com/lowprldb9mat1.html

However, the more commonly seen soldered RS-232M to IDC10 header uses a transposed configuration (which DB-9M pin numbers matches IDC-10 pin numbering EXACTLY despite one is row-major and the other is column-major), which has nothing to do with the IDC10 pin layout mentioned above.

https://store.cwc-group.com/lowprldb9mat.html
(I don’t think it’s a good idea to call it ‘crossed-config’ like CWC did.
It almost mislead me to think it happens to swap the roles of Tx and Rx.
I did the mapping on paper and it didn’t make any sense.
Let’s call it ‘transposed-configuration’)

The crazy thing about the existence of these 2 pin layouts is that there’s no easy way to tell which pin layout/mapping it is until you open the connector up and inspect the solder joints! Taking a pin and probe it with a multimeter is more work than taking the screw posts out and disassemble the connector.

So if you just buy some old scrap parts that came with old motherboards, this might confuse the heck out of you until you tested the pin mapping with a multimeter and realize things doesn’t add up!

Note to self: just open the DB9 side up whenever I see a DB9-IDC cable and mark the configuration on the DB9 end directly on the cable!

 43 total views

Freshtomato for Netgear (Nighthawk) R7000

Netgear R7000 supports these major forms of firmware

  • DD-WRT (Powerful, but very messy web interface that are sometimes non-intuitively organized)
  • FreshTomato (Powerful. I wouldn’t say easy to use but mortal souls can understand it)
  • XWRT-Vortex (Easy to use AsusWRT Merlin web interface adapted for non-Asus routers)

There are other forms of Tomato that support R7000 but only FreshTomato is actively maintained as of late 2021.

However updating it from stock firmware to FreshTomato has some model-specific quirks (that you cannot extrapolate from general procedures for other models)

First of all. You cannot update directly to the latest firmware. There’s a bootstrap (intermediate) firmware called INITIAL (usually downloaded from ‘Netgear R-series initial files’ folder) that must be installed (upgraded from stock firmware to) FIRST so the router is ready to accept the latest/full firmware.

Here’s the model specific quirk: the default login/password is non-standard for R7000! It’s not root/admin (unless you press the button to reset the NVRAM)! It’s admin/@newdig!

After logging into the bootstrap/INITIAL freshtomato with the password above, upgrade the firmware to the latest (the one intended) and choose clear NVRAM along the way. The default login/password will be root/admin as standard for freshtomato.


There’s another twist for SSH connections! The username in your web admin interface do NOT matter! The username is ‘root’ for SSH regardless of what you set in the web interface, your password is the one entered in web admin interface! This is super counterintuitive!

 49 total views

Evoluent Vertical Mouse 4 Cable Mod

The mouse cable for Evoluent Vertical Mouse 4 is extremely long, which creates a lot of clutters especially when my keyboard has a USB hub relay built in (it’s the mouse is less than a feet away from it). Instead of splicing the cable, which creates a hard junction that’s not flexible, I modified the mouse to take a micro-USB cable instead.

 53 total views

Flashing router firmware through Serial Port: CFE bootloader (usually Broadcom) based routers

Here’s a summary of learnings from dd-wrt’s serial recovery instructions:

  1. Use a UART controller that signals at 3.3V (e.g. FTDI TTL-232R-3V3) to talk to the board. Regular serial RS-232 ports requires a voltage level shifter converting the signal to swing between 0V to 3.3V.
  2. You only need 3 pins: Tx, Rx and Ground. There’s voltage contention if you plug in the Vcc from TTL-232R-3V3 (It’s the USB’s 5V despite the signaling is 3.3V) to the 3.3V supply of the router. You don’t need the Vcc pin. It didn’t harm anything or do anything when I connected the Vcc pin.
  3. Stick with all serial port defaults and only set the baud to 115200 (default is 9600) and turn off flow control (default is xon/xoff). I use Putty for terminal.
  4. The terminal serves as the monitor for the computer on the router that shows a text console. Broadcom uses CFE bootloader (others use U-Boot with busybox).
  5. CFE bootloader defaults to 192.168.1.1 with subnet mask 255.255.255.0 (aka /24). Set up the network interface to have a static IP on the same subnet to talk to the board.
  6. Good habit: nvram erase
  7. The flash program relies on TFTP protocol to receive the firmware file. So get your TFTP client ready. Microsoft included a TFTP client/server since Windows 7 but usually disabled (turn it on in Windows OptionalFeatures.exe).
  8. TFTP is a simple push(put)/pull(get) design. You can either “push a file on your computer” or “get a file as filename”. You’d want to specify -i switch (binary image transfer) with Windows tftp.exe.
  9. So type this command at the command prompt but don not press enter until your router is ready to grab the file: tftp -i 192.168.1.1 put {path to whatever TRX firmware file}
  10. Go back to the serial terminal and tell the router to accept a TFTP push (in a window of a few seconds before it time out) and flash the memory region flash1.trx with this command: flash -ctheader : flash1.trx
  11. Immediately initiate the TFTP push from your computer (Windows command line example in Step #9 above)
  12. Wait for a couple of hours! The terminal might tell you that it has received the file completely, but it won’t show anything when it’s writing to the flash! It’s a painfully slow process with no feedback. Just be patient!

Some observations

  • FreshTomato firmware absolutely won’t tell you on the screen after it has done flashing (Merlin-WRT does). Just turn the router back on after a couple of hours.
  • Merlin firmware repeats (exposes) the raw passwords to the serial port!
  • FreshTomato firmware boots to a linux prompt on the serial port

 77 total views

Asus-wrt Merlin Firmware DDNS update interval hack

The “WAN – DDNS” page only allows users to set the DDNS updater to check as frequently as every 30 minutes. My DDNS provider does not have an update frequency limit, so I’d like to have the update client check for every 1 minute. The setting is called “Verify every”:

Attempting to set it to every 1 minute gives this error message:

I searched for the “WAN-DDNS” config webpage file (Advanced_ASUSDDNS_Content.asp) in the firmware source code, and found that it’s under /www folder in the router’s linux root.

Since “Verify every” is such generic words, and Github does not support exact phrase match in search (I use “in:file” specifier in the search box), I pick “WAN IP and hostname verification” (the closest setting which I expect the code to be in the proximity of the one corresponding to “Verify every”) so it has more unique keywords. The first jump:

Since it’s just a dictionary files, we search for the associated internal variable name
DDNS_verification_enable” which points to the this line in Advanced_ASUSDDNS_Content.asp:

Since this name appeared nowhere else, I traced the “id” attribute above, which is “check_ddns_field” and I see a Javascript (.js) file that process the data from the web page forms:

The variable check_ddns_field appears in the if-else-if branches of change_ddns_settings(), so one of the few next few variables after it is likely to correspond to “Verify with”.

The variable name showed up in 4 branches of if-elseif-else switches (switching DDNS service providers), which ddns_regular_period comes right after

Searching for the class member (or struct field)

Bingo. Here’s the entry value range check code. I’ll change the “30” minutes to “1” minute to enable checking at 1 minute intervals (which I think it’s reasonably responsive for testing and general use).


I’d prefer to check if the input range check is there out of feasibility (i.e. what is the smallest increment) or it’s just set to prevent people from getting banned by the DDNS provider for checking too frequently. I looked into the last occurrence of ddns_regular_period and found this:

Which means the web forms is updating NVRAM (environmental) variable of the same name ddns_regular_period, which appears to be called only in watchdog.c:

And Dang! The code enforces if the ddns_regular_period (on NVRAM) is set to be less than the original 30 minute minimum (invalid condition), it’d be set to the default 60 minutes (1 hr).

It’s actually sloppy coding because the defaults are specified in struct fields in defaults.c:

yet that 60 minutes is hard-coded in watchdog.c. That means if I don’t catch it and only changed the default in one place, the behavior will not be what I expected given the right conditions. This is an example of why software feature expansion are likely to break things. If you have solid code, bugs on updates are likely to happen.

I was curious why it says (period*2)

and suspected the ddns_check_count is incremented in 30 second (half-minute) interval. Since it’s watchdog.c, my natural guess is that the watchdog checks every 30 seconds for these event hooks. Turns out the notes (comments) in the code has “30 seconds periods” noted everywhere.

I searched a little bit more about linux watchdogs and found this useful webpage which explained how it works. I didn’t see /dev/watchdog in my router’s rootfs (root file system) so I assumed it’s a hardware watchdog (embedded linux, so duh).

I was about to dig up the hardware manual for the chipset for my router, but I search for they string HW_RTC_WATCHDOG  first and it showed up in linux kernel code (duh):

Note that the HW_RTC_WATCHDOG is a register in this code base, not the number of seconds from Christian’s Blog. i.e. they are completely different things, but it provided a good keyword lead for me to start digging.

The code seems to be the same for various kernel version so I picked any one of them to understand the behavior. First occurrence is in wdt_enable():

The other places are suspend/resume, so I’ll ignore those for now. Note that wdt_enable() is a static function, so only need to search within the same file. The only active place that calls it is wdt_ping():

So there are only 2 things I’ll need to find out: heartbeat and WDOG_COUNTER_RATE:

…. unfinished

https://bitsum.com/firmware_mod_kit.htm


While it’s a lot of useful learning about Embedded Linux and hunting down source code, for the meantime, given that namecheap does not care if you blindly update every minute, it’s easier to just set up a cron job that runs at every N minutes using curl/wget.

dd-wrt has a place for you to enter the cron scripts with the web interface, but you might need to log into the router using SSH and register the cron job yourself:

The core command is called ‘cru‘, which typing it in the command prompt will show you the very simple usage:


admin@Router:/tmp/home/root# cru

Cron Utility
add:    cru a <unique id> <"min hour day month week command">
delete: cru d <unique id>
list:   cru l

<unique id> is just a tag that you make up to name your task. Again the one-liner command needs to be direct absolute path. My ‘curl‘ program is located in /usr/sbin, so the command is:

cru a ncddns * * * * * /usr/sbin/curl "https://dynamicdns.park-your-domain.com/update?host={subdomain or @ for root}&domain={registered domain name}&password={DDNS-specific password generated by namecheap's domain administration page under Advanced DNS}"

The “* * * * *” refers to run at every “minute, hour, date of month, month, date of week”, in other words, run at every minute in every waking moment. The wild card * means ALL-OF.

Cron job registration through CRU is not persistent, so to make it survive reboots, add the above cru command as a line to /jffs/scripts/services-start script. It should be executable by default, if not, make sure you set it to be executable or it won’t run.

 70 total views

Namecheap Dynamic DNS setup in dd-wrt

Namecheap support page explained the process of configuring your dd-wrt router firmware to use Namecheap’s REST (HTTP URL) update interface to dynamically update the IP of your (sub-)domain. The instruction works, but there are few items which doesn’t quite make sense to me as a programmer, and I did a few experiments to figured that it’s bogus and developed a few insights about what’s necessary and why they do it.

Their instructions looks like this:

and specific verbal instructions are:

  • DDNS Service: Custom
  • DYNDNS Server: dynamicdns.park-your-domain.com – the name of the server should not be changed
  • Username: yourdomain.com – replace it with your domain name
  • Password: Dynamic DNS password for your domain (Domain List >> click on the Manage button next to the domain >> the Advanced DNS tab >> Dynamic DNS)
  • Hostname: Your subdomain (@ for yourdomain.comwww for www.yourdomain.com, etc.)
  • URL/update?domain=yourdomain.com&password=DynamicDNSPassword&host=

I stroke out Username and Password fields because they are not used!

If you look at the URL, namecheap’s instructions are asking you to re-enter the domain and the password key-value pair AGAIN, which means Username and Password fields are not used.

My programmer instinct immediate screams the updater is assuming certain REST API syntax that are not properly substituted so they need to be entered manually, exposing the password without the benefit of the masks (forget about keeping the password top secret, router firmware guys aren’t top security engineers. Just re-generate one in Namecheap’s admin interface if it gets compromised).

I checked by entering bogus Username and Password fields (the web’s GUI/forms checks if they are blank, so you can’t get away with not entering). It worked as expected. This means the two fields are dummies with Namecheap’s instructions.

Based on the fact that Namecheap’s instructions being unable to substitute Username and Password fields and the host key must be put at the end for Hostname field to substitute correctly, I can safely speculate that the one who wrote this couldn’t find out what the syntax for the variables are, and exploit that the last parameter hostname gets attached at the end in the absence of substitution variables in the URL syntax.

Apparent people are doing something stupid like this because nobody in the chain remember to document the substitution variable names! It’s not in dd-wrt’s user interface (should have that printed the ‘usage’ info next to the URL box) and neither it’s in INADYN’s github readmes!

I decided to dig deeper and go after the dynamic DNS updater package in question. dd-wrt is using inadyn package to do the dynamic DNS update, as “INADYN” is shown in “DDNS status” box gives it away (confirmed by dd-wrt’s docs):

The service itself is called ddns though.

I ended up reading the /examples folder on the repository and found this:

Bingo! Here it is:

URL substitution stringdd-wrt fieldsNamecheap REST key
%uUsernamedomain
%pPasswordpassword
%hHostnamehost

generic.c is a plugin also shows the above table as well


Since namecheap’s dynamic DNS service do not mandate how frequently you can update nor they charge per update, it’s easiest and most reliable to just blindly update the IP every N minutes instead of checking against a local cache to see if the external IP has really changed before updating at each poll interval

This user interface does not have the option to set the updater to run every 1 minute, so why bother since it’s just a simple program that creates a simple URL and do a curl/wget? At the end of the day, I decided to just do a cron job:

PATH=/sbin:/usr/sbin:/bin:/usr/bin
* * * * * root curl "https://dynamicdns.park-your-domain.com/update?host={subdomain or @}&domain={domain name bought}&password={generated by namecheap's account management page}"

There are 3 things that you will need to know:

  • Paths is from a clean slate. Need to define it first
  • * * * * * means every minute. Specify numbers/range for each time unit (minute, hour, day of month, month, day of week) if desired. Asterisk means ON EVERY.
  • Need to specify the user after the time syntax and before the actual command

 68 total views

dd-wrt gotchas

dd-wrt is very a powerful firmware compared to ASUS Merlin, but the UI leaves a lot to be desired. It’s very close to editing a config file and there’s little help to what each setting. The developers of dd-wrt didn’t invest time in designing the web administration interface and used the most basic primitive HTML forms so there are no tooltip that explains the features and the interaction between different settings.

There are also some confusing (nonsensical) UI design that are a lot less work to the developer but confused users to no end. Here are the examples I’ve found so far:

  • Enabling remote admin through SSH (for embedded linux command prompt) is a two step process out of the box. You’ll need to first enable SSHd from Services -> Secure Shell before enabling SSH Management from Administration -> Management (otherwise it’s greyed out)
  • The router username (user modifiable) for dd-wrt applies to web UI only. SSH’s username remains root. They share the same password though (so login and password are decoupled in dd-wrt, they are effectively two passwords in practice except they don’t put asterisk over the username as you type). ASUS Merlin firmware’s login is consistent across both web page and SSH
  • Cron jobs is from a bare environment which means you need to manually define the paths and specify the user in the cron job syntax. e.g.
    PATH=/sbin:/usr/sbin:/bin:/usr/bin
    * * * * * root {command_to_execute}

 63 total views