MATLAB assign index for unique rows

MATLAB’s dataset/table objects’ internals often involves identifying unique contents and assigning a unique (grouping) index to it so the indices can be mapped or joined without actually going through the contents of each row.

In the old days when I were using dataset(), the first generation of table() objects before the rewrite, there is a tool called grp2idx() which assigns the same number to identical items regardless of data types. It was part of Statistics Toolbox (needs to pay extra for it) and it does not work if you have multiple columns that you want to assign an unique index unless the ROWS are identical.

Upon inspection. grp2idx() is overrated. There are two ways to get it without paying for the toolbox:

  • double(categorical(X)): cast a categorical type (technically you can use nominal/ordinal, but it’s part of statistics toolbox)
  • Use the 2nd output argument for sort() or sortrows() function. I recommend sortrows() because it’s can be overloaded on table() objects and it works on multiple rows.

Loading

DBeaver connecting to MySQL in Namecheap Shared Hosting

Namecheap already provided instructions to connect MySQL Workbench client for its shared hosting, which involves SSH-tunneling because they disallowed direct MySQL connection out of security concerns.

So here’s basically the logistics:

  1. SSH to your namecheap hostname (can use your domain name) at SSH port 21098
  2. Tunnel listens to Port 5522 and forward it to localhost (the client itself) at MySQL Port 3306
  3. Instead of connecting directly to the {namecheap shared hosting server}:3306, connect to the localhost:3306

It’s a little confusing on how to do it on DBeaver because “Advanced settings” is hidden by default which you will need. The name ‘local client’ (source) vs ‘remote’ (destination) in the dialog box is confusing. It’s actually equivalent to

ssh -L ["Local host":]"Local port":"Remote host":"Remote port"
ssh -L [bind_address:]port:host:hostport

bind_address can be left blank. If you are paranoid and don’t want other machines to use your current MySQL client machine as a gateway (they tunnel into your machine to use the tunnel you are currently establishing), set (aka bind) it to localhost, or you can bind it to the client’s network adapter’s IP which you want to allow machines on a trusted network to use this MySQL client computer as a gateway.

For some reason (I suspect it’s IPv6), “Remote host” needs to be set to the loopback adapter 127.0.0.1 (cannot use the special hostname ‘localhost‘).

Remember MySQL’s username and password is the special database-only login credentials you created at cPanel.

Loading

Text manipulation idioms in linux

awk: select columns
sed: stream editor (operations like select, substitute, add/delete lines, modify)
sed expressions can be separated by ";"
sed can substitute all occurrences with 'g' modified at the end: 's/(find)/(replace)/g'

# https://unix.stackexchange.com/questions/92187/setting-ifs-for-a-single-statement

# arg I/O
$@: unpack all input args
$*: join all inputs as ONE arg, separated by FIRST character of IFS (empty space if unspecified)

# Remember the double quotes around "$*" or "$array[*]" usages or else IFS won't function

array[@]: entire array
${array[@]}: unpacks entire array into MULTIPLE arguments
${array[*]}: join entire array into ONE argument separated by FIRST character of IFS (defaults to an empty space if unspecified)
( IFS=$'\n'; echo "${my_array[*]}" )

${#str}: length of string
${#array[@]}: length of array
${#array[@]:start:after_stop}: select array[start] ... array[after_stop-1]

${str:="my_string"}: initializes variable str with "my_string" (useful for side-effect)

$(str##my_pattern}: delete front matching my_pattern
${str%%my_pattern}: deletes tail matching my_pattern (can use one % instead)
$(str%?}: delete last character (the my_pattern is a single character wildcard "?")

$( whatever_command ): captures stdout created by running whatever_command
( $str ): tokenize to string array, governed by IFS (specify delimiter)
( $( whatever_command ) ): combines the two operations above: capture stdout from command and tokenize the results

# https://unix.stackexchange.com/questions/92187/setting-ifs-for-a-single-statement
function strjoin { local IFS="$1"; shift; echo "$*"; }

Loading

Improved code for Toner Reset SP C250SF/DN

This is based on the Raspberry Pi implementation of the Toner chip reset:

https://gist.github.com/joeljacobs/c57550cdb4e68e3b86d6b89fb58f305d

I am using a Raspberry Pi Zero W so the chip is BCM2835 instead and I can use 100Kbps/400KBps instead of 9600 baud as in the original code

The electrical pins we need is clustered on to top left, Pins 1 (3.3V), 3 (I2C SDATA), 5 (I2C SCLK), 9 (Ground)

Raspberry Pi Zero GPIO Pinout, Specifications and Programming language

While looking for the pinouts (https://pinout.xyz/pinout/i2c), I discovered a useful tool called i2cdetect that allows me to find out the address of the chips which means I can write a program automatically figure out the right image to load to the chip without looking:

sudo apt-get install i2c-tools
sudo i2cdetect -y 1
Sorry I forgot where I got this image from.
Please remind me in the comments section if you find out who should I credit it to.

Since I don’t have cheap pogo pins lying around, I took the 2.4mm pitch (the standard size used in PC, Arduino and Raspberry Pi) jumper block I have (so all pins are set at equal lengths to make simultaneous contact) and hope somehow there’s 4 pins that kind of align with the contact, and it did. See pictures here:

Can press the pins down by using jumpers

You might be worried about shorting into the next pin or hooking something up in reverse damaging the chips, but luckily the chips survived. My guess is that it’s a good design to put the Vcc next to Ground on one side instead of making it symmetric so the polarity can be reversed. When reversed, SCL is set to low (Ground), SDA is pulled up to Vcc while there is no power supply, so no damage is done. Brilliant! The worst case for my poorly aligned jumper block is that SDA and Vcc might touch each other, but it doesn’t matter because it’s a perfectly legal hookup (just not communicating)!

So no worries if you didn’t touch the pins right! The only case it might go wrong is if you intentionally flip the block and slide it by two pins (reversing Vcc and Ground). Other cases (you are likely going to run into) are pretty much data lines getting hooked high or low levels while power lines not getting any supplies.

I’ve designed the program that it’ll detect the chip if you hook it up right and immediately program the chip (takes only a second), so you don’t have to hold the jumper for too long to worry about unstable contacts.

#!/bin/bash

# This program detects rewrite the toner chips to "full" for a Ricoh SP C250SF/DN Printer using Raspberry PI (defaults to BCM2835 models such as Raspberry PI Zero)

# The chip data is in file named "black" "cyan" "magenta" and "yellow".
# The pad closest to the edge is GND (-> Pin 9), followed by VCC (-> Pin 1) , DATA (-> Pin 3), and Clock (-> Pin 5).

# Be sure i2c is enabled and installed (it's turned off by default) on Raspbian

# This line is disabled because it takes too long to unregister i2c_bcm2835 to start from a clean slate
# modprobe -r i2c_bcm2835

# This program needs sudo permissions as it involves direct access to hardware
if [[ $EUID -ne 0 ]]; then
  echo "Need root permissions as this code involves direct hardware access. Try sudo"
  exit 2
fi

# Sets the baud rate
modprobe i2c_bcm2835 baudrate=400000

# Install i2cdetect if not exist (This idiom uses short-circuit OR for conditional exec)
command -v i2cdetect > /dev/null 2>&1 || {sudo apt-get install i2c-tools}

# Create I2C address to color map
COLORS=( [50]="yellow" [51]="magenta" [52]="cyan" [53]="black" )
# Detect chip I2C address
I2C_address=$( sudo i2cdetect -y 1 | grep 50 | sed -e 's/50: //;s/-- //g' )
# Keep the 0x5* address lines since only 0x50~0x53 is valid. Strip the 50: header, discard all "--" entries, and you are left with the detected address
HEX_I2C_address="0x$I2C_address"

# LED flash function(s)
target_device="/sys/class/leds/led0/brightness"
# This accounts for breaking changes (by renaming led0 to ACT) from Bookworm
[ -f ${target_device} ] || target_device="/sys/class/leds/ACT/brightness"

function turn_off {
  echo 0 > ${target_device}
}
function turn_on {
  echo 1 > ${target_device}
}
function toggle {
  # PI Zero only has 1 power LED which is on/off 
  # Reads 0 for OFF and 255 for ON
  readin=$(cat ${target_device})
  # Condition 255 to 1 to reduce platform dependency
  state=$[$readin>0]
  # Not equal to 1 is XOR (flip)
  flip=$[$state != 1]
  # Display output
  echo $flip > ${target_device}
}

function flash_once {
  period=${1:-0.5}

  toggle
  sleep $period

  toggle
  sleep $period
}

function flash {
  times=${1:-1}
  period=$2
  for((i=1; i<=times; i++)); do
    flash_once $period
  done
}

echo "Detecting toner chip ..."
turn_off
if [ -v COLORS[I2C_address] ]; then
  # Meat
  color=${COLORS[I2C_address]}
  echo "Detected toner chip for color: $color"

   # "address" counter sync up with the hex code index in file
   printf "Writing"
   address=0;
   datafile="${BASH_SOURCE%/*}/${color}"
   for i in $(cat $datafile); do
     i2cset -y 1 ${HEX_I2C_address} $address $i; || break
     address=$(($address +1));
     printf .
     flash 1 0
   done
   echo "Done!"
   turn_on
else
  echo "Invalid I2C address for SP C250DN/SF toner chips. ${I2C_address}"
  turn_on
  exit 4
fi

Turns out Linux’s dot sourcing (bash calling) syntax is relative to your current folder, not the scripts’ folder. So if you have one script calling another in the same folder when you are not currently at the same folder as the script (e.g. startup automation), the relative paths will be wrong. So instead of ./ , you should use this prefix instead:

${BASH_SOURCE%/*}/

I chose to flash the board’s only LED light quickly before starting and blink slowly a few times after it’s done for visual clues. It’s entirely optional. Here’s the guts of the code without the fancy indicators:

#!/bin/bash

# Sets the baud rate
modprobe i2c_bcm2835 baudrate=400000

# Create I2C address to color map
COLORS=( [50]="yellow" [51]="magenta" [52]="cyan" [53]="black" )

# Detect chip I2C address
I2C_address=$( sudo i2cdetect -y 1 | grep 50 | sed -e 's/50: //;s/-- //g' )
HEX_I2C_address="0x$I2C_address"

if [ -v COLORS[I2C_address] ]; then
  # Meat
  color=${COLORS[I2C_address]}

  # "address" counter sync up with the hex code index in file
  address=0;
  for i in $(cat ${color}); do
    i2cset -y 1 ${HEX_I2C_address} $address $i || break
    address=$(($address +1));
  done
else
  echo "Invalid I2C address for SP C250DN/SF toner chips: ${I2C_address}"
fi

Download the package. Run program_toner

Note: the blog page’

Just in case if people are wondering. The L01 chip’s datasheet is here:


To make a dedicated Raspberry Pi Zero device to run this script (program_toner) repeatedly on boot, I made a script loop_program_toner that runs the same program as infinite loop.

Since writing starts immediately once the chip is identified (no reason to hold the pogo pins for longer than needed), once the chip is detected, whether the writing finished successfully or not, threre’d be 5 seconds of delay so you can reseat the pogo pins (on write failure) or remove the pogo pins (so it won’t unnecessarily rewrite the chip again).

#!/bin/bash

if [[ $EUID -ne 0 ]]; then
  echo "Needs root permissions. Try sudo"
  exit -2
fi

while :
do
  bash "${BASH_SOURCE%/*}/program_toner"
  if (($?)); then
    # Wait longer to give time to detatch probe once chip is detected
    # whether the process completed successfully or not
    sleep 5;
  else
    sleep 1;
  fi
done

There are a few steps needs to auto make this automated:

  1. Have a user account dedicate to autorun the program on boot. In the example here, we’ll use Raspberry Pi’s default admin account
  2. Auto-logged on by running sudo raspi-config, select (1) System Option > (S5) Boot / Auto Login > (B2) Console Autologin
  3. Bless the said account with password free sudo rights: Run sudo visudo and add this to the end (replace admin with something else if you chose a different username)

admin ALL=(ALL) NOPASSWD:ALL
  1. Call loop_program_toner in ./profile or ~/.bashrc (startup script). Can place this at the bottom if your package folder is ~/spc250dn :
sudo ./spc250dn/loop_program_toner

The path can start with . or ~ because you are starting with at home folder anyway.


EDIT (2023-11-1): Linux people lack the backward compatibility decency again! They changed /led0 and /led1 to /ACT and /PWR under /sys/class/leds and they don’t have the decency to put in a symbolic link so old cold won’t break! WTF!

Do they have an idea how many code they’ll break and how many man hours they will waste the world to discover this breaking change when updating the distro? Non-caught exceptions just fall through and shows up as mysterious error messages!

If you want people to get the memo and change their code to match a more sensible notation, feel free to put in a warning stub, but hell no to just flip a switch and watch the world scream! This is apparently not a difficult decision to make when adding backward compatibility would be a huge project because of some catch-22 conditions!

ARM: dts: Standardise on the upstream LED names · raspberrypi/linux@ea14f14 (github.com)

Raspberry Pi Zero only has one LED which is /led0 (/ACT), so ignore /led1 (/PWR).

Loading