Keysight Calibration (Performance Validation) for Probes Specifically 1152A

I recently sent a 1152A probe for calibration and was surprised to find out the data on the calibration report tells little about how the tests are done and under what settings. I searched the throughly and called tech support and they confirmed my observation the performance validation procedures are not mentioned anywhere in the published documents.

I called Keysight cal department and was able to reach a super-helpful tech, Markis, who did the calibration for my 1152A probe and he explained to me how the calibration process is done when I called.

HP/Agilent/Keysight probes using AutoProbe interfaces are powered by 1143A (that was intended for 54701A probes) through a N1022A adapter (the one used in 81600 Infiniium DCA) for Keysight’s calibration process, which measures uncompensated probe-only performance. I saw the calibration reports from 3rd party-labs, and probes are are calibrated inside the oscilloscope they are used in, and therefore it’s measuring a compensated system (scope+probe) performance.

There is a 30 minute warm up period.

The procedures resembles to what’s detailed in the old 1144A probe user/service manual, (page 10-14) with the exception that the ‘Gain Accuracy’ done there is ‘AC gain accuracy’ (at 1kHz, 1Vrms) instead of ‘DC Gain Accuracy’ claimed on the report. In fact, given that it’s simply measuring relative error (multimeter reading of the probe BNC output divided by the 5V Fluke Calibrator reference) at one voltage setting, I believe it should be called ‘DC measurement accuracy’. The number on the calibration report was divided by 10 times since 1152A is a 10:1 probe.

The bandwidth test for 1152A is simply looking at attenuation at the advertised bandwidth (2.5Ghz for 1152A) relative to 50Mhz (low frequency reference set at 0dBm).

 

 

14 total views, 1 views today

MX Linux 18 setup notes

I tried Ubuntu on my old laptop, but it was unbearably much slower than that Windows 7 so I was looking for lightweight options. After some shopping, I settled on MX Linux as the drivers worked right of out the box for the hardware and it gave me the best user experience so far.

Other than responsiveness, the deciding factor that moves me away from Ubuntu is the amount work required to get the basic things working out of the box. Internationalization is almost fully configured in MX Linux, while I had to jump a few hoops to get the VL Gothic (Japanese) font in and struggled to get the IME to switch using Ctrl+Space / Ctrl+Shift (or any default shortcut keys) like in Windows. In MX Linux, they are the defaults right away.

I was really turned off by the fact that Ubuntu’s (minimal install) default Archive Manager is half-working out of the box: I get weird errors and partial success extracting RAR files because unrar was not installed by default! It just showed the lack of consideration about user experience.


MX Linux defaults to ibus, which works right out of the box with mozc (Japanese) language support. But I’d like to have a Cantonese IME that allows me to swear (the ibus-table-cantonese package was censored), so I opted for Andrew Choi’s CAP, which runs on fcitx. He used to have an iBus version, but it was a decade ago and I couldn’t get it to install.

Turns out it’s not that MX Linux is not that prepared when you want to use Fctix. None of the languages shows up when I tried to add an IME! After a lot of googling, I realized it requires im-config, and you need to install zenify before installing im-config!

After that fcitx works like a charm: mozc, CAP works in harmony, and I can turn the IME on/off by Ctrl+Space and switch between IMEs using Ctrl+Shift (just like in the old days)


EDIT: After all the praise I have on MX Linux. I noticed it overlooked something very basic! It does not make you configure timezone during setup and it’s not easy to change it! To do it the GUI way, first you have to go to “MX Time Settings”, and you have to type in the EXACT timezone string (TZ database name)! Geeze! It’s so caveman that we still have to do this in 2019!!

How did I noticed that I forgot to change the timezone? I realized the time in my Windows keeps getting changed (suspiciously a time-zone offset like difference) after I booted into MX Linux and boot back to Windows. That’s insidious!

EDIT: Turns out it’s a common problem when dual-booting Windows and Linux, not just MX Linux. Windows treats the hardware clock as local time and Linux treats the hardware clock as UTC time! Linux updates the time through NTP server blindly while Windows check if the current time is within 1hr from the NTP server to avoid unintended time changes (I have to give Microsoft credit for that). The easy solution is to have Linux follow Windows’ suit:

timedatectl set-local-rtc 1 --adjust-system-clock

59 total views, 4 views today

Getting MX Linux Samba to work naturally with Windows Network

I normally disable “Computer Browser” service in Windows by default because multiple computers having it on causes errors showing up in event log complaining there are multiple masters, and it’s not necessary for my simple home network because I’m just using a workgroup (no domain controller).

However, today I found out that even after setting up MX Linux’s Samba correctly (see below), if none of the computers on my windows network runs “Computer Browser”, my Windows computer name will not show up in “Thunar File Manager” although I can access it with smb://, and linux computers running Samba server shows up fine.


There’s also another twist to get the SMB client to work in Thunar File Manager. Despite smbtree works right out of the box (it detects the Windows shares), I’ll have to add this line in /etc/samba/smb.conf for the file manager to even probe the list of computers (not timeout):

name resolve order = bcast host

The point is to use broadcast lookup BEFORE dns lookup. DNS lookups for my local resources are often temperamental (could it be my router?), and I saw Linux Mint working with Windows briefly without these settings (editing smb.conf and also enabling “Computer Browser” service in Windows), but it failed today even after I re-installed MX Linux from scratch.

The other lines mentioned like client max protocol = NT1 and netbios name mentioned on the forums are not needed.

After configuring it. Restart smbd:

sudo /etc/init.d/smbd restart

 

54 total views, 3 views today

Spyder on MX Linux

This is another example that non-commercial (open-source) Linux/Python does not have a feel of a finished product: things break out of the box when installed fresh, in the most simple, expected ways, without any tweaks.

Again, don’t get me wrong, open-source free software are good stuff (more modern concepts and people working on it for free), but it’s never going to beat professional companies (like Microsoft/MATLAB) in how well-funded they are so they can maintain their software and the user experience using their profits. So far, users are still expected to put up with a bunch of unjustifiably unnecessary work to get to where they want to go with community-maintained software like Linux/Python.

This time I’m installing Spyder on MX Linux. Look at how many damn hoops I have to jump to get Spyder 3 to function properly there:

  1. I installed python3-spyder from MX Package Installer
  2. Spyder complained about missing rope on start
  3. Installed python-rope on MX Package Installer. The complaint still won’t go away
  4. I tried follow the instruction sudo pip3 install rope_py3k and realized pip3 was not installed already with the Python that came with Spyder! (Didn’t have the problem with the Windows counterpart).
  5. Installed python3-pip from MX Package Installer.
  6. Came back and run pip3 install rope_py3k. It choked at "Command 'python  setup.oy egg_info' failed with error code 1 in /tmp/pip-build-0nnknjhi/rope-py3k". Again, known problem.
  7. Followed the solution in the comments pip3 install --upgrade setuptools
  8. Then come back and run pip3 install rope_py3k again. It says "Failed building wheel for rope-py3k" in between, but nonetheless I’ll try to move on since it says "Successfully installed rope-py3k-0.9.4.post1"

Then Spyder launch uneventfully.

These are not design decisions (sacrificing one quality for another), but inter-operability wrinkles that nobody are paid enough to do the grunt-work babysitting it. So if your business profits heavily from it, consider sponsoring the developers!


It’s also slightly annoying that the version of Spyder maitained in MX Linux’s most recent repository is a little older than what’s actually available (3.1.3+dfsg1-3 instead of 3.3.4).

At first I followed instructions to have PIP to update it: pip3 install -U spyder, but it doesn’t work. I got a lot of “failed building wheel for (package)” error.

I also realized the Python that came with it is 3.5, not the 3.7(.3) I had in Windows. I checked the MX package manager and indeed it stopped at 3.5. After some searching, I learned the base system package was frozen in 2016! MX Test Repo (at your own risk) has Python 3.7 though.

36 total views, no views today

Ubuntu to remember list – setup

Network

  • Install samba. Use smb:// as URL
    (MX Linux doesn’t need you to remember it. Windows computers and shares just shows up in File Manager right away!)

Extracting files

  • Need to install unrar or the built-in Archive Manager will show “Unsupported error during extraction” and extract RAR files partially! WTF! It’s 2019! No wonder Microsoft is still making big bucks! (MX Linux is much better about this out of the box. Not only it works, I can just drag and drop the contents directly to the desktop while I cannot do it with Ubuntu!)

Internationalization

  • Japanese fonts: sudo apt-get fonts-vlgothic
  • Get Foobar2000 to show Japanese fonts: install the above and File->Preferences->Display->Colors and Fonts->Fonts-Playlists-> VL PGothic
  • Need to log-off and re-login after installing Japanese Language Support (it’s not just for UI translation) before “Japanese (Mozc)” input option show up
  • It’s a major pain in the butt to emulate the Ctrl+Space shortcut that switches between IMEs in Windows in Ubuntu, but in MX Linux, you can set it in “IBus Preferences”->Keyboard Shortcuts.

Routine software to install

  • rdesktop / Remmina
  • foobar2000 (Clementine player that came with MX Linux worked just as good. I chose foobar2000 to conserve resource in Ubuntu on a slow computer.)

31 total views, no views today

Watch out for ‘const’ method in Python

One thing I feel a little bit not quite as intuitive when I switch to Python is I constantly have to look up whether the method directly updates the contents or it’ll return a different object (of the same type) that I’ll have to overwrite the input variable myself.

An example would be strings and bytes object. replace() sounded like an updating method, but it’s actually a ‘const’ method (a term borrowed from C++ to say that the method does not have side-effects) that does not change the state of the object.

I initially thought this has to do with whether the object is immutable or not, but I tried it on bytearray objects (which is mutable), replace() behaves consistently with the identically named methods in other immutable objects (bytes object, string object): you’ll need to assign the output to self (basically bind the name to the temporary and throw away the original).

bts = b'test'
bts.replace('es', 'oas')       # dumps the output to workspace (can be accessed by _) and do nothing else
bts = bts.replace('es', 'oas') # actually updates bts

 

41 total views, no views today

Python startup management

The startup script is simply startup.m in whatever folder MATLAB start with.

Now how about Python? For plain Python (anything that you launch in command line, NOT Spyder though), you’ll need to ADD a new environment variable PYTHONSTARTUP to point to your startup script (same drill for Windows and Linux).

For Spyder, it’s Tools>Preferences>IPython console>Startup>”Run a file”:

but you don’t need that if you already have new environment variable PYTHONSTARTUP correctly setup.

 

67 total views, no views today

MATLAB and Python paths

MATLAB’s path() is equal to Python’s sys.path().


To add paths in MATLAB, use the obviously named function addpath(). Supply the optional -end argument if you don’t want any potential shadowing (i.e. the folder to import has lower priority if there’s an existing function with the same name).

I generally avoid userpath() or the graphical tools because the results are sticky (persists between sessions). The best way is to exclusively manage your paths with startup.m so you always know what you are getting. If you want full certainty, you can start with restoredefaultpath() in MATLAB.


Python’s suggested these as equivalents of MATLAB’s addpath():

sys.path.insert(0, folder_to_add_to_path)
sys.path.append(folder_to_add_to_path)

but just like MATLAB’s addpath() which works with strings only (not cellstr), these Python options do not work correctly  with Python lists because the methods in sys.path are as primitive as doing [sys.path, new_stuff]:

  1. This means you’ll end up with list of lists if you supplied Python lists as inputs to the above
    (MATLAB will throw an exception if you try to feed it with cellstr instead of polluting your path space with garbage)
  2. This also means it doesn’t check for duplicates! It’ll keep stacking entries!

To address the first problem, we use sys.path.extend() instead. It’s like doing addpath(..., '-end') in MATLAB. If you want it to be inserted at the front (higher priority, shadows existing), you’ll need sys.path = list_of_new_paths + sys.path. For MATLAB, you can make a path string like DOS by using pathsep:

addpath(strjoin(cellstr_of_paths, pathsep)))

Note that  sys.path.extend() is still not polymorphic: it expect iterables so if you feed it a string, which Python will consider it a list of characters, you will get a bunch of one character paths inserted!

On the other hand, DO NOT TRY to get around it in Python with the same trick like MATLAB by doing sys.path.append( ';'.join(path_list)). Python recognize sys.path as a list, NOT a one long string like MATLAB/Windows path, despite insert() and append() accepts only strings!

Aargh!

The second problem (which does NOT happen in MATLAB) is slightly more work. You’ll need to subtract out the existing paths before you add to it so that you won’t drag your system down by casually adding paths as you see fit. One way to do it:

def keep_only_new_set_of_paths(p):
    return set(p)-set(sys.path)

You should organize your programs and libraries in a directory tree structure and use code to crawl the right branch into a path list! Don’t let the lack of built-in support to tempt you to organize files in a mess. Keep the visuals clean as mental gymnastics/overheads can seriously distract you from the real work such as thinking through the requirements and coming up with the right architecture and data structures. If you constantly need to jump a few hoops to do something, do it only once or twice using the proper way (aka, NOT copying-and-pasting boilerplate code), and reuse the infrastructure.

At my previous workplaces, they had dozens and dozens of MATLAB files including all laying flat in one folder. The first thing I did when I join a new team is showing everybody this idiom that recursively adds everything under the folder into MATLAB paths:

addpath(genpath())

Actually the built-in support for recursive directory search sucks for both MATLAB and Python.  Most often what we need is just a list of full paths for a path pattern that we search recursively, basically dir/w/s *. None of them has this right out of the box. They both make you go through the comprehensive data structure returned (let it be tuples from os.walk() in Python or dir() in MATLAB) and so some manipulations to get to this form.

genpath() itself is slow and ugly. It’s basically a recursive wrapper around dir() that cleans up garbage like '.' and '..'.  Instead of getting a newline character, a different row (as a char array) or a different cell (as cellstr), you get semi-colons (;) as pathsep in between. Nonetheless, I still use it because despite I have recursive path tools in my own libraries, I’ll need to load the library first in my startup file, which requires a recursive path tool like genpath(). This bootstraps me out of a chicken-and-egg problem without too much ugly syntax.


Most people will tell you to do a os.walk() and use listcomp to get it in the typical full path form, but I’m not settling for distracting syntax like this. People in the community suggested using glob for a relatively simple alternative to genpath()

Here’s a cleaner way:

def list_subfolders_recursively(p):
    p = p + '/**/' 
    return glob.glob(p, recursive=True);

It’s also worth noting that Python follows Linux’s file search pattern where directory terminates with a filesep (/) while MATLAB’s dir() command follows the OS, which in Windows, it’s *..

Both MATLAB and Python uses ** to mean regardless of levels, but you’ll have to turn on the recursive=True in glob manually. ** is already implied to be recursive in MATLAB’s dir() command.


Considering there’s quite a bit of plumbing associated with weak set of sys.path methods provided in Python, I created a qpath.py next to my startup.py:

''' This is the quick and dirty version to bootstrap startup.py
Should use files.py that issue direct OS calls for speed'''

import sys
import glob

def list_subfolders_recursively(p):
    p = p + '/**/' 
    return glob.glob(p, recursive=True);

def keep_only_new_set_of_paths(p):
    return set(p)-set(sys.path)

def set_of_new_subfolders_recursively(p):
    return keep_only_new_set_of_paths( list_subfolders_recursively(p) )

def add_paths_recursively_bottom(p):
    sys.path.extend(set_of_new_subfolders_recursively(p));

def add_paths_recursively_top(p):
    # operator+() does not take sets
    sys.path = list(set_of_new_subfolders_recursively(p)) + sys.path;

In order to be able to import my qpath module at startup.py before it adds the path, I’ll have put qpath.py in the same folder as startup.py, and request startup.py to add the folder where it lives to the system path (because your current Python working folder might be different from PYTHONSTARTUP) so it recognizes qpath.py.

This is the same technique I came up with for managing localized dependencies in MATLAB: I put the dependencies under the calling function’s folder, and use the path of the .m file for the function as the anchor-path to add paths inside the function. In MATLAB, it’s done this way:

function varargout = f(varargin)
  anchor_path = fileparts( mfilename('fullpath') );
  addpath( genpath(fullfile(anchor_path, 'dependencies')) );
  % Body code goes here

Analogously,

  • Python has __file__ variable (like the good old preprocessor days in C) in place of mfilename().
  • MATLAB’s  mfilename('fullpath') always gives the absolute path, but Python’s  __file__ is absolute if it’s is not in sys.path yet, and relative if it’s already in it.
  • So to ensure absolute path in Python, apply os.path.realpath(__file__). Actually this is a difficult feature to implement in MATLAB. It’s solved by a MATLAB FEX entry called GetFullPath().
  • Python os.path.dirname is the direct equivalent of fileparts() if you just take the first argument.

and in my startup.py (must be in the same folder as pathtools.py):

import os
import sys

sys.path.append(os.path.dirname(os.path.realpath(__file__)))

import pathtool

user_library_path = 'D:/Python/Libraries';
pathtool.add_paths_recursively_bottom(user_library_path)

This way I can make sure all the paths are deterministic and none of the depends on where I start Python.


Now I feel like Python is as mature as Octave. It’s usable, but it’s missing a lot of thoughtful features compared to MATLAB. Python’s entire ecosystem like at least 10 years behind MATLAB in terms of user friendliness. However, Python made it up with some pretty advanced language features that MATLAB doesn’t have, but nonetheless, we are still stuck with quite a bit of boilerplate code in Python, which decreases the expressiveness of the language (I’m a proponent of self-documenting code: variable and function names and their organization should be carefully designed to tell the story; comments are reserved for non-obvious tricks)

71 total views, no views today

Getting pyinstaller 3.4 to work with Python 3.7

Python is an excellent language, but given that it’s free, it also comes with a lot of conspicuous loose-ends that you will not expect in commercially supported platforms like MATLAB.

Don’t expect everything to work right out of the box in Python. Everything is like 98% there, with the last 2% frustrate the heck out of you when you are rushing to get from point A to point B and you have to iron out a few dozen kinks before you can really start working.

When I tried use pyinstaller (v3.4) to compile my Python (v3.7) program into an executable, I ended up having to jump through a bunch of hoops:

  • pip install pyinstaller gives:
    ModuleNotFoundError: No module named 'cffi'
  • Then I looked up and installed cffi
    pip install cffi
  • After the dependency was addressed manually (it shouldn’t )  pip install pyinstaller worked
  • Then I tried to compile my first Python executable with pyinstaller, and I got this exception:
    File "C:\Python37\lib\site-packages\win32ctypes\core\cffi\_advapi32.py", line 198
    
            ^
        SyntaxError: invalid syntax
  • I searched the exact string and learned that pyinstaller (v3.4) is not ready for Python 3.7 yet! How come pip installer didn’t check for it? I opened up the offending file and looked for line 198 and saw this:
    c_creds.CredentialBlobSize = \
    
        ffi.sizeof(blob_data) - ffi.sizeof('wchar_t')

    It’s a freaking line continuation character \ (actually the extraneous CR before CRLF) that rooster-blocked it.

  • I just deleted the line continuation and merged the two lines, and saved _advapi32.py, then I was able to compile my Python v3.7 code (using pyinstaller 3.4) with no issues.

This is not something you’ll experience as a MATLAB user. The same company, TMW, wrote the MATLAB compiler as well as the rest. The toolbox/packages are released together in one piece so breaking changes that causes failure for the most obvious use case are caught before they get out of the door.

Another example of breaking changes that I ran into: ipdb does not allow you to move cursor backward.

Again, this is the cost associated with free software and access to the latest updates and new features without waiting for April/October (it’s the MATLAB regular release cycle). If hassle and the extra engineering time far exceed licensing MATLAB licensing costs, MATLAB is a better choice, especially if software is just a chore to get your company from point A to point B, and you are willing to pay big bucks to get there quickly and reliably.

Even with free software on the table, your platform choice is always determined by:

  • how much your time is worth wrestling problems
  • how much flexibility do you need (for customizing to your needs)
  • how much you are willing to pay for the licenses and support

In any case, the community did good work. Please consider sponsoring PyInstaller and PSF if you profit immensely from their work.

90 total views, no views today