Category Archives: Programming

Picking an IDE for Python

Posted on by
Reply

The native features in MATLAB are often very good most of the time, as I’ve yet to hear anybody spending time to shop for a IDE outside the official one.

Atom has the feel of Maple/MathCAD, and Jupyter Notebook has the feel of Mathematica. Spyder feels like MATLAB the most, but it’s hugely primitive.

IDLE is more miserable than a command prompt. It doesn’t even have the decency to recall command history with up arrow. It’s like freaking DOS before loading doskey.com. Not to mention that single clicking on the window won’t set the cursor to the active command line, which you have to scroll all the way down to click on the bottom line. WTF! I’d rather use the command prompt and give up meaningless syntax coloring.

IPython (in Spyder) is unbearably slow (compare to MATLAB’s editor which I consider slow to the extent that it’s marginally bearable for the interactive features it offers), but at least usable unlike IDLE, and most importantly the output display is pprint (pretty printer) formatted so it’s legible. Just type locals() and see what kind of sh*t Python spits out in IDLE/cmd.exe and you’ll see what I meant.

I simply cannot live without who/whos provided in IPython, but I still don’t like it showing the accessible functions/modules along with the variables (I know, Python doesn’t tell them apart). Nonetheless it’s still weak because these are automagics that doesn’t return the results as Python data (just print). Spyder’s ‘variable explorer’ is the only place I can find that doesn’t include loaded functions/modules. Python should have provided facilities to get the user-introduced variables exclusively and leave the modules to a different function like MATLAB’s import command that shows imported packages/classes.

However, pretty printer doesn’t even come close to MATLAB in terms of the amount of dirty work disp() did to format the text to make it easy to read. Keys in the dictionary shown in pretty printer in Python are not right-aligned like MATLAB struct so we can easily tell keys and values apart. For example:

MATLAB struct shows:
          name: 'S'
          size: [9 1]
         bytes: 7765
         class: 'struct'
        global: 0
        sparse: 0
       complex: 0
       nesting: [1×1 struct]
    persistent: 0

Python with Pretty Printer shows:
{'__name__': '__main__',
 '__doc__': 'Automatically created module for IPython interactive environment',
 '__package__': None,
 '__loader__': None,
 '__spec__': None,
 '__builtin__': <module 'builtins' (built-in)>,
 '__builtins__': <module 'builtins' (built-in)>,
 '_ih': ['', 'locals()'],
 '_oh': {},
 '_dh': ['C:\\Users\\Administrator'],
 'In': ['', 'locals()'],
 'Out': {},
 'get_ipython': <bound method InteractiveShell.get_ipython of <ipykernel.zmqshell.ZMQInteractiveShell object at 0x00000000059B7828>>,
 'exit': <IPython.core.autocall.ZMQExitAutocall at 0x5a3b198>,
 'quit': <IPython.core.autocall.ZMQExitAutocall at 0x5a3b198>,
 '_': '',
 '__': '',
 '___': '',
 '_i': '',
 '_ii': '',
 '_iii': '',
 '_i1': 'locals()'}

I often convert things to MATLAB dataset() because the disp() method is excellent, such as struct2dataset(ver()). table/disp() is nice, but I think they overdid it by defaulting to fancy rich-text that bold the header, which makes it a magnitude of orders slower, and it’s not using the limited visual space effectively to show more data. Python still has a lot more to do in the user-friendly department.

Loading

Implicit ways to store data in a program

Posted on by
Reply

The most obvious way to store data is in plain data structures such as arrays and queues, or even a hashtable, but don’t forget these implicit ones:

  • Call stack. As the name say, it’s a stack data structure. It saves the local variables before making another function call. This is often exploited in recursion to avoid passing around an explicit data structure
  • Closures. What closure means is that when you create an anonymous function, the variables involved (other than the arguments) that is saved along (captured) with the function object you created. This can be exploited to make forward iterators or generators
  • Functions. You can make a function that does nothing other than returning a certain piece of data. It’s an excellent way to make avoid the overhead of managing (reading, updating promptly) a config file. Works best when your programming language requires so little typing to specify data (such as MATLAB) that your code is almost as short as a plain text config file.

Loading

Python 3 Scientific Installation To-do List

Posted on by
Reply

Although I am a big fan of MATLAB, it’s time for me to really try out Python so I can fairly compare the pros and cons of both languages.

The first tiny hurdle for Python is its scattered installation process for Windows. I thought Python(x,y) will give me everything in one place, but turns out the Spyder is stuck in Python 2.7. To install Python 3.7, I’ll need to do it from scratch. Here are the steps:

  1. Download official Python 3. You will need that for the “pip” package manager located in {python37}/scripts
  2. Update PIP first to avoid complaints. You can run it anywhere in command prompt
    python -m pip install --upgrade pip
    You don’t call PIP to update PIP because you an executable cannot write itself in WindowsNote that for 32-bit Python, you might run into Python37\python.exe: No module named pip, so you might want to use ensurepip to bootstrap: python -m ensurepip
  3. Now I’ll need Spyder3, a MATLAB-like IDE. Qt5 is one of the pre-req:
    pip install PyQt5
  4. And finally Spyder3
    pip install Spyder
    pip does not install icons in your start menu. So I’ll need to manually create a shortcut
    {Python37}/Scripts/spyder3.exe
    .py files are not associated with Spyder3 (normally it’ll just directly run the python script with python3). I usually manually change the association in Windows to Sypder3.
  5. PyVISA is the analog of “Instrument Control Toolbox” in MATLAB.
    pip install pyvisa
    MATLAB’s Instrument Control Toobox also cover serial ports, which is done in Python by PySerial
    pip install PySerial
  6. Numpy is included with scipy:
    pip install scipy
  7. Turns out that only NumPy and IPython is installed with SciPy, not the entire ecosystem.
    pip install pandas
    pip install matplotlib
    If you know the power of dataset/table objects in MATLAB like I do, you’ll jump for dataframes in panadas.
  8. SymPy, the analog of MATLAB’s symoblic math toolbox, needs to be installed separately
    pip install sympy
  9. IPython gives the ‘notebook’ feel in Mathematica, MathCAD and Maple, where the returned results are directly pasted in the same area where your command/syntax is. I rarely cared for it because I usually want the max visual real estate for my plots.

Update: I tried Anaconda (2019.03, Python 3.7.3 x64) which supposedly have everything in one place, but the Spyder it included crashes right out of the box. Jyupter is confusing as it relies on the web-browser to render the results. Feels patchy and doesn’t look like it adds more than the steps above. Uninstalled it without hesitation.

Update: To update the packages, tack -U switch at the end of each of the above pip install commands. Remember to follow the order of dependencies (e.g. update PyQt5 before Spyder)

Loading

MATLAB Techniques: variadic (variable input and output) arguments

Posted on by

I’ve seen a lot of ugly implementations from people trying to deal with variable number of input and output arguments. The most horrendous one I’ve seen so far came from MIT’s Physionet’s WFDB MATLAB Toolbox. Here’s a snippet showing how wfdbdesc.m handles variable input and output arguments:

function varargout=wfdbdesc(varargin)
% [siginfo,Fs,sigClass]=wfdbdesc(recordName)
...
%Set default pararamter values
inputs={'recordName'};
outputs={'siginfo','Fs','sigClass'};

for n=1:nargin
    if(~isempty(varargin{n}))
        eval([inputs{n} '=varargin{n};'])
    end
end
...
if(nargout>2)
    %Get signal class
    sigClass=getSignalClass(siginfo,config);
end
for n=1:nargout
    eval(['varargout{n}=' outputs{n} ';'])
end

 

The code itself reeks a very ‘smart’ beginner who didn’t RTFM. The code is so smart (shows some serious thoughts):

  • Knows to use nargout to control varargout to avoid the side effects when no output is requested
  • [Cargo cult practice]: (unnecessarily) track your variable names
  • [Cargo cult practice]: using varargin so it can be symmetric to varargout (also handled similarly). varargout might have a benefit mentioned above, but there is absolutely no benefit to use varargin over direct variable names when you are not forwarding or use inputParser().
  • [Cargo cult practice]: tries to be efficient to skip processing empty inputs. Judicially non-symmetric this time (not done to output variables)!

but yet so dumb (hell of unwise, absolutely no good reason for almost every ‘thoughtful’ act put in) at the same time. Definitely MIT: Make it Tough!

This code pattern is so wrong in many levels:

  • Unnecessarily obscuring the names by using varargin/varargout
  • Managing a list of variable names manually.
  • Loop through each item of varargin and varargout cells unnecessarily
  • Use eval() just to do simple cell assignments! Makes me cringe!

Actually, eval() is not even needed to achieve all the remaining evils above. Could have used S_in = cell2struct(varargin) and varargout=struct2cell(S_out) instead if one really wants to control the list of variable names manually!

The hurtful sins above came from not knowing a few common cell packing/unpacking idioms when dealing with varargin and varargout, which are cells by definition. Here are the few common use cases:

  1. Passing variable arguments to another function (called perfect forwarding in C++): remember C{:} unpacks to comma separated lists! 
    function caller(varargin)
   callee(varargin{:});
  2. Limiting the number of outputs to what is actually requested: remember [C{:}] on left hand side (assignment) means the outputs are distributed as components of C that would have been unpacked as comma separated lists, i.e. [C{:}] = f(); means [C{1}, C{2}, C{3}, ...] = f(); 
    function varargout = f()
// This will output no arguments when not requested,
// avoiding echoing in command prompt when the call is not terminated by a semicolon
    [varargout{1:nargout}] = eig(rand(3));
  3. You can directly modify varargin and varargout by cells without de-referencing them with braces! 
    function varargout = f(varargin)
// This one is effectively deal()
    varargout = varargin(1:nargout);
end

function varargout = f(C)
// This one unpacks each cell's content to each output arguments
    varargout = C(1:nargout);
end

One good example combining all of the above is to achieve the no-output argument example in #2 yet neatly return the variables in the workspace directly by name.

function [a, b] = f()
// Original way to code: will return a = 4 when "f()" is called without a semicolon
    a = 4;
    b = 2;
end

function varargout = f()
// New way: will not return anything even when "f()" is called without a semicolon
    a = 4;
    b = 2;
    varargout = manage_return_arguments(nargout, a, b);
end

function C = manage_return_arguments(nargs, varargin)
    C = varargin(1:nargs);
end

I could have skipped nargs in manage_return_arguments() and use evalin(), but this will make the code nastily non-transparent. As a bonus, nargs can be fed with min(nargout, 3) instead of nargout for extra flexibility.

With the technique above, wfdbdesc.m can be simply rewritten as:

function varargout = wfdbdesc(recordName)
% varargout: siginfo, Fs, sigClass
...
varargout = manage_return_arguments(nargout, siginfo, Fs, sigClass);

Unless you are forwarding variable arguments (with technique#1 mentioned above), input arguments can be (and should be) named explicitly. Using varargin would not help you avoid padding the unused input arguments anyway, so there is absolutely no good reason to manage input variables with a flexible list. MATLAB already knows to skip unused arguments at the end as long as the code doesn’t need it. Use exist('someVariable', 'var') instead.

 

 

Loading