Getting pyinstaller 3.4 to work with Python 3.7

Python is an excellent language, but given that it’s free, it also comes with a lot of conspicuous loose-ends that you will not expect in commercially supported platforms like MATLAB.

Don’t expect everything to work right out of the box in Python. Everything is like 98% there, with the last 2% frustrate the heck out of you when you are rushing to get from point A to point B and you have to iron out a few dozen kinks before you can really start working.

When I tried use pyinstaller (v3.4) to compile my Python (v3.7) program into an executable, I ended up having to jump through a bunch of hoops:

  • pip install pyinstaller gives:
    ModuleNotFoundError: No module named 'cffi'
  • Then I looked up and installed cffi
    pip install cffi
  • After the dependency was addressed manually (it shouldn’t )  pip install pyinstaller worked
  • Then I tried to compile my first Python executable with pyinstaller, and I got this exception:
    File "C:\Python37\lib\site-packages\win32ctypes\core\cffi\_advapi32.py", line 198
    
            ^
        SyntaxError: invalid syntax
  • I searched the exact string and learned that pyinstaller (v3.4) is not ready for Python 3.7 yet! How come pip installer didn’t check for it? I opened up the offending file and looked for line 198 and saw this:
    c_creds.CredentialBlobSize = \
    
        ffi.sizeof(blob_data) - ffi.sizeof('wchar_t')

    It’s a freaking line continuation character \ (actually the extraneous CR before CRLF) that rooster-blocked it.

  • I just deleted the line continuation and merged the two lines, and saved _advapi32.py, then I was able to compile my Python v3.7 code (using pyinstaller 3.4) with no issues.

This is not something you’ll experience as a MATLAB user. The same company, TMW, wrote the MATLAB compiler as well as the rest. The toolbox/packages are released together in one piece so breaking changes that causes failure for the most obvious use case are caught before they get out of the door.

Another example of breaking changes that I ran into: ipdb does not allow you to move cursor backward.

Again, this is the cost associated with free software and access to the latest updates and new features without waiting for April/October (it’s the MATLAB regular release cycle). If hassle and the extra engineering time far exceed licensing MATLAB licensing costs, MATLAB is a better choice, especially if software is just a chore to get your company from point A to point B, and you are willing to pay big bucks to get there quickly and reliably.

Even with free software on the table, your platform choice is always determined by:

  • how much your time is worth wrestling problems
  • how much flexibility do you need (for customizing to your needs)
  • how much you are willing to pay for the licenses and support

In any case, the community did good work. Please consider sponsoring PyInstaller and PSF if you profit immensely from their work.

 802 total views,  2 views today

Picking an IDE for Python

The native features in MATLAB are often very good most of the time, as I’ve yet to hear anybody spending time to shop for a IDE outside the official one.

Atom has the feel of Maple/MathCAD, and Jupyter Notebook has the feel of Mathematica. Spyder feels like MATLAB the most, but it’s hugely primitive.

IDLE is more miserable than a command prompt. It doesn’t even have the decency to recall command history with up arrow. It’s like freaking DOS before loading doskey.com. Not to mention that single clicking on the window won’t set the cursor to the active command line, which you have to scroll all the way down to click on the bottom line. WTF! I’d rather use the command prompt and give up meaningless syntax coloring.

IPython (in Spyder) is unbearably slow (compare to MATLAB’s editor which I consider slow to the extent that it’s marginally bearable for the interactive features it offers), but at least usable unlike IDLE, and most importantly the output display is pprint (pretty printer) formatted so it’s legible. Just type locals() and see what kind of sh*t Python spits out in IDLE/cmd.exe and you’ll see what I meant.

I simply cannot live without who/whos provided in IPython, but I still don’t like it showing the accessible functions/modules along with the variables (I know, Python doesn’t tell them apart). Nonetheless it’s still weak because these are automagics that doesn’t return the results as Python data (just print). Spyder’s ‘variable explorer’ is the only place I can find that doesn’t include loaded functions/modules. Python should have provided facilities to get the user-introduced variables exclusively and leave the modules to a different function like MATLAB’s import command that shows imported packages/classes.

However, pretty printer doesn’t even come close to MATLAB in terms of the amount of dirty work disp() did to format the text to make it easy to read. Keys in the dictionary shown in pretty printer in Python are not right-aligned like MATLAB struct so we can easily tell keys and values apart. For example:

MATLAB struct shows:
          name: 'S'
          size: [9 1]
         bytes: 7765
         class: 'struct'
        global: 0
        sparse: 0
       complex: 0
       nesting: [1×1 struct]
    persistent: 0

Python with Pretty Printer shows:
{'__name__': '__main__',
 '__doc__': 'Automatically created module for IPython interactive environment',
 '__package__': None,
 '__loader__': None,
 '__spec__': None,
 '__builtin__': <module 'builtins' (built-in)>,
 '__builtins__': <module 'builtins' (built-in)>,
 '_ih': ['', 'locals()'],
 '_oh': {},
 '_dh': ['C:\\Users\\Administrator'],
 'In': ['', 'locals()'],
 'Out': {},
 'get_ipython': <bound method InteractiveShell.get_ipython of <ipykernel.zmqshell.ZMQInteractiveShell object at 0x00000000059B7828>>,
 'exit': <IPython.core.autocall.ZMQExitAutocall at 0x5a3b198>,
 'quit': <IPython.core.autocall.ZMQExitAutocall at 0x5a3b198>,
 '_': '',
 '__': '',
 '___': '',
 '_i': '',
 '_ii': '',
 '_iii': '',
 '_i1': 'locals()'}

I often convert things to MATLAB dataset() because the disp() method is excellent, such as struct2dataset(ver()). table/disp() is nice, but I think they overdid it by defaulting to fancy rich-text that bold the header, which makes it a magnitude of orders slower, and it’s not using the limited visual space effectively to show more data. Python still has a lot more to do in the user-friendly department.

 350 total views,  2 views today