## Python packages, modules and imports

Python’s import structure is freaking confusing. Learning by examples (i.e. imitating example code) does not help understanding the logic of it, and there are a lot of possible invalid combinations that are dead ends. You need to understand the concepts below to use it confidently!

Just like C++ quirks, very often there’s valid reasoning behind this confusing Python design choice and it’s not immediately obvious. Each language cater certain set of use cases at the expense of making other scenarios miserable. That’s why there’s best universal language for all projects. Know the trade-offs of the languages so you can pick the right tool for the job.

## MATLAB’s one file per function/script design

MATLAB made the choice of having one file describe one exposed object/function/class/script so it maps directly into the mental model of file systems. This is good for both user’s sanity and have behavioral advantages for MATLAB’s interpreter

1. Users can reason the same same way as they do with files, which is less mental gymnastics
2. Users can keep track of what’s available to them simply by browsing the directory tree and filenames because file names are function names, which should be sensibly chosen.
3. Just like users, MATLAB also leverage the file system for indexing available functions and defer loading the contents to the memory until it’s called at runtime, which means changes are reflected automatically.

## Package/modules namespace models in MATLAB vs Python

MATLAB traditionally dumps all free functions (.m files) available in its search paths into the root workspace. Users are responsible for not picking colliding names. Classes, namespaces and packages are after-thoughts in MATLAB while the OOP dogma is the central theme of Python, so obviously such practices are frowned upon.

RANT: OOP is basically a worldview formed by adding artificial man-made constructs (meanings such as agents, hierarchy, relationships) to the idea of bundling code (programs) and data (variables) in isolated packages controlled (scoped) by namespaces (which is just the lexer in your compiler enforcing man-made rules). The idea of code and data being the same thing came from Von Neumann Architecture: your hard drive or RAM doesn’t care what the bits stands for; it’s up to your processor and OS to exercise self-restraint. People are often tempted to follow rules too rigidly or not to take them seriously when what really matters is understanding where the rules came from, why they are useful in certain contexts and where they do not apply.

Packages namespaces are pretty much the skeleton of classes so the structure and syntax is the same for both. From my memory, it was at around 2015 that MATLAB started actively encouraging users (and their own internal development) to move away from the flat root workspace model and use packages to tuck away function names that are not immediately relevant to their interests and summon them through import syntax as needed. This practice is mandatory (enforced) in Python!

However are a few subtle differences between the two in terms of the package/module systems:

• MATLAB does not have from statement because import do not have the option to exposed the (nested tree of) package name to the workspace. It always output the leaf-node to the current workspace, the same way as from import syntax is used in Python.
• MATLAB does not have an optional as statement for you to give an alternative name to the package you just imported. In my opinion, Python has to provide the as statement as an option to shorten package/module names because it was too aggressively tucking away commonly used packages (such as numpy) that forcing people to spell the informative names in full is going to be an outcry.
• Unlike free functions (.m files), MATLAB classes are cached once the object is instantiated until clear classes or the like that gets rid of all instances in the workspace. Python’s module has the same behavior, which you need to unload with del (which is like MATLAB’s clear).
• Python’s modules are not classes, though most of the time they behave like MATLAB’s static classes. Because the lack of instantiated instances, you can reload Python modules with importlib.reload(). On the other hand, since MATLAB packages merely manages when the .m files can get into the current scope (with import command), the file system still indexes the available function list. Changes in .m file functions reflects immediately on the next call in MATLAB, yet Python has to reload the module to update the function names index because the only way to look at what functions are available is revisiting the contents of an updated .py file!
• MATLAB abstracts folder names (that starts with + symbol) as packages and functions as .m files while Python abstracts the .py file as a module (like MATLAB’s package) and the objects are the contents inside it. Therefore Python packages is analogous to the outer level of a double-packed (nested) MATLAB package. I’ll explain this in detail in the next sections.

## Files AND directories are treated the same way in module hierarchy!

This comes with a few implications

• if you name your project /myproj/myproj.py with a function def myproj(), which is a very usual thing most MATLAB users would do, your module is called myproj.myproj and if you just import myproj, you will call your function as myproj.myproj.myproj()!
• you can confuse Python module loader if you have a subfolder named the same as a .py file at the same level. The subfolder will prevail and the .py file with the same name is shadowed!

The reason is that Python allows users to mix scripts, functions, classes in the same file and they classes or functions do not need to match the filenames in order for Python to find it, therefore the filename itself serves as the label for the collection (module) of functions, classes and other (script) objects inside! The directory is a collection of these files which itself is a collection, so it’s a two level nest because a directory containing a .py file is a collection of collection!

On the other hand, in MATLAB, it’s one .m file per (publicly exposed) function, classes or scripts, so the system registers and calls them by the filename, not really by how you named it inside. If you have a typo in your function name that doesn’t match your filename, your filename will prevail if there’s only one function there. Helper functions not matching the filename will not be exposed and it will have a static/file-local scope.

Packages in MATLAB are done in folders that starts with a + symbol. Packages by default are not exposed to global namespaces in your MATLAB’s paths. They work like Python’s module so you also get them into your current workspace with import. This means it’s not possible to define a module in a file like Python. Each filename exclusively represent one accessible function or classes in the package (no script variables though).

So in other words, there are no such thing called modules in MATLAB because the concept is called package. Python separated the two concepts because .py file allowing a mixture of scripts, classes and loose functions formed a logical unit with the same structure as packages itself, so they need another name called module to separate folder-based collection (logical unit) and file-based collections (logical unit).

This is very counterintuitive at the surface (because it defeats the point of directories) if you don’t know Python allowing user to mix scripts, functions and classes in a file meant the file itself is a module/collection of executable contents.

## from (package/module) import (package/module or objectS) <as (namespace)>

This syntax is super confusing, especially before we understand that

1. packages has to be folders (folder form of modules)
2. modules can be .py files as well as packages
3. packages/modules are technically objects

The hierarchy for the from import as syntax looks like this:

package_folder > file.py > (obj1, obj2, ... )

This has the following implications:

• from strips the specified namespace so import dumps the node contents to root workspace
• import without from exposes the entire hierarchy to the root workspace.
• functions, classes and variables in the scripts are ALL OBJECTS.
• if you do import mymodule, a function f in mymodule.py can only be accessed through mymodule.f(), if you want to just call f() at the workspace, do from mymodule import f

These properties also shapes the rules for where wildcards are used in the statement:

• from cannot have wildcards because they are either a folder (package) or a file (module)
• import is the only place that can have wildcards * because it is only possible to load multiple objects from one .py file.
• import * cannot be used without from statement because you need to at some point load a .py file
• it’s a dead end to do from package import * beacuse it’s trying to load the files to the root workspace which they are uncallalble.
• it also does not make sense (nor possible) to follow import * with as statement because there is no mechanism to map multiple objects into one object name

So the bottom line is that your from import as statement has to somehow load a .py file in order to be valid. You can only choose between these two usage:

• load the .py file with from statement and pick the objects at import, or
• skip the from statement and import the .py file, not getting to choose the objects inside it.

as statement can only work if you have only one item specified in import, whether it’s the .py file or the objects inside it. Also, if you understand the rationales above, you’ll see that these two are equivalent:

from package_A import module_file_B as namespace_C
import package_A.module_file_B as namespace_C

because with as statement, whatever node you have selected is accessed through the output namespace you have specified, so whether you choose to strip the path name structure in the extracted output (i.e. use from statement) is irrelevant since you are not using the package and module names in the root namespace anymore.

58 total views

## Data Type characteristics

* Shallow assignment (transferring reference only) means the LHS does not have its own copy, so modifying the new reference will modify the underlying data on the RHS.

## Syntax / Usage

Powershell specific

• The UNCAPTURED output value in the last line of the block is the return value! Unary side effect statements such as $x++ do not have output value. Watch out for statements that looks like it’s going nowhere at the end of the code as these are not nop/bugs, but return value. This has the same stench as fall-throughs. • foreach() follows the last uncaptured output value return rule above doing a 1-to-1 map from the input collection to output collection (you can assign output to foreach() as it’s also seen as a function) • Powershell suck at binary operations between two arrays. Just an elementwise A+B you’d be thinking in terms of loops and worry about dimensions. • You can put if and loop blocks inside collections list construction, like this: @( 3, if(cond1){...;$v1}  do{...; $v2}while(cond2) ) MATLAB specific • When used with classes and custom matrices/arrays, chaining fields/properties/methods by indices often do not work, when they do, they often give out only the first element instead of the entire array (IIRC, there are operator methods that needs to be coordinated in the classes involved to make sure they chain correctly). In short, just don’t chain unless in very simple, scalar cases. Always output it to a variable a access the leaf. ## Range & Indexing Negative (cyclic) indexing along with automatic descending range, along with the lack of ‘end’ keyword is a huge pain in the rear when you want to scan from left to right like A[5:end]. Instead, you’ll have to do $A[4..($A.length-1)] because the range 4..-1 inside A[4..-1] is unrolled as 4,3,2,1,0,-1 (thus scanning from right to left and wraps around) without first consulting with the array A like the end keyword in MATLAB does so it can substitute the ends of the range with the array information before it unrolls. I am willing to bet that this behavior does not have a sound basis other than people thinking negative indices and descending ranges alone are two good ideas without realizing that nearly nobody freaking wants to scan from right to left and wrap around! I had the same gripes about negative indices in Python not carefully coordinating with other combinations in common use cases which cases unintuitive behavior. ## Range indexing syntax # Powershell 1..10 # No step/skip for range creation A[1..10] # No special treatment in array such as figuring out the 'end' % MATLAB A[start:(step):stop] # Python A[range(start,stop,step)] # Slicing (it's not range) A[(start):(stop):(step)] # Can skip everything # In Python, A=X merely reassign the label A as the alias for X. # Modifying the reassigned A through A=X will modify underlying contents of X # To deep-copy contents without .Clone(), assign the full slice A[:] = X  ## Hasthtable / Dictionaries % MATLAB: Use dynamic fields in struct or containers.Map() # Python: dictionaries such as {a:1, b='x'} # Powershell: @{a=1, b='x'} ## Structs Powershell does not have direct struct or dynamic field name struct. Instead if your object is uniform (you expect the fields not to change much), use [PSCustomObject]@{}. You can also just use simple hashtable @{}, but for some reason it doesn’t work the way I expected when put into arrays when I try to reference it by array index. ## Array rules surprises • Array comparisons are filtering operation (not boolean array output like MATLAB). (0..9) -ge 5 gives 5 to 9, not a list of False … False, True … True. To get a boolean array, use this shortcut: (0..9) | % {$_ -ge 5}

Map-filter combo syntax is | ? instead of Map syntax | %

• Monad (Cells in MATLAB) are unpacked and stacked by default (in MATLAB, I had to write a lot of routines to unpack and stack cells of cells). To keep cells packed (in MATLAB lingo, it’s like ‘UniformOutput’, false in cellfun), add a comma unary operator in front of the operation that are expected to be unpacked like this:

## Libraries and Modules

• Reload module using Import-Module \$moduleName -Force

47 total views

## GUI Paradigms (1): Redux (Flutter/React) translated to MATLAB

For GUI development, we often start with controls (or widgets) that user interact with and it’ll emit/run the callback function we registered for the event when the event happens.

Most of the time we just want to read the state/properties of certain controls, process them, and update other controls to show the result. Model-View-Controller (MVC) puts strict boundaries between interaction, data processing and display.

The most common schematic for MVC is a circle showing the cycle of Controller->Model->View, but in practice, it’s the controller that’s the brains. The view can simultaneously accept user interactions, such as a editable text box or a list. The model usually don’t directly update the view directly on its own like the idealized diagram.

With MVC, basically we are concentrating the control’s callbacks to the controller object instead of just letting each control’s callback interact with the data store (model) and view in an unstructured way.

When learning Flutter, I was exposed to the Redux pattern (which came from React). Because the tutorials was designed around the language features of Dart, the documentation kind of obscured the essence of the idea (why do we want to do this) as it dwelt on the framework (structure can be refactored into a package). The docs talked a lot about boundaries but wasn’t clearly why they have to be meticulously observed, which I’ll get to later.

The core inspiration in Redux/BLoC is taking advantage of the concept of ‘listening to a data object for changes’ (instead of UI controls/widget events)!

Instead of having the UI control’s callback directly change other UI control’s state (e.g. for display), we design a state vector/dictionary/struct/class that holds contents (state variables) that we care. It doesn’t have to map 1-1 to input events or 1-1 to output display controls.

When an user interaction (input) event emitted a callback, the control’s callback do whatever it needs to produce the value(s) for the relevant state variable(s) and change the state vector. The changed state vector will trigger the listener that scans for what needs to be updated to reflect the new state and change the states of the appropriate view UI controls.

This way the input UI controls’ callbacks do not have to micromanage what output UI controls to update, so it can focus on the business logic that generates the content pool that will be picked up by the view UI controls to display the results. In Redux, you are free to design your state variables to match more closely to the input data from UI controls or output/view controls’ state. I personally prefer a state vector design that is closer to the output view than input controls.

The intuition above is not the complete/exact Redux, especially with Dart/Flutter/React. We also have to to keep the state in ONE place and make the order of state changes (thus behavior) predictable!

• Actions and reducers are separate. Every input control fires a event (action signal) and we’ll wait until the reducers (registered to the actions) to pick it up during dispatch() instead of jumping on it. This way there’s only ONE place that can change states. Leave all the side effects in the control callback where you generate the action. No side effects (like changing other controls) allowed in reducers!
• Reducers do not update the state in place (it’s read only). Always generate a new state vector to replace the old one (for performance, we’ll replace the state vector if we verified the contents actually changed). This will make timing predictable (you are stepping through state changes one by one)

In Javascript, there isn’t really a listener actively listening state variable changes. Dispatch (which will be called every time the user interacts using control) just runs through all the listeners registered at the very end after it has dispatched all the reducers. In MATLAB, you can optionally set the state vector to be Observable and attach the change listener callback instead of explicitly calling it within dispatch.

Here is an example of a MATLAB class that captures the spirit of Redux. I added a 2 second delay to emulate long operations and used enableDisableFig() to avoid dealing with queuing user interactions while it’s going through a long operation.

classdef ReduxStoreDemo < handle

% Should be made private later
properties (SetAccess = private, SetObservable)
state % {count}
end

methods (Static)
% Made static so reducer cannot call dispatch and indirectly do
% side effect or create loops
function state = reducer(state, action)
% Can use str2fun(action) here or use a function map
switch action
case 'increment'
fprintf('Wait 2 secs before incrementing\n');
pause(2)
state.count = state.count + 1;
fprintf('Incremented\n');
end
end
end

% We keep all the side-effect generating operations (such as
% temporarily changing states in the GUI) in dispatch() so
% there's only ONE PLACE where state can change
methods
function dispatch(obj, action, src, evt)
% Disable all figures during an interaction
figures = findobj(groot, 'type', 'figure');
old_fig_states = arrayfun(@(f) enableDisableFig(f, 'off'), figures);
src.String = 'Wait ...';

new_state = ReduxStoreDemo.reducer(obj.state, action);

% Don't waste cycles updating nops
if( ~isequal(new_state, obj.state) )
% MATLAB already have listeners attached.
% So no need to scan listeners like React Redux
obj.state = new_state;
end

% Re-enable figure obj.controls after it's done
arrayfun(@(f, os) enableDisableFig(f, os), figures, old_fig_states);
src.String = 'Increment';
end
end

methods
function obj = ReduxStoreDemo()
figure();
obj.state.count = 0;

h_1x  = uicontrol('style', 'text', 'String', '1x Box', ...
'Units', 'Normalized', ...
'Position', [0.1 0.3, 0.2, 0.1], ...
'HorizontalAlignment', 'left');
'PostSet', @(varargin) obj.update_count_1x( h_1x , varargin{:}));

uicontrol('style', 'pushbutton', 'String', 'Increment', ...
'Units', 'Normalized', ...
'Position', [0.1 0.1, 0.15, 0.1], ...
'Callback', @(varargin) obj.dispatch('increment', varargin{:}));

% Force trigger the listeners to reflect the initial state
obj.state = obj.state;
end
end

%% These are 'renders' registered when the uiobj.controls are created
% Should stick to reading off the state. Do not call dispatch here
% (just leave it for the next action to pick up the consequentials)
methods
% The (src, event) is useless for listeners because it's not the
% uicontrol handle but the state property's metainfo (access modifiers, etc)
function update_count_1x(obj, hObj, varargin)
hObj.String = num2str(obj.state.count);
end
end

end

164 total views

## MATLAB repeating arrays (elementwise array replication, interleaved ‘repmat’)

Since MATLAB R2015b, there’s a new feature called repelem(V, dim1, dim2, ...) which repeats each element by dimX times over dimension X. If N (dim1) is scalar, each V is uniformly repeated by N times. If N is a vector, it has to be the same length as V and each element of N says how many times the corresponding element in V is repeated.

Here are some historical ways of doing it (as mentioned in MATLAB array manipulation tips)

The scalar case (repeat uniformly) can be emulated by a Kronecker product multiplying everything with 1 (self):

kron(V, ones(N,1))

Kron method is conceptually smart but it has unnecessary arithmetic (multiply by 1). Nonetheless this method is reasonable fast until TMW finally developed a built-in function for it that outperforms all the tricks people have accumulated over decades.

The vector case (each element is repeated a different number of times according to vector N) is basically decoding Run-Length Encoding (RLE), aka counts to placements, which you can download maturely written programs on MATLAB File Exchange (FEX). There are a bunch of cumsum/diff/accumarray/reshape tricks but at the end of the day, they are RLE decoding in vectorized forms.

There’s a name for almost each recurring problem that we can think of in MALTAB. Before jumping in and implementing your for loop, ask around and try to find the right keyword/terms to describe your problem! >99.9% of the time your problem is not new!

The most odd-ball MATLAB algorithm scenario I’ve ever came across that requires original thought is the ‘Jenga Matrix‘ (I coined the name) while I was working at Stanford University Medical School as a research assistant for MADIT-CRT.

MATLAB’s OOP was not mature at that time, so dataset() objects didn’t surface. The reason for the ‘Jenga Matrix’ was to create ‘sparse cells’ which uses a sparse matrix with non-zero indices mapping to a cell vector so I can make a table (that’s approximately the guts of heterogenous data structure).

As I remove elements of the ‘sparse cell matrix’, I didn’t want holes in it to accumulate so I’ll have to periodically compact the underlying cell vector and shift the indices to reflect the indices after compacting. Normally if you have to mess with these kind of ingenious indexing algorithms, you are working on some generic abstractions/tools rather than the business logic itself.

There’s no ultimate correct way to implement something in MATLAB, but there are tons of bad ways that is strictly worse under all circumstances! Being smart with these little toy (Cody) problems like array manipulation do not really show practical proficiency in MATLAB. Anybody can spend a day or two to solve a genuinely new algorithm puzzle or just ask around in the forums if you run into it once in a blue moon. Who cares if you can do it 5 times faster if it’s just <1% of the development time?

Most of your time should be spent on using MATLAB to succinctly and intuitively describe your business logic (which requires exploring and understanding your project requirements deeply), and hide the boring background work with generic abstractions (e.g. RDBMS and RLE)! People should be able to read your function and variable names and form a clear picture of what your codebase is trying to achieve instead of stumbling over smart-ass idioms that’s not immediately obvious (which should buried in the lowest level of generic tool functions if you had to develop it in-house).

Even a mathematician in Linear Algebra using MATLAB for 40 years doesn’t mean he’s good at MATLAB! The real MATLAB skills are keeping up with MATLAB has to offer for a variety of scenarios relevant to the task at hand (or know enough abstract concepts like functional programming, OOP, database, etc, to be able to find out the right tools quickly), which is a hell lot of knowledge considering MATLAB covered most common scenario imaginable (the vast majority of MATLAB users aren’t aware of the full offerings and used MATLAB the wrong/hard way)!

71 total views

## Data Relationships of Spreadsheets: Relational Database vs. Heterogenous Data Tables

This blog post is development in process. Will fill in the details missing details (especially pandas) later. Some of the MATLAB syntax are inaccurate in the sense that it’s just a description that is context dependent (such as column names can be cellstr, char string or linear/logical indices).

From data relationship point of view, relation database (RDMBS), heterogenous data tables (MATLAB’s dataset/table or Python Panda’s Dataframe) are the same thing. But a proper database have to worry about concurrency issues and provide more consistency tools (ACID model).

Heterogenous data tables are almost always column-oriented database (mainly for analyzing data) where MySQL and Postgres are row-store database. You can think of column-store database as Struct of Arrays (SoA) and row-store database as Array of Struct (AoS). Remember locality = performance: in general, you want to put the stuff you frequently want to access together as close to each other as possible.

Mechanics:

Core database concepts:

Formal databases has a Table Definition (Column Properties) that must be specified ahead of time and can be updated in-place later on (think of it as static typing). Heterogenous Data Tables can figure most of that out on the fly depending on context (think of it as dynamic typing). This impacts:

• data type (creation and conversion)
• unspecified entries (NULL).
Often NaN in MATLAB native types but I extended it by overloading relevant data types with a isnull() function and consistently use the same interface
• default values
• keys (Indices)

SQL features not offered by heterogenous data tables yet:

• column name aliases (AS)
• wildcard over names (*)
• pattern matching (LIKE)

SQL features that are unnatural with heterogeneous data tables’ syntax:

• implicitly filter a table with conditions in another table sharing the same key.
It’s an implied join(T, T_cond)+filter operation in MATLAB. Often used with ANY, ALL, EXISTS

Fundamentally heterogenous data types expects working with snapshots that doesn’t update often. Therefore they do not offer active checking (callbacks) as in SQL:

• Invariant constraints (CHECK, UNIQUE, NOT NULL, Foreign key).
• Auto Increment
• Virtual (dependent) tables (CREATE VIEW)

• Tall vs wide tables

Language logistics (not related to database)

87 total views

MATLAB’s dataset/table objects’ internals often involves identifying unique contents and assigning a unique (grouping) index to it so the indices can be mapped or joined without actually going through the contents of each row.

In the old days when I were using dataset(), the first generation of table() objects before the rewrite, there is a tool called grp2idx() which assigns the same number to identical items regardless of data types. It was part of Statistics Toolbox (needs to pay extra for it) and it does not work if you have multiple columns that you want to assign an unique index unless the ROWS are identical.

Upon inspection. grp2idx() is overrated. There are two ways to get it without paying for the toolbox:

• double(categorical(X)): cast a categorical type (technically you can use nominal/ordinal, but it’s part of statistics toolbox)
• Use the 2nd output argument for sort() or sortrows() function. I recommend sortrows() because it’s can be overloaded on table() objects and it works on multiple rows.

955 total views

## Anonymous Functions (MATLAB) vs Lambdas (Python) Anonymous Functions in MATLAB is closure while Lambdas in Python are not

Lambdas in Python does not play by the same rules as anonymous functions in MATLAB

• MATLAB takes a snapshot of (capture) the workspace variables involved in the anonymous function AT the time the anonymous function handle is created, thus the captured values will live on afterwards (by definition a proper closure).
• Lambda in Python is NOT closure! [EDIT: I’ll need to investigate the definition of closure more closely before I use the term here] The free variables involved in lambda expressions are simply read on-the-fly (aka, the last state) when the functor is executed.

It’s kind of a mixed love-and-hate situation for both. Either design choice will be confusing for some use cases. I was at first thrown off by MATLAB’s anonymous function’s full variable capture behavior, then after I get used to it, Python’s Lambda’s non-closure tripped me. Even in the official FAQ, it address the surprise that people are not getting what they expected creating lambdas in a for-loop.

To enable capture in Python, you assign the value you wanted to capture to a lambda input argument (aka, using a bound variable as an intermediary and initialize it with the free variable that needs to be captured), then use the intermediary in the expression. For example:

lambda: ser.close()      # does not capture 'ser'
lambda s=ser: s.close()  # 'ser' is captured by s.

I usually keep the usage of nested functions to the minimum, even in MATLAB, because effectively it’s kind of a compromised ‘global’ between nested levels, or a little bit like protected classes in C++. It breaks encapsulation (intentionally) for functions in your inner circle (nest).

It’s often useful for coding up GUI in MATLAB quick because you need to share access to the UI controls within the same group. For GUI that gets more complicated, I actually avoided nested functions altogether and used *appdata() to share UI object handles.

Functors of nested functions are closures in both MATLAB and Python! Only Lambdas in Python behave slightly differently.

795 total views

## Handling resources that needs to be cleaned up in Python and MATLAB Files, sockets, ports, etc.

Using try/catch to handle resource (files, ports, etc) cleanup is out of fashion in both MATLAB and Python. In MATLAB, it uses a very slick idea based on closures to let you sneak your cleanup function (as a function object) into an onCleanup object that will be executed (in destructor) when that object is cleaned up by being out of scope.

Python does not provide the same mechanism. Instead, it relies on the resource class (like file IO or PySerial) to implement as a Context Manager (has __enter__) and provide the cleanup in the manager’s __exit__ method. Then you use the with keyword with the returned resource object put after as keyword, like this:

with File('test.txt', 'w') as f:
f.write('SPFCCsMfT!')

The body of with-block will not run (and therefore object f won’t be created) if the with-statement throws an exception. Unfortunately, it’s a fill-or-kill (or try/finally) instead of try/catch. So if the resource failed to open, resource object f is simply not created. No other clue is generated. This is what I hate about the with-statement. There are two ways to kind of get around it, but they are not reliable and might cause other bugs if you don’t keep track of the variable names in the local context:

1. Check for the existence of the resource object
if 'f' in dir(): del f   # Avoid name conflicts
with File() as f
print("Done");
if not 'f' in dir():
print('Cannot create file');
2. Use the body code to indicate that the with-block is executed
isSuccess = false     # Signaling
with File() as f
isSuccess = true
print("Done')
if not isSuccess
print("Cannot create file");


3. Back to the old way
try:
f = File();
except:
print("Cannot open file")
else:
cleanup_obj = onCleanup(lambda x = f: x.custom_cleanup())
# run core code that uses resource f


Actually there’s another mess here. In PySerial, creating the Serial object with a wrong port string will throw an exception as well, which with-as statements cannot handle. Therefore you’ll need to do both:

try:
ser = serial.Serial(dev_str)
except:
print(dev_str + " not accessible (either the wrong port of something else opened it)");
else:
with ser:
# meat

If your resource initializer does not have context manager built in, and you want a quick-and-dirty solution (given your cleanup is a one-liner). Use my library (lang.py) that recreates onCleanup():

"""
@author: [2019-04-23] Hoi Wong
"""
class onCleanup:
'''make sure you 'capture' the lambdas by initializing an intermediate running variable
e.g. lambda s=ser: s.close()
lambda: ser.close() will NOT work as ser is not 'captured''''
def __init__(self, functor):
def __del__(self):
self.task()

Then you can use the old way without nesting try/except:

try:
f = File()
except:
print("Cannot open file")
else:
cleanup_obj = onCleanup(lambda x = f: x.custom_cleanup())
# run core code that uses resource f

Check with the provider of your resource initializer to see if context manager is already implemented. Use onCleanup() only when you don’t have this facility and you don’t want to build a whole context manager (even with decorators) for a one-liner cleanup.

587 total views

## Programmatically determining platforms on MATLAB and Python

 MATLAB Python OS (32-bit or 64-bit) computer(‘arch’) platform.machine().endswith(’64’) Engine (32-bit or 64-bit) mexext() platform.architecture()[0].startswith(’64’); OS Type (Broad) ismac/isunix/ispc or computer() platform.system()

588 total views

## Python startup management

The startup script is simply startup.m in whatever folder MATLAB start with.

Now how about Python? For plain Python (anything that you launch in command line, NOT Spyder though), you’ll need to ADD a new environment variable PYTHONSTARTUP to point to your startup script (same drill for Windows and Linux).

For Spyder, it’s Tools>Preferences>IPython console>Startup>”Run a file”:

but you don’t need that if you already have new environment variable PYTHONSTARTUP correctly setup.

864 total views