breakpoint(): Python’s version of MATLAB’s keyboard() command
callable(): Like MATLAB’s isfunction() but it really checks if there’s a __call__ method
getattr()/hasattr(): MATLAB’s getfield()/isfield(). The 3rd parameter of getfield() is a shortcut to spit out a default if there’s no such field/attribute, which MATLAB doesn’t have
globals()/locals(): more convenient than MATLAB because the whole workspace (current variables) are accessed as a dictionary in Python by calling locals() and globals()
id(): memory address of the item where the variable (reference) is pointing to. Think of it as &x in C.
isinstance(): MATLAB’s isa()
next(): Python favors not actually computing the values until needed so instead it offers a generator (forward iterable) function that spits out one value at each time you kick it with next() and you can’t go back.
chr()/ord(): analogous to MATLAB’s char()/double() cast for characters
Python’s exponentiation is **, not ^ like most other languages (C does not have exponentiation symbol, and ^ was used for xor)
print(…, flush=false) allows a courtesy flush
repr(): MATLAB’s version of disp(), also overloadable standard interface
slice(): MATLAB’s equivalent of colon() special interface
Context Manager
@contextlib.contextmanager decorators basically splits a set-try-yield-finally boilerplate function into 3 parts: __enter__ (everything above yield), BODY (where the yield goes to) and __exit__ (everything below yield), since a with-as statement is a rigidly defined try-finally block, roughly like this:
with EXPR as f:
BODY(using f)
__enter__: f=EVAL(EXPR)
try:
# f isn't evaluated till yield
yield f # Goes to BODY
finally:
__exit__: cleanup(f)
Python uses garbage collectors, so onCleanup() might often work, but it’s not guaranteed to. So any code based on that should not be in production
Answer: Context Manager, a glorified try-catch (more specifically try-finally) block with a rigid structure. It’s a pain in the butt and not fun to deal with if you want to deviate from the native ContextManager that came with the resource opener
‘switch’-case is back as ‘match’-case (the advanced uses are different)
New Python finally supports it, ‘switch’ in C is called ‘match‘ in Python and there are many handy and intuitive syntax just like in MATLAB! Horray!
If you try to do anything fancy with mutables in the cases, be careful about the side effects!
Pass by Variable (Copy-on-Write) like MATLAB
Variables are by large references in Python. Everything including integers are some sort of classes (which are in turn dictionaries with special treatment to certain key names). The garbage collector scans for the last guy using that part of memory not referencing it anymore before cleaning it.
Python even have one ‘None’ for the entire universe with a gazillion things going on pointing to the same memory address where None is stored (that’s why None is idoimatically checked by ‘is’ keyword which checks the address for speed instead of ‘==’ which actually verify the contents for speed). If you look up the reference count (see garbage collector) for commonly used numbers like 1s and 0s, there are thousands of ‘users’ of it!
With C++, it’s a mixture. Complex objects are usually passed by reference for performance reasons but simple structs and data types can be passed as variables (C/C++’s nomenclature calls the non-reference/pointers variables though technically references/pointers are just the same integers identified as addresses) that gets cloned and destroyed when they move across function (stack) boundaries.
In MATLAB, they want it industrial strength, so that’d rather not allow anything insidious/non-transparent to happen in your code by keeping it all pass-by-variable, that is everything is supposed to be treated as different copies as it crosses function boundaries. For performance reason, they figured if you passed a big matrix just for the function to read, MATLAB doesn’t really have to clone that so under the hood you can peak the same matrix that belong to the caller. Once your function changes the contents (they are pretending that it’s a separate copy so of course you can), MATLAB painfully makes a whole copy of it (copy-on-write) which you then have to lug the 2nd (modified) copy around when it travels past function boundaries.
Python takes this idea a lot further by having anything that’s exactly the same (including None or string literals) to point to the same object until you ask to change the contents, then it makes a new copy for you to change and point to the new copy specifically for the variable name you are referencing with.
Answer: The way Python prevents variables passed as parameters passed into a function from getting modified is to separate variables into mutable (lists, dicts, sets), and immutable (tuples, frozendict is a package right now, frozenset, numerics, strings) types. Anything immutable going past the function boundary gets their own local copy.
Classes Boundaries
MATLAB and C++ has stringent access control, but not in Python. There’s not even const correctness. Just signal your intention with variable naming schemes like all caps and __ prefixes.
C++ do not separate helper functions and class (non-instance) methods. What C++ called static members are really just glorified free functions and global variables tucked under the namespace that happened to be in the class (classes started off as namespace for structs then people add features like overloading and dispatch mechanisms). partitioning the global workspace. Whether a scoped helper function call a scoped variable that happened in the same namespace is nothing special to C++
Python does separate these two concepts though. Class method in Python (decorated by @classmethod), on the other hand, are equivalent to C++’s static methods which they are not allowed to touch anything instance-specific, but they can access anything class-specific. Helper functions, which is called ‘Static Method‘ (decorated by @staticmethod) in Python cannot even touch anything specific to the class.
Variable arguments
C++ doesn’t generally do variable arguments because it defeats signature-based method overloading that uses a function signatures (which is a list of your argument types) to figure out which function to dispatch.
MATLAB uses cell to pack variable arguments. The common idioms are varargin{:} and [varargout{1:nargout}]. To accommodate variable arguments, MATLAB have to give up method overloading but they still have a little bit of it left: they do dispatch based on the first argument type and it’s very useful in avoiding a lot of stupid switching by detecting data types: just use a consistent function name interface and have each data type implement its own method with the same name.
In Python, there’s no such thing as multiple outputs (return variables) on Python: you output a list and it always gets unpacked (just like MATLAB’s deal() function) when you type a list out on left hand side. If the left hand side is a singleton, it will get the full list that’s still packed. If you write out the elements (which makes the left hand side a list), the returned list will have elements assigned to the left hand side depending on your syntax.
This is often a point of agony deciding on output format when I develop MATLAB code. Apparently TMW wondered the same thing too because their own factory code is all over the place on this too. Most of the time it’s not a good idea to have a context-dependent (depending on how many outputs the user supplied) even if you can technically do that by detecting nargout in MATLAB.
Answer: My recommendation is to make sure the simple, most common case got priority, and stuff the juicy side info in packaged data structures (such as array or cells/monads/lists) and stick to a fixed output format whether you are in Python or MATLAB
Late Binding in Lambda / Anonymous Functions: Capture it!
This often throw people off in Python. MATLAB uses early binding, which means when you created that anonymous function (aka lambda), the free variables (parameters that are not running, aka the input arguments to the lambda/anonymous-function) captured the snapshot of the local workspace at the moment the lambda/anonymous-function was created!
Python on the other hand, uses the same approach as C++: late binding. This means you have to explicitly capture (make a snapshot copy) the free variables if you want to associate it with the values when the anonymous function (lambda) was created, not to wait until the lambda was actually called/used to look for what values to use in the free variables.
P = 612
# P was not captured, thus late binding
f = lambda x : print{f'input/running:{x}, param/free:{P}')
# P was captured as p, thus early binding
g = lambda x, p=P: print{f'input/running:{x}, param/free:{p}')
P = 721
f(8964) # shows "input/running: 8964, param/free: 721"
g(8964) # shows "input/running: 8964, param/free: 612"
This article is not for Pythonisms (like ContextManager), etc, the way things should normally be done in Python, but the more non-obvious way to solve problems or new features that are available specifically in Python.
Use / and * argument as separator for different forms of parameter entries!
There is special syntax to separate positional-only, positional/keyword, and keyword-only parameters.
Make a variant of existing class/object
This ninja technique is useful when you want to keep the object mostly the way it is but add/override a few things you think it didn’t do right without inheriting or use composition (hide a copy of object as a member) and write a proxy mirror for every member of it.
In C++, this situation is often used when you want to modify a concrete class that doesn’t have a virtual destructor, most notoriously STL which you are not supposed to inherit from (or else the client might pass a pointer to the parent/base so the child object’s destructors are not called as there are no vtables to keep track of which method to dispatch). In C++, this is often the few use cases that calls for private inheritance.
In Python, because everything is a recursive dict that specially named (magic) functions are recognized, there is a __getattr__ method that gets called whenever a member is accessed, which is the case when the member is called (in Python you simply get the functor as an attribute, aka value in the key-value pairs, in the dictionary and add brackets to call it). This means you can re-route what attributes (members) are returned simply by overloading this method!
If you can overload __getattr__, it also mean you can redefine the member interface of your entire class! So a strategy to make a class have the same exact interface as another class is to hide a copy/reference to the underlying class and re-route the __getattr__ to the underlying object’s __getattr__! Here’s the gist of it:
This can be improved a little bit. Just pick a member name that won’t clash with the underlying object’s attributes/members and simply return the ‘hidden’ object when specifically requested, not self.__obj.__obj.
This is a very powerful luxury that makes Python so lovable if you are not here for industrial strength programming. MATLAB’s classes are hard-wired to your class definition .m file, not something you can update on the fly as you please. Any attempts to do so (aka breaking the safeguards) are Undocumented MATLAB territory where you mess with the Java under the hood and change the metadata property to fool the objects to do what you want.
[This is probably common knowledge that are repeated in many different places, but I want to arrange it in the perspective that helps my other article to explain how this influence the idea of ContextManager in Python and why ContextManager is still a clumsy way and there is a much neater way to do tackle this classic cleanup problem]
In C, there’s no built-in exception handling. Yet the goal for cleanups is for all manually checked and handled conditions (‘exceptions’) to land exactly in the same graveyard where the resources stands a chance to be released before the program ends. This screams goto (or longjump which is a non-local goto that can march outside the current function) and indeed it’s the only legit use of the goto statement I know that doesn’t make the code more error prone and confusing by littering the end of all your loops with if(error){break;}.
With this feedback approach, all hell breaks loose if you later add another layer and forget to add this. The mistake will break the feedback chain and the code continues to run in the layer after the end of the the while loop you forgot to place the check in, which is an insidious bug if the unwanted execution are benign under most situations.
The break clause will also become meaningless (and compiler invalid) if you convert your loops to non-loops or when you work in the top layer which is likely not inside a loop (even if you are in a bare metal embedded system with a while(0) loop, you don’t want to break that either).
break is a black-list approach which denies the rest of the code in the loop from running when the first error struck. Without break, you can do the reverse (the white-list approach) and put all code after the first check in if(!error) check blocks to authorize their execution instead
One hack to use break statement at the top-layer (no-loop) is to wrap the top layer with a do{BLOCK}while(false); loop which runs once, but the intention is not intuitive from the code so I wouldn’t do this to other programmers who don’t know the idiom without making a TRY-CATCH-FINALLY macro.
// Messy approach without do-while loop wrapper hack in top layer
// Convention: error=0 (false) means success
// The error code matches the check# it fails in this example
int error = 0;
if( int* f = grab_resource_and_spit_zero_if_fail() )
{
// This is hell of messy if you are not in a loop
// that can take advantage of break-statement
//
// If you don't use breaks (which require it to be in a loop),
// you have to explicitly surround all code in if(!error) blocks
...
// Approach 1: Nesting
// Upside: Visually draws out what the logic relation is
// when you're doing checks all over the place
// It's hard to get it conceptually wrong
// (i.e. can debug blindly with mere semantics)
// Downside: more checks means more nests
// you either have excessive indentation
// or have fun tracking brackets
if( !is_fail_1() )
{
...
if( !is_fail_2() )
{
...
if( !is_fail_3() )
{
(you get the idea)
...
} else { error=3; }
} else { error=2; }
} else { error=1; }
// Approach 2: Linear approach
// Idea: Surround everything under if(!error) check.
// Error code will stick to the first error as
// any non-zero error will short-circuit the
// checks after it so the original error code stays
// Upside: No nesting. Easy to follow to recipe consistently
// WARNING: Thou shalt not be tempted to modify error code
// in if(!error) blocks!
// Downside: If you violate the cardinal rule above, unintended
// chunks of code in if(!error) blocks might run and
// it's hard to debug/discover
if(!error)
{
...
}
// this idiom is a branchless way to do
// if(!error){ if(is_fail_13){error=13;} }
// The error==0 should be placed in front to
// take advantage of the short-circuit evaluation
// to avoid actually running the check if there's
// pre-existing error
error = (error>0)*error + (error==0 && !is_fail_13())*13
if(!error)
{
...
}
error = (error>0)*error + (error==0 && !is_fail_14())*14
if(!error)
{
...
}
error = (error>0)*error + (error==0 && !is_fail_15())*15
if(!error)
{
...
}
// Now you are in the first loop so you are allowed to use break
for(...)
{
...
(is_fails 16 .. 41)
...
if( is_fail_42() )
{
error = 42;
break;
}
..
while(...)
{
...
(deep down in the nest)
...
(is_fails 43 .. 101)
...
if( is_fail_102() )
{
error = 102;
break;
}
...
}
if(error) { break; }
...
}
if(error) { break; }
...
clean_the_f_up(f);
}
// goto approach
if( int* f = grab_resource_and_spit_zero_if_fail() )
{
...
(is_fails #1 .. 18)
...
for(...)
{
...
(is_fails 19 .. 41)
...
if( is_fail_41() )
{
goto graveyard;
}
...
while(...)
{
...
(deep down in the nest)
...
(is_fails 43 .. 101)
...
if( is_fail_102() )
{
goto graveyard;
}
...
}
...
}
...
graveyard:
clean_the_f_up(f);
}
By jumping to the graveyard, we don’t need to litter the code with a long chain of error message/signal feedback and/or guard all chunks of code with if(!error) blocks, which is messy because it’s basically re-inventing a lightweight custom exception handling infrastructure that propagates the fault back to the top and give the intermediate layers a chance to intercept it.
As long as you are not using the goto approach to do complicated maneuvers and keep it simple: all faults go to the same bucket, no ifs-and-buts or detours (i.e. no code elsewhere/in-between can intercept the flow), it isn’t spaghetii code: there are no complicated code flow graphs, just every branch pointing to the same destination in one step. You don’t need to feel guilty about using the goto approach if your error handling flow is like this:
The idea of bundling code and program into a layout (classes) and injecting it with different data (objects) leads to a ‘new’ way (newer than C) of organizing our programs through the worldview of objects.
Every unit is seen as
a state: all member variables
possible actions: methods = member functions.
that is ready to interact with other objects.
Encapsulation (through access control)
The first improvement upon OOP is privacy (data encapsulation). You can have finer controls of what to share and with who. In C++, your options are:
public: everybody
private: only within yourself (internal use)
protected: only shared with descendants (inheritance discussed below)
Granting certain class as friend (anywhere in the class declaration with friend class F) exposes the non-public sections specifically to the friend F. This is often a ‘loophole’ to access control that finds few legitimate uses other than testing.
friend functions are traditionally used in binary (2-input) operator overloading, but the modern wisdom is to screw it and just leave it out there as free functions!
protected has very few good uses other than preventing heap deletethrough base pointer non-polymorphically (child destructor not called: BAD) by making the base destructor non-public (i.e. meaning it’d be impossible to have base objects on stack) while letting the child chain the parent’s destructor (child can’t access it if it’s marked as private).
The second improvement is to allow classes to build on top of existing ones. What gets interesting (and difficult) is when the child ‘improve’ on the parent by either by replacing what they have (member variables) and what they do (methods) with their own.
Inheritance AT LEAST always inherits an interface (can optionally inherit implementation).
Base implementation MUST NOT be inherited
pure virtual methods
Base implementation inherited by default
virtual
Base implementation MUST be inherited
non-virtual (and not shadow it)
Shadowing
Whenever the member (function or variable) name is used in any form (even with different argument types or signatures), the parent member with the same name will be hidden. The behavior is called shadowing, and it applies unless you’ve overridden ALL versions (signatures) of virutal parent methods which shares the same function name mentioned in child.
Any non-overriden method with the same name as the parent appearing in the child will shadow all parent methods with the same name regardless of whether they are declared virtual and overriden at child.
You can unhide parent methods with the same name (but different signature) by using Parent::f(..) declared at the child class.
Shadowing implies there’s always one parent version and one child version stored separately under all conditions {static or non-static}x{function or variable}
Static members don’t really ‘shadow’ because there’s only one global storage for each (parent and child) if you declare the same variable name again in the child. There’s nothing to hide because you cannot cast or slice a namespace! With static members, you have to be explicit about which class you are calling from with SRO like Parent::var or Child::var so there’s no potential for ambiguities.
Overriding
Just like C, C++ uses static binding that takes the programmer’s word for it for their declared types, especially through handles. Overriding is a concept only needed when you plan to upcast your objects (child accessed through pointer/reference) to handle a broader class of objects but intend to the underlying object’s own version (usually child) of the methods (especially destructors) called by default.
We do this by declaring the parent method virtual and implement the child versions (must be of the same function signature). Overriding only make sense for non-static methods because
data members cannot be overridden (it’d confusing if it’s possible. We down-delegate functions/behavior but not the data/state). It’s better off hiding data members behind getters/setters to declare the intention.
static members and methods behaves like static variable/functions (living in .data or .bss) using namespaces, so we can only refer to them with SRO by the class names like Parent::f() and Child::a, not a class type like Parent p; p.f() and Child c; c.a. There’s no object c for you to upcast to Parent so there’s place for polymorphic behavior.
Overriding involves leaving clues in objects so the upcasted references can figure out the correct methods of the underlying objects to call. In C++ it’s done with having a vtable (pointers to overridable methods, often stored in .rodata with string literals) for each class in the hierarchy and each object contains a pointer to the vtable that matches its underlying class.
[38] virtual only applies to methods’ signatures (function name and the data types in the argument list). vtable do not keep track of argument’s default values (if assigned) for efficiency (it’ll always read the static upcast, aka parent methods’ default values).
