{"id":1738,"date":"2019-05-03T12:18:17","date_gmt":"2019-05-03T20:18:17","guid":{"rendered":"http:\/\/wonghoi.humgar.com\/blog\/?p=1738"},"modified":"2025-12-31T01:33:33","modified_gmt":"2025-12-31T09:33:33","slug":"matlab-and-python-paths","status":"publish","type":"post","link":"https:\/\/wonghoi.humgar.com\/blog\/2019\/05\/03\/matlab-and-python-paths\/","title":{"rendered":"MATLAB and Python paths"},"content":{"rendered":"\n
\n

EDIT<\/strong> (2025-12-31): I finally bit the bullet and wrote an OS system call version which is faster. Please see the link to the Github page<\/a>. It’s no longer called qpath<\/code> but I left the code listing in the blog page to illustrate the thought process.<\/p>\n<\/blockquote>\n\n\n\n

MATLAB’s path()<\/code> is the analog to Python’s sys.path<\/code>. <\/p>\n\n\n\n

However, MATLAB’s path is stored as strings while Python’s sys.path<\/code> is stored as lists (MATLAB’s analog of cells). The tradeoffs between strings vs list\/cells (hetereogenous wrappers)<\/p>\n\n\n\n

\n\n\n\n

To add paths in MATLAB, use the obviously named function addpath()<\/code>. Supply the optional -end<\/code> argument if you don’t want any potential shadowing (i.e. the folder to import has lower priority if there’s an existing function with the same name).<\/p>\n\n\n\n

I generally avoid userpath()<\/code> or the graphical tools because the results are sticky (persists between sessions). The best way is to exclusively manage your paths with startup.m so you always know what you are getting. If you want full certainty, you can start with restoredefaultpath()<\/code> in MATLAB.<\/p>\n\n\n\n

\n\n\n\n

Python’s suggested these<\/a> as equivalents of MATLAB’s addpath()<\/code>:<\/p>\n\n\n\n

sys.path.insert(0, folder_to_add_to_path)  # Add to the front\nsys.path.append(folder_to_add_to_path)     # Add to the end<\/pre>\n\n\n\n
but just like MATLAB’s addpath()<\/code> which works with strings only (not cellstr<\/code>), these Python options do not work correctly with Python lists<\/code> as inputs (typically needed for multiple paths) because lists’ methods in sys.path<\/code> are as primitive as doing [sys.path, new_stuff]<\/code>:<\/p>\n\n\n\n
    \n
  1. This means you’ll end up with list of lists if you supplied Python lists as inputs to the above
    (MATLAB will throw an exception if you try to feed it with cellstr<\/code> instead of polluting your path space with garbage)<\/li>\n\n\n\n
  2. This also means it doesn’t check for duplicates! It’ll keep stacking entries!<\/li>\n<\/ol>\n\n\n\n
    To address the first problem, we use sys.path.extend()<\/code> instead. It’s like doing addpath(..., '-end')<\/code> in MATLAB. If you want it to be inserted at the front (higher priority, shadows existing), you’ll need sys.path = list_of_new_paths + sys.path<\/code>. <\/p>\n\n\n\n
    MATLAB is the other way round: if you have a cell (python’s version of list) of paths to enter, you can make a path string by using pathsep<\/code>:<\/p>\n\n\n\n
    addpath(strjoin(cellstr_of_paths, pathsep)))<\/pre>\n\n\n\n
    Note that\u00a0 sys.path.extend()<\/code> expect list, sets or tuples so if you feed it a string, which Python will consider it a list of characters, you will get a bunch of one character paths inserted!<\/p>\n\n\n\n
    On the other hand, DO NOT TRY to get multipe path entries to work with insert<\/code>\/append <\/code>by flattening the list of path into one string becaust sys.path<\/code> is not a string and it’s not smart enough to auto switch depending on data types! In other words, sys.path.append( ';'.join(path_list))<\/code> is INCORRECT as it’ll add an invalid path when more than 1 path is in the path_list<\/code>. Python recognize sys.path<\/code> as a list, NOT a one long string like MATLAB\/Windows path<\/code>, despite insert()<\/code> and append()<\/code> accepts only strings!<\/p>\n\n\n\n
    MATLAB’s uses string, which is a much more efficient path representation, so many duplicate entries won’t slow down the system much. Path order do affect shadowing (which functions with the same name gets exposed) though.<\/p>\n\n\n\n
    The second problem (which does NOT happen in MATLAB) is slightly more work. You’ll need to subtract out the existing paths before you add to it so that you won’t drag your system down by casually adding paths as you see fit. One way to do it:<\/p>\n\n\n\n
    def keep_only_new_set_of_paths(p):\n    return set(p)-set(sys.path)\n<\/pre>\n\n\n\n
    \n\n\n\n
    You should organize your programs and libraries in a directory tree structure and use code to crawl the right branch into a path list! Don’t let the lack of built-in support to tempt you to organize files in a mess<\/strong>. Keep the visuals clean as mental gymnastics\/overheads can seriously distract you from the real work such as thinking through the requirements and coming up with the right architecture and data structures. If you constantly need to jump a few hoops to do something, do it only once or twice using the proper way (aka, NOT copying-and-pasting boilerplate code), and reuse the infrastructure.<\/p>\n\n\n\n
    At my previous workplaces, they had dozens and dozens of MATLAB files including all laying flat in one folder. The first thing I did when I join a new team is showing everybody this idiom that recursively adds everything under the folder into MATLAB paths:<\/p>\n\n\n\n
    addpath(genpath())<\/pre>\n\n\n\n
    Actually the built-in support for recursive directory search sucks for both MATLAB and Python.\u00a0 Most often what we need is just a list of full paths for a path pattern that we search recursively, basically dir\/w\/s *.<\/code>\u00a0None of them has this right out of the box. They both make you go through the comprehensive data structure returned (let it be tuples from os.walk()<\/code>\u00a0in Python or dir()<\/code> in MATLAB) and massage the output to a simple path list.<\/p>\n\n\n\n
    genpath()<\/code> itself is slow and ugly. It’s basically a recursive wrapper around dir()<\/code> that cleans up garbage like '.'<\/code> and '..'<\/code>.\u00a0 Instead of getting a newline character, a different row (as a char<\/code> array) or a different cell (as cellstr<\/code>), you get semi-colons (;<\/code>) as pathsep<\/code> in between. Nonetheless, I still use genpath()<\/code> because despite I have written faster recursive path tools in my own libraries, I’ll need to load the library first in my startup file, which requires a recursive path tool like genpath()<\/code>. This bootstraps me out of a chicken-and-egg problem without too much ugly syntax.<\/p>\n\n\n\n
    \n\n\n\n
    Most people will tell you to do a os.walk()<\/code> and use listcomp to get it in the typical full path form, but I’m not settling for distracting syntax like this. People in the community suggested using glob<\/code> for a relatively simple alternative to genpath()<\/code><\/p>\n\n\n\n
    Here’s a cleaner way:<\/p>\n\n\n\n
    def list_subfolders_recursively(p):\n    p = p + '\/**\/' \n    return glob.glob(p, recursive=True);<\/pre>\n\n\n\n
    It’s also worth noting that Python follows Linux’s file search pattern where directory terminates with a filesep (\/<\/code>) while MATLAB’s dir()<\/code> command follows the OS, which in Windows, it’s *.<\/code>.<\/p>\n\n\n\n
    Both MATLAB and Python uses **<\/code> to mean regardless of levels, but you’ll have to turn on the recursive=True<\/code> in glob<\/code> manually. **<\/code>\u00a0is already implied to be recursive in MATLAB’s dir()<\/code> command.<\/p>\n\n\n\n
    \n\n\n\n
    Considering there’s quite a bit of plumbing associated with weak set of sys.path<\/code> methods provided in Python, I created a qpath.py<\/code> next to my startup.py<\/code>:<\/p>\n\n\n\n
    ''' This is the quick and dirty version to bootstrap startup.py\nShould use files.py that issue direct OS calls for speed'''\n\nimport sys\nimport glob\n\ndef list_subfolders_recursively(p):\n    p = p + '\/**\/' \n    return glob.glob(p, recursive=True);\n\ndef keep_only_new_set_of_paths(p):\n    return set(p)-set(sys.path)\n\ndef set_of_new_subfolders_recursively(p):\n    return keep_only_new_set_of_paths( list_subfolders_recursively(p) )\n\ndef add_paths_recursively_bottom(p):\n    sys.path.extend(set_of_new_subfolders_recursively(p));\n\ndef add_paths_recursively_top(p):\n    # operator+() does not take sets\n    sys.path = list(set_of_new_subfolders_recursively(p)) + sys.path;<\/pre>\n\n\n\n
    In order to be able to import my qpath<\/code> module at startup.py<\/code> before it adds the path, I’ll have put qpath.py<\/code> in the same folder as startup.py<\/code>, and request startup.py<\/code> to add the folder where it lives to the system path (because your current Python working folder might be different from PYTHONSTARTUP<\/code><\/a>) so it recognizes qpath.py<\/code>.<\/p>\n\n\n\n
    This is the same technique I came up with for managing localized<\/em><\/strong> dependencies in MATLAB: I put the dependencies under the calling function’s folder, and use the path of the .m<\/code> file for the function as the anchor-path to add paths inside the function. In MATLAB, it’s done this way:<\/p>\n\n\n\n
    function varargout = f(varargin)\n  anchor_path = fileparts( mfilename('fullpath') );\n  addpath( genpath(fullfile(anchor_path, 'dependencies')) );\n  % Body code goes here<\/pre>\n\n\n\n
    Analogously,<\/p>\n\n\n\n