MATLAB Features: Persistent Excel ActiveX (DCOM) for xlsread() and xlswrite() R2015b

xlsread() and everything that calls it, such as readtable(), is terribly slow, especially when you have a boatload of Excel files to process. The reason behind it is that xlsread() closes that DCOM handle (which closes the Excel COM session) after it finishes, and restart Excel (DCOM) again when you call xlsread() again to load another spreadsheet file.

That means there’s a lot of opening and closing of the Excel application if you process multiple spreadsheets. The better strategy, which is covered extensively in MATLAB’s File Exchange (FEX), is to have your program open one Excel handle, and process all the spreadsheets within the same DCOM handle before closing it.

This strategy is quite overwhelming for a beginner, and even if you use FEX entries, you still cannot get around the fact that you have to know there’s a handle that manages the Excel session and remember to close it after you are done with it. Nothing beats having xlsread() do it automatically for you.

Starting with R2015b, the Excel DCOM handle called by xlsread() is now persistent: that after you make the first call to xlsread(), Excel.exe will stay in the memory unless you explicitly clear persistent variables or exit MATLAB, so you can reuse them every time xlsread() or xlswrite() is called. Finally!

The code itself is pretty slick. You can find it in ‘matlab.io.internal.getExcelInstance()’. Well, I guess it’s not hard to come up with it, but I guess in TMW, they must have a heated debate about whether it’s a good idea to keep Excel around (taking up resources) when you are done with it. With the computation power required to run R2015b, an extra Excel.exe lying around should be insignificant. Good call!

 

Loading

MATLAB Techniques: Who’s your daddy? Ask dbstack(). Unusual uses of dbstack()

Normally having your function know about its caller (other than through the arguments we pass onto the stack) is usually a very bad idea in program organization, because it introduces unnecessary coupling and hinders visibility.

Nonetheless, debugger is a built-in feature of MATLAB and it provides dbstack() so you have access to your call stack as part of your program. Normally, I couldn’t come up with legitimate uses of it outside debugging.

One day, I was trying to write a wrapper function that does the idiom (mentioned in my earlier blog post)

fileparts( mfilename('fullpath') );

because I want the code to be self-documenting. Let’s call the function mfilepath(). Turns out it’s a difficult problem because mfilename(‘fullpath’) reports the path of the current function executing it. In the case of a wrapper, it’s the path of the wrapper function, not its caller that you are hoping to extract its path from.

In other words, if you write a wrapper function, it’s the second layer of the stack that you are interested in. So it can be done with dbstack():

function p = mfullpath()
  ST = dbstack('completenames');
  try
    p = ST(2).file;
  catch
    p = '';
  end

Since exception handling is tightly knit into MATLAB (unlike C++, which you pay for the extra overhead), there aren’t much performance penalty to use a try…catch() block than if I checked if ST actually have a second layer (i.e. has a non-base caller). I can confidently do that because there is only one way for this ST(2).file access operation to fail: mfullpath() is called from the base workspace.

Speaking of catchy titles, I wonder why Loren Shure, a self-proclaimed lover of puns and the blogger of the Art of MATLAB, haven’t exploited the built-in functions ‘who’ and ‘whos’ in her April Fools jokes like

whos your daddy
who let the dogs out

Note that these are legitimate MATLAB syntax that won’t throw you an exception. Unless you have ‘your’, ‘daddy’, ‘let’, ‘the’, ‘dogs’, ‘out’ as variable names, the above will quietly show nothing. It’d be hilarious if they pass that as an easter egg in the official MATLAB. They already have ‘why’,

why not

Error using rng (line 125)
First input must be a nonnegative integer seed less than 2^32, 'shuffle', 'default', or generator settings captured previously using S = RNG.

Error in why (line 10)
 dflt = rng(n,'v5uniform');

 

 

 

Loading

Ebay customer support’s on-hold music: Weird Al?!!

A couple of months ago, before this blog started, I called eBay’s customer support and I was put on hold for a whole hour. It would have made my day miserable, but the on-hold music they played made my day: it’s Weird Al Yankovic’s parody song eBay!

Couldn’t stop chuckling for the first half hour despite the gruesome wait … I just couldn’t get enough of it even if loops every minute or two! I kept thinking: this can’t be true!

Yes, the eBay’s official hotline plays Weird Al’s parody of eBay to their customers while they wait! They didn’t even bleep out the word “crap” as in “this crap, shows up in, bubble wrap, almost every day“. Well, I guess they have to, as “crap” is essential to the spirit of the entire parody!

I love these people. They really have a sense of humor, and they are willing to make fun of themselves!

Loading

Title of this blog site

Initially I started with “TMI: Too Much Information” as the title of this blog, given that my plan was to put fragments of technical information or insights I came across that might be useful for solving problems. That means the blog posts contain more than what you want to know, unless you are looking to solve a specific problem with the help of the post or you are just outright nerdy.

But soon I realized I have some non-technical stuff like gags, music, and the technical stuff covers more than just electronic measurement instruments, so I need a title that’s less common and more catchy.

Today, I came across a reddit post, which user “llllIlllIllIlIRogue Sysadmin “says:

If you don’t they’ll just hear jargon and glaze over completely and not even try to follow you. If you draw a pretty layout of everything, though, they’ll make some token effort to follow along.

They’ll still get lost but now you’re not just a nerd rambling… you’re a rambling nerd with a plan.

Very catchy! Also, I liked the original comment because it covers:

  • Passion for geeky topics
  • It’s important to communicate well so that people will bother to follow what we have to say.

I did a google search with quotes “rambling nerd with a plan” and only one entry: the original post, showed up, so it’s not a commonly used phrase. I’ll take it 🙂

Loading

MATLAB Quirks: struct with no fields are not empty

As far as struct() is concerned, I’m more inclined to using Struct of Array (SoA) over Array of Structs (AoS), unless all the use cases screams for SoA. Performance and memory overhead are the obvious reasons, but the true motivation for me to use SoA is that I’m thinking in terms of table-oriented programming (which I’ll discuss in later posts. See table() objects.): each field of a struct is a column in a table (heterogeneous array).

Since a table() is considered empty (by isempty()) if it has EITHER 0 rows INCLUSIVE OR 0 columns (no fields) and the default constructor creates a 0 \times 0 table, I thought struct() would do the same. NOT TRUE!

First of all, the default constructor of struct() gives ONE struct with NO FIELDS (so it’s supposed to correspond to a 1 \times 0 table). What’s even harder to remember is that struct2table(struct()) gives a 0 \times 0 table.

The second thing I missed is that a struct() with NO fields is NOT empty. You can have 3 structs with NO fields! So isempty(struct()) is always false!

I usually run into this problem when I want to seed the execution with an empty struct() and have the loop expand the fields if the file has contents in it, and I’ll check if the seeded struct was untouched to see if I can read data from the file. Next time I will remember to call struct([]) instead of struct(). What a trap!

At the end of the day, while struct is powerful, but I rarely find AoS necessary to do what I wanted once table() is out. AoS has pretty much the same restrictions as in table() that you cannot put different types in the same field across the AoS, but table allows you to index with variables (struct’s field) or rows (struct array index) without changing the data structure (AoS <-> SoA). So unless it’s a performance critical piece of the code, I’ll stick with tables() for most of my struct() needs.

 

Loading