Official Dart docs is sometimes too simple to provide ultimate answers for various language questions. I discoverd an alternative syntax for named lambda here and here. In Dart,
(args) => expr
is the shorthand for
(args) {
return expr
}
So the Lambda-style (C++11) names variable by assigning it to a messy functor type (inferred with var in Dart and auto in C++11):
var f = (args) => expr
Which can also be rewritten using C-style function pointer-like declaration except
the body expr is included (which is not allowed in C),
returnmust be explicit in curly braces block.
arrow notation => takes whatever the expr evaluates to (which can be null for statements like print)
it’s simply the function name f in Dart instead of pointer (*f) in C
Since MATLAB R2015b, there’s a new feature called repelem(V, dim1, dim2, ...) which repeats each element by dimX times over dimension X. If N (dim1) is scalar, each V is uniformly repeated by N times. If N is a vector, it has to be the same length as V and each element of N says how many times the corresponding element in V is repeated.
The scalar case (repeat uniformly) can be emulated by a Kronecker product multiplying everything with 1 (self):
kron(V, ones(N,1))
Just replace all the elements b with 1 so we are left with elements of A repeating the way we wanted
Kron method is conceptually smart but it has unnecessary arithmetic (multiply by 1). Nonetheless this method is reasonably fast until TMW finally developed a built-in function for it that outperforms all the tricks people have accumulated over decades.
The vector case (each element is repeated a different number of times according to vector N) is basically decoding Run-Length Encoding (RLE), aka counts to placements, which you can download maturely written programs on MATLAB File Exchange (FEX). There are a bunch of cumsum/diff/accumarray/reshape tricks but at the end of the day, they are RLE decoding in vectorized forms.
This blog post is development in process. Will fill in the details missing details (especially pandas) later. Some of the MATLAB syntax are inaccurate in the sense that it’s just a description that is context dependent (such as column names can be cellstr, char string or linear/logical indices).
From data relationship point of view, relation database (RDMBS), heterogenous data tables (MATLAB’s dataset/table or Python Panda’s Dataframe) are the same thing. But a proper database have to worry about concurrency issues and provide more consistency tools (ACID model).
Heterogenous data tables are almost always column-oriented database (mainly for analyzing data) where MySQL and Postgres are row-store database. You can think of column-store database as Struct of Arrays (SoA) and row-store database as Array of Struct (AoS). Remember locality = performance: in general, you want to put the stuff you frequently want to access together as close to each other as possible.
T=[T; {v1, v2, ...}] (Cannot default for unspecified column*)
update records/elements
UPDATE table SET column = content WHERE row_cond
T.(col)(row_cond) = content
New table from selection
SELECT vars INTO t2 FROM t1 WHERE rows
T2 = T1(rows, vars)
clear table
TRUNCATE TABLE t
T( :, : )=[]
delete rows
DELETE FROM t WHERE cond (if WHERE is not specified, it kills all rows one by one with consistency checks. Avoid it and use TRUNCATE TABLE instead)
T( cond, : ) = []
* I developed sophisticated tools to allow partial row insertion, but it’s not something TMW supports right out of the box. This involves overloading the default value generator for each data type then extract the skeleton T( [], : ) to identify the data types.
Core database concepts:
Concepts
SQL
MATLAB (table/dataset)
Pandas (Dataframe)
linear index
CREATE INDEX idx ON T (col)
T.idx = (1:size(T,1))'
group index
CREATE UNIQUE INDEX idx ON T (cols)
[~, T.idx] = sortrows(T, cols) (old implementation is grp2idx())
set operations
UNION INTERSET
union() intersect() setdiff(), setxor()
sort
ORDER BY
sortrows()
unique
SELECT DISTINCT
unique()
reduction aggregration
F()
@reductionFunctions
grouping
GROUP BY
Specifying ‘GroupingVariables’ in varfun(), rowfun(), etc.
Function programming concepts map (linear index), filter (logical index), reduce (summary & group) are heavily used with databases
Formal databases has a Table Definition (Column Properties) that must be specified ahead of time and can be updated in-place later on (think of it as static typing). Heterogenous Data Tables can figure most of that out on the fly depending on context (think of it as dynamic typing). This impacts:
data type (creation and conversion)
unspecified entries (NULL). Often NaN in MATLAB native types but I extended it by overloading relevant data types with a isnull() function and consistently use the same interface
default values
keys (Indices)
SQL features not offered by heterogenous data tables yet:
column name aliases (AS)
wildcard over names (*)
pattern matching (LIKE)
SQL features that are unnatural with heterogeneous data tables’ syntax:
implicitly filter a table with conditions in another table sharing the same key. It’s an implied join(T, T_cond)+filter operation in MATLAB. Often used with ANY, ALL, EXISTS
Fundamentally heterogenous data types expects working with snapshots that doesn’t update often. Therefore they do not offer active checking (callbacks) as in SQL:
Invariant constraints (CHECK, UNIQUE, NOT NULL, Foreign key).
MATLAB’s dataset/table objects’ internals often involves identifying unique contents and assigning a unique (grouping) index to it so the indices can be mapped or joined without actually going through the contents of each row.
In the old days when I were using dataset(), the first generation of table() objects before the rewrite, there is a tool called grp2idx() which assigns the same number to identical items regardless of data types. It was part of Statistics Toolbox (needs to pay extra for it) and it does not work if you have multiple columns that you want to assign an unique index unless the ROWS are identical.
Upon inspection. grp2idx() is overrated. There are two ways to get it without paying for the toolbox:
double(categorical(X)): cast a categorical type (technically you can use nominal/ordinal, but it’s part of statistics toolbox)
Use the 2nd output argument for sort() or sortrows() function. I recommend sortrows() because it’s can be overloaded on table() objects and it works on multiple rows.
Namecheap already provided instructions to connect MySQL Workbench client for its shared hosting, which involves SSH-tunneling because they disallowed direct MySQL connection out of security concerns.
So here’s basically the logistics:
SSH to your namecheap hostname (can use your domain name) at SSH port 21098
Tunnel listens to Port 5522 and forward it to localhost (the client itself) at MySQL Port 3306
Instead of connecting directly to the {namecheap shared hosting server}:3306, connect to the localhost:3306
It’s a little confusing on how to do it on DBeaver because “Advanced settings” is hidden by default which you will need. The name ‘local client’ (source) vs ‘remote’ (destination) in the dialog box is confusing. It’s actually equivalent to
bind_address can be left blank. If you are paranoid and don’t want other machines to use your current MySQL client machine as a gateway (they tunnel into your machine to use the tunnel you are currently establishing), set (aka bind) it to localhost, or you can bind it to the client’s network adapter’s IP which you want to allow machines on a trusted network to use this MySQL client computer as a gateway.
For some reason (I suspect it’s IPv6), “Remote host” needs to be set to the loopback adapter 127.0.0.1 (cannot use the special hostname ‘localhost‘).
Remember MySQL’s username and password is the special database-only login credentials you created at cPanel.
