Rewatching Strang’s lectures on Linear Algebra after many years amazes me. Right now I am watching his lecture on Matrix multiplication. In just the first 20 minutes, he blew my mind. His explanation on how matrix multiplication works (the five ways) make theorems that may appear look very intuitive.
Columns at a time reminds me of embeddings in low dimensional spaces and specifically Johnson–Lindenstrauss lemma and the improvement by Achlioptas. One key insight of the idea of the embedding is that each column of B in the multiplication A * B gives a testimony (linear combination) of the elements in a row of A. The theorems in these seminal works examine the conditions that should apply to B in order to minimize the distortion of A’s projection to a lower dimensional space. Nevertheless, one key insight for these theorems is explained in the third lecture of a first year course in such a wonderful way!
Columns of A x rows of B make low-rank approximations look obvious how they work. I am pretty sure the other three (classic, row of A at a time, by blocks) ways also contribute to things that didn’t pop to my mind directly.
I would definitely suggest to anyone to (re)watch these lectures.