Old and New Matrix Algebra Useful for Statistics

A concise reference on advanced matrix theory, including:

• an easy way to compute matrix derivatives and second derivatives
• a general framework for inverting partitioned matrices
• useful properties of Kronecker product, Hadamard product, and diag
• the column-stacking operator “vec” and its generalization to “vec-transpose”

with applications to multilinear models, principal component analysis, blind source separation, Lyapunov equations, model alignment, and more.