why relevant: powerful class of transforms with nice properties (continuous) for learning represented as differential manifolds, which also interested in Also working on biologically-inspired vision systems mentored by Bruno Olshausen. Lie groups + sparse codes have great emperical results
tooling from topology and abstract algebra
paper notes and derivations
Introduction to smooth manifold optimization… Notes from this textbook
Formulations that motivate manifold optimization. Logistic regression Taking a well known formulation from linear algebra and teasing out a curvy structure. Consider $x_1,\ldots,x_m \in \xi$ each with a corresponding binary label $y_1,\ldots,y_m \in \{0,1\}$. We can define some vector $\theta \in \xi$ that we can “grab onto” with each $x_i$ using an inner-product defined over the space. Applying a logistic transform $\sigma$, we can obtained well-behaved probabilities: $$P[y=1|x,\theta] = \sigma(\lang\theta, x_i\rang) $$ $$P[y=0|x,\theta] = 1 - \sigma(\lang\theta, x_i\rang) $$