TY - JOUR
TI - Backprop as Functor: A compositional perspective on supervised learning
AU - Fong, Brendan
AU - Spivak, David I.
AU - Tuyéras, Rémy
T2 - arXiv:1711.10455 [cs, math]
AB - A supervised learning algorithm searches over a set of functions $A \to B$ parametrised by a space $P$ to find the best approximation to some ideal function $f\colon A \to B$. It does this by taking examples $(a,f(a)) \in A\times B$, and updating the parameter according to some rule. We define a category where these update rules may be composed, and show that gradient descent---with respect to a fixed step size and an error function satisfying a certain property---defines a monoidal functor from a category of parametrised functions to this category of update rules. This provides a structural perspective on backpropagation, as well as a broad generalisation of neural networks.
DA - 2019/05/01/
PY - 2019
DP - arXiv.org
ST - Backprop as Functor
UR - http://arxiv.org/abs/1711.10455
Y2 - 2019/11/23/14:42:07
KW - Categorical ML
KW - Machine learning
KW - Purely theoretical
ER -