TY - JOUR
TI - Characterizing the invariances of learning algorithms using category theory
AU - Harris, Kenneth D.
T2 - arXiv:1905.02072 [cs, math, stat]
AB - Many learning algorithms have invariances: when their training data is transformed in certain ways, the function they learn transforms in a predictable manner. Here we formalize this notion using concepts from the mathematical field of category theory. The invariances that a supervised learning algorithm possesses are formalized by categories of predictor and target spaces, whose morphisms represent the algorithm's invariances, and an index category whose morphisms represent permutations of the training examples. An invariant learning algorithm is a natural transformation between two functors from the product of these categories to the category of sets, representing training datasets and learned functions respectively. We illustrate the framework by characterizing and contrasting the invariances of linear regression and ridge regression.
DA - 2019/05/06/
PY - 2019
DP - arXiv.org
UR - http://arxiv.org/abs/1905.02072
Y2 - 2019/10/10/11:53:28
ER -
TY - JOUR
TI - Algebraic Machine Learning
AU - Martin-Maroto, Fernando
AU - de Polavieja, Gonzalo G.
T2 - arXiv:1803.05252 [cs, math]
AB - Machine learning algorithms use error function minimization to fit a large set of parameters in a preexisting model. However, error minimization eventually leads to a memorization of the training dataset, losing the ability to generalize to other datasets. To achieve generalization something else is needed, for example a regularization method or stopping the training when error in a validation dataset is minimal. Here we propose a different approach to learning and generalization that is parameter-free, fully discrete and that does not use function minimization. We use the training data to find an algebraic representation with minimal size and maximal freedom, explicitly expressed as a product of irreducible components. This algebraic representation is shown to directly generalize, giving high accuracy in test data, more so the smaller the representation. We prove that the number of generalizing representations can be very large and the algebra only needs to find one. We also derive and test a relationship between compression and error rate. We give results for a simple problem solved step by step, hand-written character recognition, and the Queens Completion problem as an example of unsupervised learning. As an alternative to statistical learning, algebraic learning may offer advantages in combining bottom-up and top-down information, formal concept derivation from data and large-scale parallelization.
DA - 2018/03/14/
PY - 2018
DP - arXiv.org
UR - http://arxiv.org/abs/1803.05252
Y2 - 2019/10/10/11:42:39
ER -
TY - CONF
TI - Neural Networks, Knowledge and Cognition: A Mathematical Semantic Model Based upon Category Theory
AU - Healy, Michael J.
AU - Caudell, Thomas P.
AB - Category theory can be applied to mathematically model the semantics of cognitive neural systems. We discuss semantics as a hierarchy of concepts, or symbolic descriptions of items sensed and represented in the connection weights distributed throughout a neural network. The hierarchy expresses subconcept relationships, and in a neural network it becomes represented incrementally through a Hebbian-like learning process. The categorical semantic model described here explains the learning process as the derivation of colimits and limits in a concept category. It explains the representation of the concept hierarchy in a neural network at each stage of learning as a system of functors and natural transformations, expressing knowledge coherence across the regions of a multi-regional network equipped with multiple sensors. The model yields design principles that constrain neural network designs capable of the most important aspects of cognitive behavior.
DA - 2004///
PY - 2004
DP - Semantic Scholar
ST - Neural Networks, Knowledge and Cognition
ER -
TY - SLIDE
TI - Algebra and Artiļ¬cial Intelligence
A2 - Murfet, Daniel
LA - en
KW - Algebra
KW - Classical ML
KW - Machine learning
KW - Sketchy
ER -