TY - JOUR
TI - The Differentiable Curry
AU - Vytiniotis, Dimitrios
AU - Belov, Dan
AU - Wei, Richard
AU - Plotkin, Gordon
AU - Abadi, Martin
AB - We revisit the automatic differentiation (AD) of programs that contain higher-order functions, in a statically typed setting. Our presentation builds on a recent formulation of AD based on...
DA - 2019/09/20/
PY - 2019
DP - openreview.net
UR - https://openreview.net/forum?id=ryxuz9SzDB
Y2 - 2019/11/22/22:22:04
KW - Automatic differentiation
KW - Differentiation
KW - Programming language theory
ER -
TY - JOUR
TI - Functional probabilistic programming for scalable Bayesian modelling
AU - Law, Jonathan
AU - Wilkinson, Darren
T2 - arXiv:1908.02062 [stat]
AB - Bayesian inference involves the specification of a statistical model by a statistician or practitioner, with careful thought about what each parameter represents. This results in particularly interpretable models which can be used to explain relationships present in the observed data. Bayesian models are useful when an experiment has only a small number of observations and in applications where transparency of data driven decisions is important. Traditionally, parameter inference in Bayesian statistics has involved constructing bespoke MCMC (Markov chain Monte Carlo) schemes for each newly proposed statistical model. This results in plausible models not being considered since efficient inference schemes are challenging to develop or implement. Probabilistic programming aims to reduce the barrier to performing Bayesian inference by developing a domain specific language (DSL) for model specification which is decoupled from the parameter inference algorithms. This paper introduces functional programming principles which can be used to develop an embedded probabilistic programming language. Model inference can be carried out using any generic inference algorithm. In this paper Hamiltonian Monte Carlo (HMC) is used, an efficient MCMC method requiring the gradient of the un-normalised log-posterior, calculated using automatic differentiation. The concepts are illustrated using the Scala programming language.
DA - 2019/08/06/
PY - 2019
DP - arXiv.org
UR - http://arxiv.org/abs/1908.02062
Y2 - 2019/11/27/20:48:16
KW - Automatic differentiation
KW - Bayesian inference
KW - Differentiation
KW - Implementation
KW - Probabilistic programming
KW - Programming language theory
ER -
TY - JOUR
TI - Differentials and distances in probabilistic coherence spaces
AU - Ehrhard, Thomas
T2 - arXiv:1902.04836 [cs]
AB - In probabilistic coherence spaces, a denotational model of probabilistic functional languages, mor-phisms are analytic and therefore smooth. We explore two related applications of the corresponding derivatives. First we show how derivatives allow to compute the expectation of execution time in the weak head reduction of probabilistic PCF (pPCF). Next we apply a general notion of "local" differential of morphisms to the proof of a Lipschitz property of these morphisms allowing in turn to relate the observational distance on pPCF terms to a distance the model is naturally equipped with. This suggests that extending probabilistic programming languages with derivatives, in the spirit of the differential lambda-calculus, could be quite meaningful.
DA - 2019/02/13/
PY - 2019
DP - arXiv.org
UR - http://arxiv.org/abs/1902.04836
Y2 - 2019/11/28/11:57:10
KW - Coherence spaces
KW - Denotational semantics
KW - Differential Linear Logic
KW - Differentiation
KW - Linear logic
KW - Probabilistic programming
KW - Programming language theory
ER -
TY - CONF
TI - Higher-Order Distributions for Differential Linear Logic
AU - Kerjean, Marie
AU - Pacaud Lemay, Jean-Simon
A2 - Bojańczyk, Mikołaj
A2 - Simpson, Alex
T3 - Lecture Notes in Computer Science
AB - Linear Logic was introduced as the computational counterpart of the algebraic notion of linearity. Differential Linear Logic refines Linear Logic with a proof-theoretical interpretation of the geometrical process of differentiation. In this article, we construct a polarized model of Differential Linear Logic satisfying computational constraints such as an interpretation for higher-order functions, as well as constraints inherited from physics such as a continuous interpretation for spaces. This extends what was done previously by Kerjean for first order Differential Linear Logic without promotion. Concretely, we follow the previous idea of interpreting the exponential of Differential Linear Logic as a space of higher-order distributions with compact-support, which is constructed as an inductive limit of spaces of distributions on Euclidean spaces. We prove that this exponential is endowed with a co-monadic like structure, with the notable exception that it is functorial only on isomorphisms. Interestingly, as previously argued by Ehrhard, this still allows the interpretation of differential linear logic without promotion.
C1 - Cham
C3 - Foundations of Software Science and Computation Structures
DA - 2019///
PY - 2019
DO - 10/ggdmrj
DP - Springer Link
SP - 330
EP - 347
LA - en
PB - Springer International Publishing
SN - 978-3-030-17127-8
KW - Denotational semantics
KW - Differential Linear Logic
KW - Differentiation
KW - Linear logic
ER -
TY - SLIDE
TI - Diffeological Spaces and Denotational Semantics for Differential Programming
A2 - Kammar, Ohad
A2 - Staton, Sam
A2 - Vákár, Matthijs
DA - 2018///
PY - 2018
LA - en
KW - Automatic differentiation
KW - Differentiation
KW - Programming language theory
ER -
TY - JOUR
TI - An introduction to Differential Linear Logic: proof-nets, models and antiderivatives
AU - Ehrhard, Thomas
T2 - arXiv:1606.01642 [cs]
AB - Differential Linear Logic enriches Linear Logic with additional logical rules for the exponential connectives, dual to the usual rules of dereliction, weakening and contraction. We present a proof-net syntax for Differential Linear Logic and a categorical axiomatization of its denotational models. We also introduce a simple categorical condition on these models under which a general antiderivative operation becomes available. Last we briefly describe the model of sets and relations and give a more detailed account of the model of finiteness spaces and linear and continuous functions.
DA - 2016/06/06/
PY - 2016
DP - arXiv.org
ST - An introduction to Differential Linear Logic
UR - http://arxiv.org/abs/1606.01642
Y2 - 2019/11/28/11:52:31
KW - Denotational semantics
KW - Differential Linear Logic
KW - Differentiation
KW - Linear logic
ER -
TY - JOUR
TI - A Simply Typed λ-Calculus of Forward Automatic Differentiation
AU - Manzyuk, Oleksandr
T2 - Electronic Notes in Theoretical Computer Science
T3 - Proceedings of the 28th Conference on the Mathematical Foundations of Programming Semantics (MFPS XXVIII)
AB - We present an extension of the simply typed λ-calculus with pushforward operators. This extension is motivated by the desire to incorporate forward automatic differentiation, which is an important technique in numeric computing, into functional programming. Our calculus is similar to Ehrhard and Regnierʼs differential λ-calculus, but is based on the differential geometric idea of pushforward rather than derivative. We prove that, like the differential λ-calculus, our calculus can be soundly interpreted in differential λ-categories.
DA - 2012/09/24/
PY - 2012
DO - 10/ggdm57
DP - ScienceDirect
VL - 286
SP - 257
EP - 272
J2 - Electronic Notes in Theoretical Computer Science
LA - en
SN - 1571-0661
UR - http://www.sciencedirect.com/science/article/pii/S1571066112000473
Y2 - 2019/11/28/18:09:40
KW - Automatic differentiation
KW - Differentiation
KW - Programming language theory
ER -
TY - JOUR
TI - The cartesian closed bicategory of generalised species of structures
AU - Fiore, M.
AU - Gambino, N.
AU - Hyland, M.
AU - Winskel, G.
T2 - Journal of the London Mathematical Society
AB - The concept of generalised species of structures between small categories and, correspondingly, that of generalised analytic functor between presheaf categories are introduced. An operation of substitution for generalised species, which is the counterpart to the composition of generalised analytic functors, is also put forward. These deﬁnitions encompass most notions of combinatorial species considered in the literature—including of course Joyal’s original notion—together with their associated substitution operation. Our ﬁrst main result exhibits the substitution calculus of generalised species as arising from a Kleisli bicategory for a pseudo-comonad on profunctors. Our second main result establishes that the bicategory of generalised species of structures is cartesian closed.
DA - 2008/02//
PY - 2008
DO - 10/bd2mr9
DP - Crossref
VL - 77
IS - 1
SP - 203
EP - 220
LA - en
SN - 00246107
UR - http://doi.wiley.com/10.1112/jlms/jdm096
Y2 - 2019/11/28/16:31:36
KW - Denotational semantics
KW - Differential Linear Logic
KW - Differentiation
KW - Linear logic
ER -
TY - JOUR
TI - The differential lambda-calculus
AU - Ehrhard, Thomas
AU - Regnier, Laurent
T2 - Theoretical Computer Science
AB - We present an extension of the lambda-calculus with differential constructions. We state and prove some basic results (confluence, strong normalization in the typed case), and also a theorem relating the usual Taylor series of analysis to the linear head reduction of lambda-calculus.
DA - 2003/12/02/
PY - 2003
DO - 10/bf3b8v
DP - ScienceDirect
VL - 309
IS - 1
SP - 1
EP - 41
J2 - Theoretical Computer Science
LA - en
SN - 0304-3975
UR - http://www.sciencedirect.com/science/article/pii/S030439750300392X
Y2 - 2019/11/24/17:23:34
KW - Differentiation
KW - Linear logic
KW - Programming language theory
ER -