Curvature and automatic differentiation

Curvature is tedious to calculate by hand because it involves calculating first and second order derivatives. Of course other applications require derivatives too, but curvature is the example we'll look at in this post.

Computing derivatives

It would be nice to write programs that only explicitly implement the original function and let software take care of finding the derivatives.

Numerical differentiation

Finite difference approximations for derivatives are nothing new. For example, Euler (1707-1783) used finite differences to numerically solve differential equations. But numerical differentiation can be inaccurate or unstable, especially for higher order derivatives.

Symbolic differentiation

Symbolic differentiation is another approach, having the computer manipulate expressions much as a person would do by hand. It works well for many problems, though it scales poorly for large problems. It also requires functions to be presented in traditional mathematical form, not in the form of source code.

Automatic differentiation

Automatic differentiation is a third way. Like numerical differentiation, it works with floating point numbers, not symbolic expressions. But unlike numerical differentiation, the result does not have approximation error.

As someone put it, automatic differentiation applies the chain rule to floating point numbers rather than to symbolic expressions.

Python implementation

I'll use the Python library autograd to compute curvature and illustrate automatic differentiation. autograd is not the most powerful automatic differentiation library for Python, but it is the simplest I've seen.

We will compute the curvature of a logistic curve.

The curvature of the graph of a function is given by

Here's Python code using autograd to compute the curvature.

 import autograd.numpy as np from autograd import grad def f(x): return 1/(1 + np.exp(-x)) f1 = grad(f) # 1st derivative of f f2 = grad(f1) # 2nd derivative of f def curvature(x): return abs(f2(x))*(1 + f1(x)**2)**-1.5

Curvature plots

The graph is relatively flat in the middle and at the far ends. In between, the graph bends creating two regions of higher curvature.

 import matplotlib.pyplot as plt x = np.linspace(-5, 5, 300) plt.plot(x, f(x)) plt.xlabel("$x$") plt.ylabel("$y$") plt.title("Logisitic curve") plt.savefig("logistic_curve.svg")

Now let's look at the curvature.

 y = [curvature(t) for t in x] plt.plot(x, y) plt.xlabel("$x$") plt.ylabel(r"$\kappa(x)$") plt.title("Curvature") plt.savefig("plot_logistic_curvature.svg")

As we should expect, the curvature is small at the ends and in the middle, with local maxima in between.

We can also look at the signed curvature, the same expression as curvature but without the absolute value.

We plot this with the following code.

 def signed_curvature(x): return f2(x)*(1 + f1(x)**2)**-1.5 y = [signed_curvature(t) for t in x] plt.plot(x, y) plt.xlabel("$x$") plt.ylabel(r"$k(x)$") plt.title("Signed curvature") plt.savefig("graph_signed_curvature.svg")

The result looks more like a sine wave.

The positive values mean the curve is bending counterclockwise, and the negative values mean the curve is bending clockwise.

Related post: Squircles and curvature