Image: Neuron firing rate with kink and tangent line
The blue curve is a caricature of the firing rate $r(i)$ of a neuron in response to an input $i$. This firing rate function has a kink at $i=I_0$. The green tangent line is calculated at some input $i=a>I_0$; the existence of the tangent line shows the function $r(i)$ is differentiable at that point. The function $r(i)$ is not differentiable at $i=I_0$ because there is no tangent line at $i=I_0$. Due to the kink there, no matter what line we chose, it will fail to match the graph on either the left or the right side of $I_0$.
Image file: neuron_firing_rate_kink_tangent_line.png
Image links
This image is found in the pages