Math Insight

Video: Solving pure-time differential equations with the Forward-Euler algorithm

Video links

This video is found in the pages

List of all videos

Video slides

Transcript of video

The Forward Euler algorithm is an algorithm to estimate the solution to differential equations. Let's illustrate how to use it to solve the pure-time differential equation $x'(t) = t^3-5t-2$ with the initial condition that $x(t)$ is 10 at time $t=0$. Our goal is to estimate the solution $x(t)$.

We know we must start at $x(0)=10$, and the next question is to determine how fast $x(t)$ is changing at $t=0$. This rate of change is simply the slope, or derivative, of the function $x(t)$ at the initial time $t=0$. We can calculate this derivative directly from the differential equation, by plugging in $t=0$. We calculate that $x'(0)$ is zero cubed minus five times zero minus two, or negative two.

The slope of the graph of $x(t)$ is negative two only at $t=0$. For larger values of $t$, it will change according to the equation $x'(t)=t^3-5t-2$. But, let's pretend that we don't realize that fact; let's pretend that the slope $x'(t)$ stays fixed at the value negative two. A function with constant slope is a line, so let's use the linear approximation to $x(t)$ evaluated at $t=0$. We denote this linear approximation as $L_0(t)$, where the zero indicates we are matching the function and its derivative at $t=0$. We obtain the equation for a line with a slope of negative two and $y$-intercept of ten. The graph of the linear approximation looks like this. The slope looks shallower than negative two because the vertical scale is much larger than the horizontal scale. But, indeed, it drops by eight units by the time we get to $t=4$.

We could claim that $L_0(t)$ is a good approximation to the solution if the derivative $x'(t)$ stayed constant at negative two for all time, or at least didn't change much. However, if we look at the formula for $x'(t)$, it's pretty clear that $x'(t)$ is not constantly negative two but instead changes to much different values. Since $x'(t)$ is continuous, though, we can expect it to stay relatively close to negative two for a short amount of time; we can expect the linear approximation $L_0(t)$ to be an OK approximation to the real solution $x(t)$ for small values of $t$.

If we are OK with approximating $x(t)$ by the linear function $10-2t$ for a small amount of time, how can we estimate values of the solution for longer times. One option is the Forward Euler algorithm, in which we pick some small length of time $\Delta t$ and use $L_0(t)$ for that small interval of time. Then, we'll calculate a new linear approximation and continue the process. Let's illustrate the Forward Euler algorithm by choosing an interval length $\Delta t$ of one.

We use our linear approximation that we calculated from $t=0$, i.e., our $L_0(t)$, to estimate the value of the solution at $t=1$. This approximation for $x(1)$ is ten minus two times one, or eight. I've highlighted this result in red on the graph.

Now that we've estimated $x(1)$ to be eight, we can repeat the process by calculating a new linear approximation at $t=1$. We call this linear approximation $L_1(t)$ to emphasize it is calculated at $t=1$. The value of $x$ and its derivative are calculated at $t=1$ and we subtract one from $t$, as highlighted in red. This linear approximation is the equation for the line that is equal to $x(1)$ when $t=1$ and has a slope equal to $x'(1)$. We have an estimate of $x(1)$; the only new piece of information we need is $x'(1)$. We calculate this derivative directly from the differential equation. Plugging in $t=1$ into the differential equation, we find that $x'(1)$ is equal to negative six. The derivative decreases from negative two at $t=0$ to negative six at $t=1$; the function $x(t)$ is beginning to decrease more steeply.

Plugging $x(1)=8$ and $x'(1)=-6$ into the linear approximation formula, we obtain a new approximation for $x(t)$ for $t>1$. When we graph this second linear approximation, we see how the estimate of $x(t)$ decreases more steeply. Since we are using time intervals of length $\Delta t=1$, we'll use this linear approximation for a step of that length and estimate the value of $x(t)$ at $t=2$. The linear approximation yields $x(2)=2$, which we plot on the graph with another red point.

To summarize our result so far with $\Delta t=1$, we used the linear approximation at $t=0$ to estimate that $x(1)$ is eight, then used the linear approximation at $t=1$ to estimate that $x(2)$ is two. Let's continue this process for two more time steps to estimate $x(t)$ at $t=3$ and at $t=4$. For these steps, I won't describe all the calculations; you can pause the video if you'd like to follow all the details.

We write down a linear approximation for $x(t)$ at $t=2$. The important fact to note in this calculation is that $x'(2)$ is negative four; the derivative has increased a little bit. We draw a line sloping downward with slope negative 4. From this line, we estimate that $x(3)$ is about negative two.

For the fourth and final step using $\Delta t=1$, we write down a linear approximation for $x(t)$ at $t=3$. Note that the estimate for $x'(3)$ is ten. The slope has become large and positive and this last linear approximation increases rapidly. The Forward Euler estimate of $x(4)$ is eight.

When $\Delta t=1$, the Forward Euler algorithm took four steps to get to $t=4$. I've gathered formulas for all four steps together. Let's massage them to make them prettier and easier to remember. First, let's get rid of some extra information and just keep the final formulas. Next, notice that in each step, we are multiplying the derivative by the same number. One minus zero, two minus one, three minus two, etc., are all the length of the time interval, $\Delta t$, which in this case is one. Let's rewrite the formula to show this dependence on $\Delta t$.

We evaluated the functions and derivatives at the points $0,1,2$, etc., which are all multiples of the time interval $\Delta t$. Let's define the time points $t_0$, $t_1$, $t_2$, etc., as these multiples of $\Delta t$, and replace the numbers with these time points. In this way, our formulas are no longer specific to $\Delta t=1$. In fact, let's remove any reference to value one and just keep definitions in terms of $\Delta t$. Now our formulas for Forward Euler will work for any value of the time interval $\Delta t$.

Of course, we can continue this process beyond the fourth time point. If we like, we can write the Forward Euler formula for an arbitrary time point $t_i$. The value $x(t_{i+1})$ at the next time point is approximated as the value $x(t_i)$ of the previous time point plus the value of the derivative times $\Delta t$.

More important than memorizing this formula is understanding the idea behind the Forward Euler algorithm. In each step, we simply calculate the slope of the function as the derivative given by the differential equation. Then, we take a step of size $\Delta t$ with that slope. We can continue this process for as long as we want, though if we are calculating the Forward Euler algorithm by hand, there is always the danger we might fall asleep in the middle of repeating these tedious steps.

The Forward Euler algorithm is nice in that it gives us an answer. You might have some lingering doubts about the accuracy of the crude approximation of assuming the slope does not change in the middle of these intervals of length $\Delta t$. Rather than directly trying to estimate its accuracy here, though, let's think about how we could improve the accuracy of the Forward Euler algorithm. We could increase the accuracy by taking smaller time steps $\Delta t$ so that we recalculate the slope of the function more frequently.

For example, we could cut the time interval in half to $\Delta t=1/2$. This sounds like a good idea, but the cost is that we have to take twice as many steps. To get to $t=4$, we have to take eight steps of the algorithm. Ready to crunch some numbers? There, those calculations weren't so bad. OK, I don't feel like reading those numbers either. Let's just summarize the results of this graphically.

We start off with this same slope of negative two, but this time only use this slope until $t=1/2$. Then, we recalculate the slope at $t=1/2$ and find that the function already should be decreasing more rapidly. This change in slope was undetected by the first calculation with $\Delta t=1$. For the next three time steps, the slopes don't change much, and the new calculation with $\Delta t=1/2$ isn't too different from the first calculation with $\Delta t=1$. We see that the slope already becomes positive at $t=2.5$. When we get to $t=3.5$, we see that slope has increased dramatically and that the we significantly underestimated the value of $x(4)$ in the first calculation with $\Delta t=1$. In fact, we exceeded the scale of the graph, so let's increase the vertical scale so that everything fits.

The second calculation, with $\Delta t=1/2$ (the green curve), is more accurate than the first calculation with $\Delta t=1$ (the blue curve). Because we took a smaller time step, we could adjust the direction more frequently. However, the improved accuracy came at a cost, as we had to do twice as much work.

We should really estimate the solution with even smaller $\Delta t$, but that would be a big pain to do by hand. Instead, we wrote a computer program to the calculation for us. The solution with $\Delta t=1/4$ (16 steps total) is shown by the cyan curve. We could even try $\Delta t=1/8$ (32 steps total), as shown by the thin magenta curve. Again, we see that we've been underestimating the value of $x(4)$, as the last curve exceeded our scale again. The calculations with larger time steps $\Delta t$ have been significantly underestimating this value, and we must increase the vertical scale of the graph even further.

In summary, the Forward Euler algorithm is a fairly simple algorithm to estimate the solution of a differential equation. However, for accuracy, one needs to take many small time steps, which is best done with a computer program.