Numerical Methods for DEs - Theory & Concepts

Numerical Methods for DEs

$Learning Objectives$

Understand the Initial Value Problem (IVP) context for numerical methods.
Apply Euler's Method to approximate solutions for first-order DEs.
Apply the Improved Euler's Method (Heun's Method) using a predictor-corrector approach.
Apply the 4th-order Runge-Kutta (RK4) method for high-accuracy approximations.
Analyze local and global truncation errors in numerical methods.

While analytical methods provide exact solutions to differential equations, many real-world problems are too complex or non-linear to solve analytically. Numerical methods allow us to approximate solutions by taking discrete steps from an initial condition. These methods are extensively used in civil engineering for structural analysis (finite element method), fluid dynamics, and geotechnical modeling.

$The Initial Value Problem (IVP)$

Numerical methods typically solve first-order Initial Value Problems. We seek to approximate the value of $y$ at discrete points $x_1, x_2, \dots, x_n$ , where $x_{i+1} = x_i + h$ , and $h$ is the step size. The approximation at $x_n$ is denoted $y_n \approx y(x_n)$ .

Initial Value Problem

The standard form of a first-order initial value problem.

\frac{dy}{dx} = f(x, y), \quad y(x_0) = y_0

Variables

Symbol	Description	Unit
$y$	The dependent variable function to approximate	-
$x$	The independent variable	-
$y_0$	The initial condition value at x_0	-

Euler's Method

Euler's method is the simplest, most intuitive numerical method. It relies on a linear approximation (the tangent line) at the current point to estimate the value of the function at the next point.

Euler's Method Formula

The iterative formula for Euler's method.

y_{n+1} = y_n + h \cdot f(x_n, y_n)

Variables

Symbol	Description	Unit
$y_{n+1}$	The approximated value at the next step	-
$y_n$	The current value	-
$h$	The step size	-
$f(x_n, y_n)$	The slope at the current point	-

$Applying Euler's Method$

STEP-BY-STEP

Initialize: Start at the given initial point $(x_0, y_0)$ .
Find Slope: Calculate the derivative (slope) at this point using the DE: $m = f(x_0, y_0)$ .
Step Forward: Estimate the next $y$ -value by moving along the tangent line: $y_1 = y_0 + m \cdot h$ .
Update: Advance the independent variable: $x_1 = x_0 + h$ .
Iterate: Repeat the process using the new point $(x_1, y_1)$ as the starting point to find $(x_2, y_2)$ , and so on.

Improved Euler's Method (Heun's Method)

Euler's method is often inaccurate because it assumes the slope remains completely constant over the entire step $h$ , which is rarely true for non-linear functions. The Improved Euler's method calculates an average slope using the derivative at the start and an estimated derivative at the end of the interval.

$Heun's Method Predictor-Corrector$

It is a two-step "predictor-corrector" method. First, estimate the intermediate value at the end of the step (predictor), and then average the slopes evaluated at the current point and the predicted point (corrector).

Heun's Method - Predictor Step

Predicts the next value using standard Euler's method.

y^*_{n+1} = y_n + h \cdot f(x_n, y_n)

Variables

Symbol	Description	Unit
$y^*_{n+1}$	The predicted intermediate value	-
$h$	The step size	-

Heun's Method - Corrector Step

Corrects the prediction by averaging the slopes at the start and the predicted end of the interval.

y_{n+1} = y_n + \frac{h}{2} [f(x_n, y_n) + f(x_{n+1}, y^*_{n+1})]

Variables

Symbol	Description	Unit
$y_{n+1}$	The final corrected value at the next step	-
$f(x_{n+1}, y^*_{n+1})$	The slope at the predicted end point	-

Runge-Kutta Methods (RK4)

The 4th-order Runge-Kutta method (RK4) is a widely used explicit fixed-step method for sufficiently smooth, non-stiff initial-value problems, offering a useful balance of accuracy and computational work. Instead of using just two points, it evaluates the slope at four strategically chosen points within the interval to compute a weighted average slope.

RK4 Formula

The 4th-order Runge-Kutta method for approximating solutions.

y_{n+1} = y_n + \frac{h}{6}(k_1 + 2k_2 + 2k_3 + k_4)

Variables

Symbol	Description	Unit
$y_{n+1}$	The approximated value at the next step	-
$h$	The step size	-
$k_1$	Slope at the beginning of the interval: f(x_n, y_n)	-
$k_2$	Slope at the midpoint: f(x_n + h/2, y_n + h \cdot k_1/2)	-
$k_3$	Slope at the midpoint: f(x_n + h/2, y_n + h \cdot k_2/2)	-
$k_4$	Slope at the end of the interval: f(x_n + h, y_n + h \cdot k_3)	-

Error Analysis in Numerical Methods

No numerical method is perfect; they all introduce errors compared to the exact analytical solution $y(x)$ . Understanding the type and magnitude of these errors is crucial for choosing the right method and step size.

$Types of Truncation Errors$

Local Truncation Error ( $E_{local}$ ): The error introduced in a single step, assuming the starting point for that step was perfectly accurate. It results from cutting off (truncating) the Taylor series approximation.
- For Euler's Method, $E_{local} \propto O(h^2)$ . Halving $h$ reduces the local error by a factor of 4.
- For RK4, $E_{local} \propto O(h^5)$ . Halving $h$ reduces the local error by a factor of 32!
Global Truncation Error ( $E_{global}$ ): The cumulative error after many steps, representing the total difference between the numerical approximation $y_n$ and the true analytical value $y(x_n)$ . Because calculating a solution requires $n = (x_{final} - x_0)/h$ steps, errors accumulate.
- For Euler's Method, $E_{global} \propto O(h)$ . It is a "first-order" method.
- For RK4, $E_{global} \propto O(h^4)$ . It is a "fourth-order" method, making it vastly superior over long integrations.

Numerical Methods Simulator

Explore how different numerical methods approximate the exact solution to a differential equation. Notice how decreasing the step size $h$ improves accuracy, and explicitly compare the massive difference in global error between Euler's method and RK4.

$Interactive Simulation$

Interact with the simulation below to compare Euler's method with the 4th order Runge-Kutta method.

Numerical Methods Comparison

Solving $y' = y$ with $y(0)=1$ .

Compare the simple Euler method (1st order) to the Runge-Kutta method (4th order). Notice how much faster RK4 converges to the exact solution as step size $h$ decreases.

Step Size (

h

)1.00

Fine (0.1)Coarse (1.5)

Exact (

e^x

)

Euler's MethodE ~ O(h)

RK4 MethodE ~ O(h^4)

x ∈ [0, 3]

Visual Relationships in Numerical Integration

An IVP Is Approximated on a Stepwise Mesh

Numerical integration advances from a known initial state through discrete independent-variable values separated by a chosen step size.

Start from x₀ and y₀

Choose step size h

Advance xₙ₊₁ = xₙ + h

Store yₙ as an approximation at each mesh point

Euler's Method Extends the Current Tangent

Euler uses one slope evaluation at the start of a step and follows that tangent estimate across the interval.

Evaluate f(xₙ, yₙ)

Use the tangent slope for the full step h

Set yₙ₊₁ = yₙ + h f(xₙ, yₙ)

Repeat from the new approximate point

Heun's Method Corrects an Euler Prediction

The predictor estimates the endpoint; the corrector averages the initial and predicted endpoint slopes for the updated value.

Predict y*ₙ₊₁ with the starting slope

Evaluate the slope at the predicted endpoint

Average the start and predicted-end slopes

Correct yₙ₊₁ using that average

Method Order Balances Slope Work and Global Error

For sufficiently small steps on a sufficiently regular IVP, higher global order reduces asymptotic error faster as h decreases.

Euler | one slope evaluation | global error O(h)

Heun | two slope evaluations | global error O(h²)

RK4 | four slope evaluations | global error O(h⁴)

Accuracy and stability still depend on the IVP and step size

Key Takeaways

Numerical Methods approximate IVP solutions by iterating in discrete steps of size $h$ , crucial when analytical integration is impossible.
Euler's Method ( $y_{n+1} = y_n + h \cdot f(x_n, y_n)$ ) is simple but only first-order accurate ( $O(h)$ global error); it accumulates error quickly unless $h$ is impractically small.
Improved Euler's (Heun's) Method uses a predictor-corrector approach to average the slope across the step, improving global accuracy to second-order ( $O(h^2)$ ).
Runge-Kutta 4th Order (RK4) is a widely used fixed-step explicit method with fourth-order global accuracy for sufficiently smooth, non-stiff IVPs. Its practical accuracy and stability depend on the equation and step size; stiff problems may call for a different solver.
Error Analysis: Local truncation error happens per step; global truncation error is the accumulated total error. Higher-order methods (like RK4) drastically reduce both compared to simpler methods.

$References$

Previous TopicSeries Solutions - Examples & Applications

Quiz Me

Next TopicNumerical Methods for DEs - Examples & Applications

Prev Next

Quiz Me