Numerical Integration

Numerical integration, also known as quadrature, involves approximating the definite integral of a mathematical function or discrete data. Geometrically, it represents finding the area under a continuous curve.

Newton-Cotes Formulas

The Newton-Cotes formulas are the most common numerical integration schemes. They are based on the strategy of replacing a complicated function or tabulated data with an approximating interpolating polynomial that is easy to integrate analytically.

Open and Closed Forms

Newton-Cotes formulas have two forms: closed forms, where data points at the beginning and end of the limits of integration (aa and bb) are known and utilized, and open forms, where integration limits extend beyond the range of the data points. Open forms are highly useful for improper integrals or for solving initial-value ordinary differential equations.

Trapezoidal Rule (Closed Form)

The simplest closed Newton-Cotes formula. It approximates the area under the curve using a straight line (a first-order polynomial) connecting the two points at the ends of the interval.
I(ba)f(a)+f(b)2I \approx (b - a)\frac{f(a) + f(b)}{2}
The true error for a single application is Et=112f(ξ)(ba)3E_t = -\frac{1}{12} f''(\xi)(b-a)^3, indicating the rule is exact for linear functions (where f=0f''=0). For improved accuracy, the multiple-application (or composite) trapezoidal rule divides the interval into nn multiple segments of width h=banh = \frac{b-a}{n} and applies the rule to each.

Composite Trapezoidal Rule

The total integral is the sum of nn individual trapezoids: Ih2[f(x0)+2i=1n1f(xi)+f(xn)]I \approx \frac{h}{2} \left[ f(x_0) + 2\sum_{i=1}^{n-1} f(x_i) + f(x_n) \right] Global Error Bound: Because errors from individual segments accumulate, the total global error EaE_a is bounded by the second derivative maximum over the entire domain [a,b][a,b]: Ea(ba)312n2maxx[a,b]f(x)=(ba)h212maxx[a,b]f(x)|E_a| \le \frac{(b-a)^3}{12n^2} \max_{x \in [a,b]} |f''(x)| = \frac{(b-a)h^2}{12} \max_{x \in [a,b]} |f''(x)| This demonstrates that the composite rule has an overall truncation error of O(h2)O(h^2).

Composite Trapezoidal Rule

Approximating the integral of f(x)f(x) from a=0a=0 to b=0.8b=0.8 using multiple segments.

00.8xf(x)
1 Segment20 Segments

Integration Results

Step Size (h)
0.4000
Exact Value
1.640533
Approximate Integral (Area)
1.068800
Formula
Ih2[f(x0)+2i=1n1f(xi)+f(xn)]I \approx \frac{h}{2} \left[ f(x_0) + 2 \sum_{i=1}^{n-1} f(x_i) + f(x_n) \right]
True Percent Relative Error:
34.8504%
As n increases, the error approaches zero.

Midpoint Rule (Open Form)

The simplest open Newton-Cotes formula is the midpoint rule, which approximates the area using a rectangle based on the function value at the midpoint xm=(a+b)/2x_m = (a+b)/2. Since it doesn't use the endpoints aa or bb, it's highly useful when the integrand has an integrable singularity at a boundary limit.
I(ba)f(xm)I \approx (b - a) f(x_m)

Simpson's Rules

Simpson's rules use higher-order polynomials to connect points, yielding better accuracy for smooth functions than the trapezoidal rule.

Simpson's 1/3 Rule

Uses a parabola (second-order polynomial) connecting three points. The error is proportional to the fourth derivative, making it exact for polynomials up to the third degree (cubics). Ih3[f(x0)+4f(x1)+f(x2)]I \approx \frac{h}{3} [f(x_0) + 4f(x_1) + f(x_2)] Multiple-Application Form: Requires an even number of segments (an odd number of points). Ih3[f(x0)+4i=1,3,5n1f(xi)+2j=2,4,6n2f(xj)+f(xn)]I \approx \frac{h}{3} \left[ f(x_0) + 4\sum_{i=1,3,5}^{n-1} f(x_i) + 2\sum_{j=2,4,6}^{n-2} f(x_j) + f(x_n) \right] Global Error Bound: The composite 1/31/3 rule error is strictly bounded by: Ea(ba)5180n4maxx[a,b]f(4)(x)=(ba)h4180maxx[a,b]f(4)(x)|E_a| \le \frac{(b-a)^5}{180n^4} \max_{x \in [a,b]} |f^{(4)}(x)| = \frac{(b-a)h^4}{180} \max_{x \in [a,b]} |f^{(4)}(x)| Thus, Simpson's 1/3 rule has a high accuracy of order O(h4)O(h^4).

Simpson's 3/8 Rule

Uses a cubic curve (third-order polynomial) connecting four points. It also has a truncation error of O(h4)O(h^4) globally, and is primarily used when there is an odd number of segments, often combined with the 1/3 rule for the remaining even segments. I3h8[f(x0)+3f(x1)+3f(x2)+f(x3)]I \approx \frac{3h}{8} [f(x_0) + 3f(x_1) + 3f(x_2) + f(x_3)]

Integration with Unequal Segments

Experimental data often involves unequal spatial or temporal intervals. In this case, standard composite rules fail, and the basic single-segment trapezoidal rule is typically applied individually to each sub-interval and summed.

Romberg Integration

Romberg integration applies Richardson extrapolation recursively to the composite trapezoidal rule to systematically generate highly accurate integral estimates. By halving the step size and combining the results, it eliminates consecutive leading error terms (O(h2)O(h^2), O(h4)O(h^4), O(h6)O(h^6)). It requires the function to be evaluated at evenly spaced points, doubling the points at each refinement level.

Adaptive Quadrature

Standard Newton-Cotes formulas use a constant step size hh across the entire domain. However, if a function has regions of sharp variation (e.g., a steep spike) alongside regions of slow variation, a constant hh must be inefficiently small everywhere simply to capture the spike accurately.

How Adaptive Quadrature Works

Adaptive methods dynamically adjust the step size hh based on an estimated local error. The algorithm computes the integral over a specific interval using two methods (e.g., a basic Simpson's rule and a refined Simpson's rule with twice the segments). If the absolute difference exceeds a specified tolerance, the interval is recursively halved in the problematic region. This concentrates computational effort only where the integrand is "difficult," drastically improving overall efficiency.

Gauss Quadrature

While Newton-Cotes formulas use fixed, equally spaced evaluation points, Gauss quadrature treats both the evaluation points (roots or nodes) and their corresponding weights as unknowns, selecting them optimally to maximize the polynomial degree that can be integrated exactly.

Gauss-Legendre Formulas

The most common form uses the roots of Legendre polynomials. To apply the method, the limits of integration must be algebraically transformed from the original domain [a,b][a, b] to the standardized interval [1,1][-1, 1] using a linear change of variables: Change of Limits Formula: Let x=b+a2+ba2tx = \frac{b+a}{2} + \frac{b-a}{2}t, which implies dx=ba2dtdx = \frac{b-a}{2}dt. The integral becomes a weighted sum evaluated at the roots tit_i: abf(x)dxba2i=1ncif(b+a2+ba2ti)\int_{a}^{b} f(x) dx \approx \frac{b-a}{2} \sum_{i=1}^n c_i f\left( \frac{b+a}{2} + \frac{b-a}{2}t_i \right) Where cic_i are the optimal weights and tit_i are the roots of the nn-th degree Legendre polynomial. An nn-point formula perfectly integrates polynomials up to degree 2n12n-1, making it exceptionally efficient for smooth functions. Explicit Values:
  • Two-point formula (n=2n=2): Exact for 3rd degree polynomials.
    • Roots: t1=1/30.57735t_1 = -1/\sqrt{3} \approx -0.57735, t2=1/30.57735t_2 = 1/\sqrt{3} \approx 0.57735
    • Weights: c1=1c_1 = 1, c2=1c_2 = 1
  • Three-point formula (n=3n=3): Exact for 5th degree polynomials.
    • Roots: t1=3/50.77459t_1 = -\sqrt{3/5} \approx -0.77459, t2=0t_2 = 0, t3=3/50.77459t_3 = \sqrt{3/5} \approx 0.77459
    • Weights: c1=5/9c_1 = 5/9, c2=8/9c_2 = 8/9, c3=5/9c_3 = 5/9

Improper Integrals

Improper integrals feature limits of ±\pm \infty or integrands with singularities at a boundary. They can be evaluated numerically by using open Newton-Cotes formulas (which inherently avoid evaluating the boundary points) or by applying an algebraic change of variable (e.g., x=1/tx = 1/t, dx=1/t2dtdx = -1/t^2 dt) to transform an infinite limit into a finite limit (like t0t \to 0).

Multiple Integrals

Double or triple integrals (e.g., calculating volumes, moments of inertia, or centroids) can be evaluated by successive, nested applications of 1D numerical integration rules. For instance, a double integral f(x,y)dxdy\iint f(x,y) \,dx \,dy is solved numerically by holding yy constant, integrating over xx at each discrete yy-node, and then integrating those resulting 1D areas over the yy-axis.

Monte Carlo Integration

For low-dimensional integrals (1D, 2D, 3D), Gauss quadrature or Romberg integration are highly efficient. However, for high-dimensional multiple integrals (e.g., 10D or more), the number of required function evaluations grows exponentially for grid-based methods (the "curse of dimensionality").

Monte Carlo Method

Monte Carlo integration uses random uniform sampling to approximate the integral. By randomly sampling NN points xi\mathbf{x}_i within the multidimensional integration domain VV, the integral is approximated as the average function value multiplied by the total domain volume: Vf(x)dxV1Ni=1Nf(xi)\int_V f(\mathbf{x}) d\mathbf{x} \approx V \cdot \frac{1}{N} \sum_{i=1}^N f(\mathbf{x}_i) While the convergence rate is slow (O(1/N)O(1/\sqrt{N})), it is fundamentally independent of the number of dimensions. Therefore, for very high-dimensional problems (common in statistical mechanics, quantum physics, and quantitative finance), Monte Carlo methods remain the only computationally feasible approach.
Key Takeaways
  • Newton-Cotes formulas replace the exact integrand with an easily integrable polynomial. They exist in both closed (uses boundaries) and open (avoids boundaries) forms.
  • The Trapezoidal rule uses a linear approximation with a global composite error strictly proportional to O(h2)O(h^2).
  • Simpson's rules use parabolic (1/3 rule) and cubic (3/8 rule) approximations, offering much higher accuracy with a global composite error proportional to O(h4)O(h^4).
  • Composite rules divide the domain into multiple smaller segments to drastically increase accuracy while maintaining simple low-order polynomial evaluations.
  • Romberg integration elegantly uses Richardson extrapolation to efficiently improve simple trapezoidal rule estimates to high orders of accuracy.
  • Adaptive Quadrature dynamically halves the local step size hh only in regions where the integrand varies rapidly, optimizing computational effort.
  • Gauss quadrature treats both nodes and weights as unknowns to maximize polynomial degree accuracy (2n12n-1). It uses the Change of Limits formula to map any interval [a,b][a, b] onto the standardized interval [1,1][-1, 1].
  • Improper integrals can be evaluated using open formulas (like the midpoint rule) or by transforming limits algebraically (e.g., mapping infinity to zero).
  • Multiple integrals are solved by successive nested applications of single-variable integration rules.
  • Monte Carlo Integration relies on random sampling and becomes the method of choice for extremely high-dimensional integrals, bypassing the curse of dimensionality.