5.4 Prediction Limits
Prediction limits have several uses in groundwater monitoring, all of which involve predicting the upper limit of possible future values based on a backgroundNatural or baseline groundwater quality at a site that can be characterized by upgradient, historical, or sometimes cross-gradient water quality (Unified Guidance). or baseline data set and comparing that limit to compliance point measurements or statistics. An upper prediction limit is constructed from upgradient or historical data and is designed to equal or exceed a specified number of future comparisons. If any of those values exceed the prediction limit, then the analysis suggests that groundwater concentrations have risen above the background levels. Prediction limits explicitly account for the degree of variation in the background population and the size of the sample of measurements used to construct the limit.
Prediction limits can be constructed with either a parametricA statistical test that depends upon or assumes observations from a particular probability distribution or distributions (Unified Guidance). or nonparametricStatistical test that does not depend on knowledge of the distribution of the sampled population (Unified Guidance). statistical model. Parametric prediction limitsIntervals constructed to contain the next few sample values or statistics within a known probability (Unified Guidance). are based on the meanThe arithmetic average of a sample set that estimates the middle of a statistical distribution (Unified Guidance). and standard deviation of the background or baseline data set, whereas nonparametric prediction limits are based on ranking of the observations. Table F-2 includes information about checking assumptions for prediction limits.
Interwell prediction limits compare background and compliance data collected from distinct spatial locations (upgradient versus downgradient). Intrawell prediction limits compare historical data (labeled intrawellComparison of measurements over time at one monitoring well (Unified Guidance). background) versus current data from a single location (see Section 3.6.5: Should I use interwell or intrawell sampling?).
Applications and Relevant Study Questions
Prediction limits are primarily used as formal detection monitoring tests of compliance data against background. Since they are very flexible statistical tools, prediction limits can be used successfully as interwell or intrawell tests and can be readily adapted to modern sampling schedules.
- For interwell testing, data from upgradient wells are used as background data to construct the upper prediction limit, to which the compliance data are compared.
- For intrawell testing, historical data from each compliance well are used as background data to construct the upper prediction limit, to which the current data are compared.
- It is possible to use prediction limits to compare means, medians, or other statistical measures of background and compliance data sets.
- Lower bound prediction limits are occasionally used to indicate that certain parameters (for example, pH, dissolved oxygen) are below a target interval
- Study Question 2: Are concentrations greater than background concentrations?
- Study Question 3: Are concentrations above or below a criterionGeneral term used in this document to identify a groundwater concentration that is relevant to a project; used instead of designations such as Groundwater Protection Standard, clean-up standard, or clean-up level.?
Assumptions
- Parametric prediction limits assume the data follow a known distribution or can be transformed to a known distribution.
- Nonparametric prediction limits for testing future medians do not assume normality or any particular distributional form.
- All prediction limits assume the population is stable (or stationaryA distribution whose population characteristics do not change over time or space (Unified Guidance).) over the period of time during which measurements are collected. That is, no obvious trends or temporal patterns should exist in the background data.
- Prediction limits used for interwell testing assume minimal (nonsignificant) spatial variabilitySpatial variability exists when the distribution or pattern of concentration measurements changes from well location to well location (most typically in the form of differing mean concentrations). Such variation may be natural or synthetic, depending on whether it is caused by natural or artificial factors (Unified Guidance).. Intrawell prediction limits can be used when spatial variation is significant, but require the assumption that intrawell background is uncontaminated.
- Prediction limits for testing future means assumes a normal distributionSymmetric distribution of data (bell-shaped curve), the most common distribution assumption in statistical analysis (Unified Guidance). or data transformed to a normal distribution. Perform all computations and comparisons on the transformed scale to avoid transformation biasSystematic deviation between a measured (observed) or computed value and its true value. Bias is affected by faulty instrument calibration and other measurement errors, systematic errors during data collection, and sampling errors such as incomplete spatial randomization during the design of sampling programs (Unified Guidance)..
- Comparison of compliance data against an upper prediction limit assumes that the two populations being compared have similar variances. This condition can be assessed using a homogeneity of varianceThe square of the standard deviation (EPA 1989); a measure of how far numbers are separated in a data set. A small variance indicates that numbers in the dataset are clustered close to the mean. test, but will be difficult to test directly unless you have at least four independent observations from each population (background and compliance).
Requirements and Tips
- A minimum of 8-10 values is recommended, a larger data set may be required if data are skewed or contain nondetectsLaboratory analytical result known only to be below the method detection limit (MDL), or reporting limit (RL); see "censored data" (Unified Guidance)..
- The number of future comparisons (m) against the prediction limit must be pre-specified.
- Evaluate the distribution of the data (for example, test for normality) to determine whether a parametric or nonparametric model is appropriate. It may also be possible to transform data to fit a normal distribution.
- Check for temporal correlationAn estimate of the degree to which two sets of variables vary together, with no distinction between dependent and independent variables (USEPA 2013b). and rule out the presence of temporal trends in the background data.
- If you suspect outliers, examine the background data using a probability plot, Dixon's test, Rosner's test, or another appropriate method. See Section 5.7 for information regarding the treatment of nondetects.
- Check for spatial variation before using as an interwell test in detection monitoring.
- A confidence levelDegree of confidence associated with a statistical estimate or test, denoted as (1 – alpha) (Unified Guidance). must be pre-specified when constructing a parametric prediction limit. For nonparametric prediction limits, the achieved confidence level is computed after the fact based on the available sample size.
Strengths and Weaknesses
- A prediction limit estimates a firm ‘cap’ on the background population for a specified number of future sampling events (comparisons). It thus allows for clear interpretation of when background levels have been exceeded.
- Prediction limits have the advantage of incorporating a formal re-testing strategy into the calculation of the test statistic, making it possible to precisely control false positive rates and statistical powerStrength of a test to identify an actual release of contaminated groundwater or difference from a criterion (Unified Guidance)..
- Prediction limits for future means are a powerful but more flexible alternative to analysis of variance (ANOVA)A statistical method for identifying differences among several population means or medians., more statistically powerful than prediction limits for future individual observations, and more adaptable to groundwater sampling than ANOVA.
- Nonparametric prediction limits typically require a much larger sample size than parametric prediction limits to achieve the desired confidence level. However, the formal retesting strategy can partially mitigate this requirement.
- Prediction limits are only ‘valid’ for a pre-specified number of future comparisons; when those comparisons have been exhausted, the prediction limit may need to be recomputed and a new test set up.
Further Information
Prediction limits are discussed in Chapter 18, Unified Guidance. Parametric prediction limits are discussed in Chapter 18.2, Unified Guidance. Nonparametric prediction limits are discussed in Chapter 18.3, Unified Guidance. See Example 18-1 and Example 18-2, Unified Guidance for applications of parametric prediction limits for future values and for a future mean, respectively. See also Example 18-3 and Example 18-4, Unified Guidance for the application of nonparametric prediction limits for future values and for a future mean, respectively.
Publication Date: December 2013