Statistical testing

Statistical trends can be analyzed in many ways. The approaches used in Health, United States to analyze trends in health measures over time depend primarily on the data source (that is, National Center for Health Statistics surveys, vital statistics, and other data sources), including data availability and comparability over time. Approaches also consider the type of dependent variable and the number of data points. With enough data points, statistical analyses can detect not only whether an increase or decrease in the dependent variable has occurred but also a change in trend. Some trends are analyzed using the weighted least squares regression method in the National Cancer Institute’s Joinpoint software version 4.9.1.0, which identifies the number and location of joinpoints (that is, inflection points) when changes in trend have occurred. For more information on Joinpoint software, see the Joinpoint Trend Analysis Software website. (Also see Sources and Definitions, Joinpoint trend analysis software; Statistical significance.)

NCHS survey data

Trends in NCHS survey data

Trend testing of NCHS survey data is based on record-level data and generally follow the steps laid out in the NCHS guidelines for analysis of trends. For more information, see Ingram DD, Malec DJ, Makuc DM, Kruszon-Moran D, Gindi RM, Albert M, et al. National Center for Health Statistics guidelines for analysis of trends. National Center for Health Statistics. Vital Health Stat 2(179). 2018. When 4 or more time points are available, the data are first assessed for the presence of a nonlinear trend polynomial regression (SUDAAN PROC REGRESS). Linear, quadratic (when assessing 7 or more time points), and cubic (when assessing 11 or more time points) trends are tested in separate regression models. Quadratic trends are tested with both linear and quadratic terms in the model, and cubic trends are tested with linear, quadratic, and cubic terms in the model.

If a cubic trend is statistically significant, Joinpoint software is used to search for the location of up to two inflection points with as few as two observed time points allowed in the beginning, middle, and ending line segments (not counting the inflection points). Although this exceeds the software default of one inflection point for analyses using 11 time points, the NCHS trends analysis guidelines (see: National Center for Health Statistics Guidelines for Analysis of Trends) state this is not a problem for the analysis of record-level survey data, because appropriate survey analysis software is used as a follow-up to the Joinpoint software analysis. If a cubic trend is not statistically significant and a quadratic trend is, Joinpoint is used to search for the location of up to one inflection point in the trend. In each case, an overall p value of 0.05 and the grid search method are used. If neither a cubic nor quadratic trend is statistically significant in the nonlinearity assessment—that is, there is no inflection point—then Joinpoint is not used for further analysis.

In all Joinpoint analyses of survey data, the Bayesian information criterion (BIC) model is used because it increases the sensitivity to detect potential inflection points. Because Joinpoint is not able to fully account for the complex survey design, the presence and location of inflection points are verified in SUDAAN, which does account for survey design. The difference in slopes between two segments on either side of a potential inflection point is assessed using piecewise linear regression (SUDAAN PROC REGRESS). To conduct piecewise linear regression of age-adjusted estimates, survey weights are adjusted for age. For more information about this survey adjustment, see Li X, Bush MA. Approaches for performing age-adjustment in trend analysis. Joint Statistical Meetings 2019, Proceedings of the American Statistical Association: 741–50. Denver, CO. 2019.

When two or three time points are available, pairwise differences between two percentages are tested using two-sided significance tests (z tests). The differences between two time points are assessed for statistical significance at the 0.05 level using z tests without correction for multiple comparisons. (Also see Sources and Definitions, Statistical reliability of estimates.)

Vital records

Trends in vital statistics data

Analyses of birth data, infant mortality, and death rates using vital statistics data from the National Vital Statistics System follow the NCHS guidelines for analysis of trends and use aggregated point estimates and their standard errors rather than record-level data. For more information, see Ingram DD, Malec DJ, Makuc DM, Kruszon-Moran D, Gindi RM, Albert M, et al. National Center for Health Statistics guidelines for analysis of trends. National Center for Health Statistics. Vital Health Stat 2(179). 2018. Increasing or decreasing trends in the estimates are assessed using Joinpoint and the grid search method with an overall p value of 0.05. In analyses with fewer than 10 time points, BIC is used to select the model. In analyses with 10 or more time points, the permutation test is used to select the model. The maximum number of inflection points searched for is limited to 1, the software default when 11 time points are included in any analysis. The NCHS guidelines for analysis of trends (see: National Center for Health Statistics Guidelines for Analysis of Trends) recommend against searching for more inflection points than the software default for vital statistics data, because this increases the likelihood of estimation issues. As few as two observed time points are allowed in beginning and ending line segments (not counting the inflection points). Trend analyses using Joinpoint are carried out on the log scale for birth, infant mortality, and death rates so that results provide estimates of average annual percent change.

Note that all calculations described in this section are performed on the most accurate unrounded values available while using SUDAAN or Joinpoint to ensure the most accurate test results. Where possible, estimates and standard errors are to five or more decimal places. Final published content (figures and data sets) may have been rounded for presentation purposes. Using these rounded figures to reproduce calculations may lead to slightly different results.

Other data sources

Trends in other data sources

In analyses of other data sources, the difference between two points is assessed for statistical significance using either z tests, when standard errors are available, or the statistical testing methods recommended by the data systems. For analyses of two time points, the differences between the two points are assessed for statistical significance at the 0.05 level using z tests without correction for multiple comparisons. For other data sources, significant comparisons are generally based on the recommendations of the sources. For health expenditure, physician, and dentist data, changes are based on absolute differences. For HIV cases, differences greater than 5% are generally discussed in the text. For life expectancy, changes of 0.1 year or greater are usually discussed. For other data sources with no standard errors, relative differences greater than 10% are generally discussed.