Probability of Causation (PC) News & Updates

Increasing the Accuracy and Precision of Probability of Causation

August 2024

NIOSH has been evaluating the accuracy and precision of the 99th percentile Probability of Causation (PC) calculations. This is important when the PC value is close to the 50.0% compensation threshold. We found that we need larger sample sizes when cases are near the 50.0% compensation threshold.

Starting in June 2006, NIOSH processed claims with PC between 45% – 52% using a different methodology. The claims were processed on the IREP Enterprise Edition, using the following steps:

  1. Increase the sample size from 2,000 to 10,000 iterations.
  2. Re-run the simulation 30 more times with new random seed values.
  3. The Average of PC values from all 30 runs determines the claim outcome.

For cases with PC outside the 45% – 52% interval, the calculations used a sample size of 2,000 iterations.

We chose the original sample size of 2,000 to balance the precision and speed of the calculations. This ensured accurate results while keeping the calculation time of the PC reasonable. During the evaluations, NIOSH found a small negative bias at the 99th percentile of the PC; this was more pronounced when using a small sample size of 2,000 iterations. We found out that the bias becomes less significant as the sample size increases.

Therefore, NIOSH recommends increasing the number of iterations from 2,000 to 20,000 iterations in IREP for all cases. With modern computing power, this implementation should have a minimal programmatic impact on the PC calculation time. For most cases, the IREP computation time will increase from a few seconds to less than a minute. Using a larger sample size will reduce the bias to an insignificant level, and it will increase the PC precision.

Since June 2006, we have used the Average PC value from 30 runs with 10,000 iterations to determine a claim outcome; this method was used to increase the precision for claims with PC between 45% and 52%. The methodology has worked well for the past 18 years. It also continues to be an important part of the PC determination. At 10,000 iterations, the bias noted above is much smaller (when compared to 2,000 iterations), but it is still present. Therefore, NIOSH recommends using 30 runs with 20,000 iterations for claims in the 45% – 52% range.

For certain claims very near the compensation threshold, the uncertainty around the Average PC can still be greater than desired. Such claims include large dose uncertainty and/or many exposures. NIOSH recommends a change in the analysis methodology; the following two-step process will be used only for cases with PC between 49.5% to 50.5%:

  1. First, IREP will perform a rapid calculation to predict the sample size needed to achieve the desired margin of error of 0.1% for the Average PC.
  2. Next, IREP will use 300 runs with the sample size predicted in the previous step to calculate the Average PC value. A minimum sample size of 20,000 iterations will be used. The Average PC values will determine the claim outcome.

We will apply this new procedure only to a very small number of cases. NIOSH determined that we need this new procedure only for cases that are very close to the compensation threshold of 50.0%.

We announced these proposed changes, and the Advisory Board approved them during their meeting in August 2023.

The extensive research conducted by NIOSH on the accuracy and precision of the PC calculations is described in four papers which we listed as references below.

Soon, NIOSH will develop a program evaluation report reviewing all past claims in the 45% to 52% range. If the new methods described above will indicate a change in PC that may affect the compensation decision, NIOSH will request those claims back from DOL for re-evaluation.

Please note: We are committed to ensuring that all our content is accessible to everyone. Fully accessible versions of the documents below are being prepared and will be available soon. Thank you for your patience and understanding. If you need immediate assistance with these documents, please contact our Website Coordinator.


New Procedure for Resolving Cases Close to 50% PC

August 2024

NIOSH updated its procedure for resolving cases close to the 50% compensation threshold.

Key points:

  • More precise calculations performed for claims near the 50% threshold.

  • Multiple simulations used to ensure accurate results.

  • Optimized sample sizes used to enhance precision and minimize running time.

Background

Original Method (used at the beginning of the program starting in 2002)

All cases were run with the following procedure:

  1. Used a single simulation (or run) with sample size of 2,000.

  2. Used a random seed value of 99.

  3. PC obtained with these simulation settings determined the claim outcome.

Improved Method (used from 2004)

If the PC from the original method fell between 45% and 50%:

  1. Increased sample size from 2,000 to 10,000.

  2. Re-ran the simulation with a new random seed value.

  3. PC obtained with these simulation settings determined the claim outcome.

Multiple Runs Method (used from June 2006)

If the PC from the original method fell between 45% and 52%:

  1. Increased sample size from 2,000 to 10,000.

  2. Re-ran the simulation 30 more times with new random seed values.

  3. The Average of PC values from all 30 runs determined the claim outcome.

New Methodology as of August 1, 2024

NIOSH’s most recent method builds on the previous ones. The new method ensures that calculations are precise and reliable, with a focus on claims that are close to the 50% cutoff point for compensation.

Presentation and Approval:

  • The new method was presented to the Advisory Board on August 16, 2023.

  • The Board approved the method.

  • The new method was implemented and became operational on August 1, 2024.

  • The NIOSH-IREP User’s Guide was revised to reflect the updates.

Main changes introduced by the new method (starting August 2024)

  • The default sample size is increased to 20,000 for all claims.

  • If the PC falls between 45% and 52%:

    • 30 more simulations are run with new random seed values.

  • If the PC falls between 49.5% and 50.5%:

    • 300 additional simulations are run with new random seed values.

Processing Claims with One Primary Cancer

Initial Review

  • All claims are first run with a sample size of 20,000 and a random seed of 99.

  • The PC obtained determines the claim outcome (unless this value is between 45% and 52%).

Increased Precision (if needed)

If the PC obtained based on one run falls between 45% and 52%:

  • Use 30 additional runs of sample size 20,000, and each run will use a new random seed value.

  • The Average PC from the 30 runs will determine the claim outcome (unless this average is between 49.5% and 50.5%).

Enhanced Precision (if needed)

If the average of the PC values from the 30 samples falls between 49.5% and 50.5%:

  • The IREP Predictive Tool will be used by NIOSH to determine the sample size needed for enhanced precision.

  • If the sample size estimated by the IREP Predictive Tool is less than 20,000, a sample size of 20,000 will be used for the 300 runs.

  • Run 300 simulations with the recommended sample size.

  • Calculate the Average PC value from the 300 runs.

  • The average PC value from the 300 runs determines the claim outcome.

Processing Claims with Multiple Primary Cancers

Initial Review

  • Each cancer for a claim with multiple primary cancers is first processed using one run with a sample size of 20,000, and a random seed value of 99.

  • The PC values from all cancers are combined in IREP using the Multiple Primary Cancers Equation.

  • The Combined PC obtained using the Multiple Primary Cancers Equation determines the claim outcome (unless this value is between 45% and 52%).

Increased Precision (if needed)

If the Combined PC obtained from the initial review falls between 45% and 52%:

  • Use 30 additional runs of 20,000 for each cancer, and each run will use new random seed values.

  • The Average PC from the 30 runs for each cancer are calculated.

  • The Average PC values from all cancers are combined in IREP using the Multiple Primary Cancers Equation.

  • The Combined PC value obtained using the Multiple Primary Cancers Equation determines the claim outcome (unless this value is between 49.5% and 50.5%).

Enhanced Precision (if needed)

If the Combined PC obtained from the 30 runs falls between 49.5% and 50.5%:

  • The IREP Predictive Tool will be used by NIOSH for each cancer to determine the sample size needed for an enhanced precision.

  • The maximum value of all the sample sizes suggested by the IREP Predictive Tool for each cancer will be used as the new sample size for all cancers in the next re-run.

  • If the maximum value of all the sample sizes is less than 20,000, use a sample size of 20,000 for the 300 runs.

  • For each cancer, run 300 more simulations with the maximum sample size previously determined.

  • For each cancer, calculate the Average PC value from the 300 runs.

  • The Average PC values from all cancers are combined in IREP using the Multiple Primary Cancers Equation.

  • The Combined PC value obtained using the Multiple Primary Cancers Equation determines the claim outcome.

IREP Predictive Tool: Ensuring Precise Results

NIOSH developed a special tool to determine the optimal number of iterations needed for accurate results when the PC is very close to 50% (between 49.5% and 50.5%).

Key points:

  • The tool ensures precise results by adjusting the number of samples used in a simulation.

  • PC precision increases with increasing number of samples in a simulation.

How it works:

  1. Run 300 simulations with a sample size of 1,000.

  2. Using a prediction formula, the IREP Predictive Tool estimates the number of samples needed to achieve a very high precision for the PC value (0.1% margin of error).

  3. If the sample size estimated by the IREP Predictive Tool is less than 20,000, then a minimum sample size of 20,000 will be used to run the 300 additional IREP runs, either for claims with a single primary cancer or for claims with multiple primary cancers.

When is it used?

  • Only for claims with PC values between 49.5% and 50.5%.

  • For both single and multiple primary cancer claims.

Output:

  • The tool provides an estimate of the sample size needed for very accurate results.

  • NIOSH includes this information in the dose reconstruction input file sent to DOL.

The following Program Evaluation Report details the effect of this new procedure on previous non-compensable cases.


Technical Amendments to the Guidelines for Determining the Probability of Causation

August 2019

On August 1, 2019, the Department of Health and Human Services (HHS) is revising its regulations to update references to the International Classification of Disease (ICD) codes from ICD–9–CM to ICD–10– CM, and remove outdated references to chronic lymphocytic leukemia from Energy Employees Occupational Illness Compensation Program regulations. These technical amendments have no effect on the cancer eligibility requirement under the Program because all cancer types are eligible to receive a dose reconstruction from NIOSH. Thus, no eligible claimant will be adversely impacted by this rulemaking.


Dependence of Cancer Risk on Dose and Dose Rate of Low-LET Radiation

June 2017

This report describes a study for the National Institute for Occupational Safety and Health (NIOSH) to re-evaluate dose and dose-rate effectiveness factors (DDREFs) for low-LET radiation (photons and electrons) that are incorporated in cancer risk models in the Interactive RadioEpidemiological Program (IREP). The objective of this study was to develop recommendations that provide unbiased representations of the current state of knowledge of DDREFs for low-LET radiation.


Revision of Guidelines on Non-Radiogenic Cancers

March 2012

On March 21, 2011, the Department of Health and Human Services (HHS) proposed to treat chronic lymphocytic leukemia (CLL) as a radiogenic cancer under the Energy Employees Occupational Illness Compensation Program Act of 2000 (EEOICPA or the Act) (76 FR 15268).

CLL is now treated as being potentially caused by radiation and as potentially compensable under the Act. This reverses the earlier decision by HHS to exclude this cancer from consideration. The final rule was published on February 6, 2012. This change became effective on March 7, 2012.

Please Note: The Final Rule does not add CLL to the list of “specified cancers” or qualifying cancers for the Special Exposure Cohort.

Public comment on this rulemaking closed on June 20, 2011. A complete electronic docket containing reviews of CLL radiogenicity and the CLL Risk Model and public comments can be found on the NIOSH Docket page, under Docket Number 209: PC – Non-radiogenic Cancer Reconsideration.


Changes to the NIOSH-IREP Lung Cancer Risk Model

December 2010


Radiogenicity of Specific Cancers

December 2009


Changes to the Dose Reconstruction Target Organ Selection for Lymphoma

March 2007


Revision 1985 NIH Radioepidemiological Tables

September 2003