NIOSH Conformity Assessment Notice

Upon completion of the system development, DSA delivered the LabVIEW™ test setup to NPPTL. The system was initially setup at NPPTL’s Pittsburgh laboratory. To verify the LabVIEW™ system produced statistically similar results to the Gould system, testing was conducted using two readily available Self-Contained Breathing Apparatus (SCBAs), a Scott Air Pak 75 SCBA with a 60-minute cylinder and an MSA Firehawk® M7 SCBA with a 30 or 45-minute cylinder. Both units met NFPA 1981-2007. The units were tested on the Gould RS3200 system and the LabVIEW™ based system. Figures 1a and 1b show the corresponding test setup.

Figure 1a: NPPTL Gould Strip Chart Setup

Figure 1b: NPPTL LabVIEW™ Strip Chart Setup

The units were tested under baseline conditions as well as under a condition where the zero point was adjusted to approximately +0.1 to +0.2 inches of water. This condition caused negative spikes to be displayed on the Gould RS3200 that had a minimum detectable area below the zero line. This is critical for accepting a negative spike on the Gould system at a speed of 20 mm/sec.

During the testing, the LabVIEW™ strip chart recorder display was adjusted to match the Gould speed. It was verified that the minimum detectable negative spikes produced on the Gould recorder were logged as negative spikes that met the criteria for being reported and logged as failures. The comparison verified that the minimum and maximum pressures recorded during the testing matched between the Gould recorder and the LabVIEW™ recorder at a tolerance of +/- 0.05 inches of water. The shapes of the breathing curves were also subjectively evaluated to ensure a match between the two methods. After this preliminary work was completed, several additional tests were performed on other approved makes and models of SCBAs.

All of the aforementioned tests were performed by running the Gould system and the LabVIEW™ system simultaneously. Results from the two recordings were compared for minimum and maximum pressures, and both measurements were within +/- 0.05 inches of water.

After the Gould system and LabVIEW™ system were validated at the NPPTL Pittsburgh laboratory, it was time to collect data at the NPPTL Morgantown laboratory. A second LabVIEW™ system, as shown in Figure 1b, was setup in the Morgantown laboratory. A detailed test plan was written which included the Gould system in Pittsburgh, the LabVIEW™ system in Pittsburgh, and the LabVIEW™ ™ system in Morgantown.

The test plan included eight SCBAs. Again, these SCBAs were chosen as they were readily available. The SCBAs chosen included Sperian-Honeywell (1 unit), Dräger (1 unit), Avon-ISI (1 unit), Interspiro (1 unit), Scott Health and Safety (3 units), and MSA (1 units). One of the Scott units was adjusted to allow the waveform to go slightly negative.

The test plan that was developed was a full comparison of the Pittsburgh laboratory to the Morgantown laboratory. Therefore, the test plan included all five of the current SCBA STPs (0120, 0121, 0122, 0123, and 0124). However, the information provided in this notice is focused on the comparison of the Gould system to the LabVIEW™ system. Therefore, the data presented is from STP-0120.

Each unit was tested three times. Each unit completed all of the tests at one laboratory before being moved to the other laboratory. All of the cylinders were charged using identical settings. The facepiece seal to the headform was tested by utilizing a vacuum delay method. The facepiece negative pressure was lowered, by vacuum, to below -1.0 inches of water. The time of the vacuum delay from -1.0 to -0.8 inches of water had to be less than 15.0 seconds. All tests were completed in the same order at both locations.

Table 1 shows the test results for the Positive Pressure Tests (STP-0120) for each unit tested. As mentioned previously, each unit was tested three times on each system. It is also worth mentioning again that the units were tested simultaneously on the Pittsburgh LabVIEW™ and Pittsburgh Gould systems. The data is shown by unit with the results of each run shown under each system.

The “Gould Difference” column shows the difference between the Pittsburgh systems only (Pittsburgh LabVIEW™ – Pittsburgh Gould). The “LabVIEW™ Difference” column shows the difference between the LabVIEW™ systems only (Morgantown LabVIEW™ – Pittsburgh LabVIEW™).

Table 1: Positive Pressure Test (STP-0120) Data (inches of water)

A statistician at NPPTL analyzed all data. Since each system was tested three times in Morgantown and three times in Pittsburgh, this type of data is referred to as “paired replicates” data. The difference within each run was used for statistical analysis. For comparing the LabVIEW™ data, the difference measure (Morgantown LabVIEW™ Data – Pittsburgh LabVIEW™ Data) was used for statistical testing. To compare the Gould Data, the difference measure (Pittsburgh LabVIEW™ Data – Pittsburgh Gould Data) was used for statistical testing.

Because of the small sample sizes (n=3) for each unit, the Wilcoxon Signed Rank (WSR) test was used to test if the difference was zero (p-value is shown in the tables). The WSR test is preferable to a parametric test (i.e. Student’s t-test with degrees of freedom = 2) when the sample size is small, because the WSR uses exact probabilities and does not require a random sample. Note that the smallest possible p-value for this test (with n=3) is 0.25.

There are four possible p-values for the WSR when the sample size is three: 0.25, 0.50, 0.75, or 1.0. There can be no differences with a traditionally statistically significant test result (with α=0.05) and if one or more of the differences are the same (tied observations) the p-value cannot be calculated. However, p-values with the smallest value of 0.25 should be considered as possibly significant differences. The estimate of the difference is a non-parametric location estimator known as the pseudo-median or Hodges-Lehmann estimator calculated for each unit separately and for all the data combined.

For all practical purposes, the estimate of the difference, whether nearly statistically significant or not, may be too small to represent any important difference or may be within the measurement error of the laboratory. The estimates of the difference and tests of significance can only provide information to aid the determination.

The Kruskal-Wallis test can be used to test if the systems are different. To determine which of the systems are different from each other, typically the confidence intervals of the differences can be compared. Again, with the small sample size of n=3, the WSR test provides (if there are no ties in the data) a 75% confidence interval instead of the more familiar 95% confidence interval. In fact, given only three observations, the 75% confidence interval is the same as the range of a sample size of 3, so the minimum and maximum of the sample are the same as the 75% confidence limits. This way, there is some indication if a particular system tends to have higher measurements in one particular location compared to the other systems.

The WSR test on the differences, using all the data for the systems combined, tests if the Morgantown laboratory values are different from the Pittsburgh laboratory values. It also estimates the magnitude of the difference. This statistic is reported at the bottom of the tables for each test. It is also possible to compare the number of positive differences to the number of negative differences to determine if there is a tendency for measurements to be higher in one location or the other. This is known as a binomial test or sign test. This result should generally agree with the WSR test; but is generally less efficient than the WSR test.

Table 2 shows the statistical analysis for the Pittsburgh data only (LabVIEW™ vs Gould). Using the WSR test, the results indicate there is no difference between the LabVIEW™ data and the Gould data (p=0.1094). The WSR test on the median estimate for the combined data (all units) is 0.00, which indicates that the LabVIEW™ data is not statistically different from the Gould data.

Table 2: Positive Pressure Test (STP-0120) Statistical Analysis (LabVIEW™ vs Gould)

Out of the 24 difference estimates (LabVIEW™ – Gould) there were six negative values, 17 zero values, and one positive value suggesting that there is a tendency for higher measurements with the Gould system.

Examining each of the eight systems separately, the median estimate of the difference was zero (indicating no difference between the LabVIEW™ data and Gould data) for five systems (Dräger, Avon-ISI, Scott NXG, Scott 2.2, and MSA). The median estimate of the difference was negative (indicating higher results for the Gould recorder) for three of the systems (Sperian-Honeywell, Interspiro, and Scott 4.5). In order, from smallest to largest the system estimates are -0.05, -0.0125, -0.0125, 0, 0, 0, 0, and 0 for the respective systems: Sperian-Honeywell, Interspiro, Scott 4.5, Avon-ISI, Scott NXG, Scott 2.2, MSA. and Dräger.

A Kruskal-Wallis test comparing the difference between the LabVIEW™ data and the Gould data indicates there is no statistically significant difference between the two systems (p=0.2100).

Table 3 shows the statistical analysis for the LabVIEW™ data only (Morgantown vs Pittsburgh). Using the WSR test, the results indicate that the Morgantown results are higher than the Pittsburgh results (p=0.0224). The WSR test on the median estimate for the combined data (all units) is 0.0250, which indicates that the Morgantown measurements are higher than the Pittsburgh measurements.

Table 3: Positive Pressure Test (STP-0120) Statistical Analysis (LabVIEW™ Only)

Out of the 24 difference estimates (Morgantown – Pittsburgh) there were two negative values, 13 zero values, and nine positive values suggesting that there is a tendency for higher measurements in Morgantown.

Examining each of the eight systems separately, the median estimate of the difference was negative (which would indicate higher results for Pittsburgh) for only one of the systems (Dräger), for which two of the three data points were zero. The median estimate was zero (indicating no difference between Morgantown and Pittsburgh estimates) for two of the systems (Avon-ISI and Scott 2.2) for which all three data points were zero. The median estimate of the difference was positive (indicating higher results for Morgantown) for five of the systems (Sperian-Honeywell, Interspiro, Scott NXG, Scott 4.5, and MSA).

A Kruskal-Wallis test comparing the difference between Morgantown and Pittsburgh indicates there is no statistically significant difference among the systems (p=0.1102).

To address the need to potentially replace a piece of aging test equipment (Gould RS3200) that can no longer be purchased or repaired, NPPTL had a PC based LabVIEW™ program developed. During this time, NPPTL also setup a new laboratory in Morgantown, WV. Eight SCBAs were used to conduct a correlation study between the LabVIEW™ system and the Gould system, as well as the LabVIEW™ systems in Pittsburgh and Morgantown. Statistical analysis of the data indicates that there is no statistically significant difference (p=0.2100) in the data comparing the LabVIEW™ system to the Gould system in Pittsburgh. Likewise, there was no statistically significant difference (p=0.1102) in the data comparing the LabVIEW™ system in Pittsburgh to the LabVIEW™ system in Morgantown.