Defense Acquisition Research Journal Issue 95

January 2021

The Three Systems Are Statistically Different with No Intervening Variables The research questions we are attempting to answer in this section are: • Will different users of the technology with different backgrounds affect the results? That is, are there any controllable or blocking variables that need additional attention? Using the ANOVA with Blocking Variables model, we see the results in Table 8. In the experiment, the active-duty military either had experience with similar technology or they did not. The ANOVA test is run with blocking or controlling the user background.

TABLE 8. ANOVA WITH RANDOMIZED BLOCKS Model Inputs: VAR296; VAR297; VAR298 SUS(A), SUS(B), SUS(C) ANOVA Randomized Blocks Multiple Treatments DF SS MS

F Stat

P -value

Block Factor (Row)

18 4384.65

243.59

1.5282

0.1367

0.0000

Treatment Factor (Column)

2

11369.96

5684.98

5.6650

Error

36 5738.38

159.40

Total

56 21492.98

F Critical (Treatment) @ 0.01

5.247893

F Critical (Blocking) @ 0.01 2.479730 Note. SUS = System Usability Score (for systems A, B, and C). We conclude that:

• The treatment factor indicates that statistically significantly different results are shown among the three systems, but whether a soldier has experience with similar technology does not affect the results. Nonparametric Kruskal–Wallis The research question we are attempting to answer in this section is: • Does a nonparametric approach yield different results than a parametric model? Table 9 shows the results from the nonparametric Kruskal–Wallis test. As discussed, this test is the nonparametric equivalence of the ANOVA. Researchers use it to confirm the results of the ANOVA.

21

Defense ARJ, January 2021, Vol. 28 No. 1 : 2-39