Defense Acquisition Research Journal Issue 95

January 2021

We conclude that: • Comparable to the ANOVA (from Table 6), the Kruskal–Wallis results show that out of the 43 models, the same 21 combina tions have statistical significance. Table 10 shows the additional results from the nonparametric Kruskal– Wallis test. Similar to the ANOVA, the Kruskal–Wallis shows that out of the 33 models, the same 16 combinations show statistical significance.

TABLE 10. ANOVA AND KRUSKAL–WALLIS II

ANOVA

KW

ANOVA & KW on VAR15; VAR115; VAR415

0.0000

0.0000

ANOVA & KW on VAR16; VAR116; VAR416

0.0000

0.0000

ANOVA & KW on VAR17; VAR117; VAR417

0.0000

0.0000

ANOVA & KW on VAR18; VAR118; VAR418

0.0000

0.0000

ANOVA & KW on VAR19; VAR119; VAR419

0.0008

0.0015

ANOVA & KW on VAR20; VAR120; VAR420

0.0003

0.0000

ANOVA & KW on VAR21; VAR121; VAR421

0.0001

0.0003

ANOVA & KW on VAR22; VAR122; VAR422

0.0000

0.0000

ANOVA & KW on VAR23; VAR123; VAR423

0.0001

0.0000

ANOVA & KW on VAR24; VAR124; VAR424

0.0000

0.0000

ANOVA & KW on VAR28; VAR128; VAR428

0.0232

0.0128

ANOVA & KW on VAR29; VAR129; VAR429

0.0157

0.0127

ANOVA & KW on VAR31; VAR131; VAR431

0.0114

0.0085

ANOVA & KW on VAR35; VAR135; VAR435

0.0089

0.0008

ANOVA & KW on VAR38; VAR138; VAR438

0.0472

0.0631

ANOVA & KW on VAR43; VAR143; VAR443

0.0324

0.0614

The Data Are Reliable and Valid The research question we are attempting to answer in this section is: • Are the collected data reliable and valid for the research? The Interrater Reliability Test with Interclass Correlation (ICC) tests were run to determine if the data received were statistically reliable (Table 11). As mentioned, the ICC tests the reliability of the users’ ratings by comparing the variability of all the ratings of the same subject to the total variation across all ratings and all users simultaneously. A high ICC indicates a high level of reliability (Mun, 2018).

23

Defense ARJ, January 2021, Vol. 28 No. 1 : 2-39