Table 3 Common checklist items. Items marked with * were below our threshold for reliability, and were not included in the final rubric. Items with Kappa marked as “undefined” were performed in 100 % of cases

Discussed Code Status?

67 %

0.40

Assessed Airway?*

47 %

−0.21

Assessed Breathing?

100 %

Undefined

Assessed Circulation?*

100 %

Undefined

Assessed Disability? (Neurologic status)*

80 %

0.12

Assessed Skin/exposure?

80 %

0.56

Obtained vitals?*

100 %

Undefined

Obtained temperature?

80 %

0.66

Obtained past medical history?

100 %

1.00

Obtained medications?

93 %

0.78

Obtained allergies?

80 %

0.65

Obtained labs?

93 %

0.45

Administered oxygen?*

93 %

Undefined

Verbalized Differential Diagnosis?*

73 %

0.40

Pooled reliability:

84.8 %

0.61