Example 11: Unstacking Data in Attribute Agreement Analysis

AttributeAgreement.sta and AttributeAgreement-Unstacked.sta both contain similar data. A company is training five new appraisers for the written portion of a standardized essay test. The ability of the appraiser to rate essays relative to standards needs to be assessed. Fifteen essays were rated by each appraiser on a five point scale (-2,-1, 0, 1, 2).

Stacked data set AttributeAgreement.sta contains four columns: Appraiser, Sample, Rating, and Attribute. Unstacked data set AttributeAgreement-Unstacked.sta contains eight columns: Sample, Appraiser 1 Trail 1, Appraiser 2 Trail 1, Appraiser 3 Trail 1, Appraiser 4 Trail 1, Appraiser 5 Trail 1, Attribute, and Appraiser.

For this unstacking example, open data file AttributeAgreement-Unstacked.sta via the File - Open Examples menu; it is in the Datasets folder.

From the Statistics - Industrial Statistics & Six Sigma submenu, select Process Analysis to display the Process Analysis Procedures Startup Panel.

Select Attribute agreement analysis, and click the OK button to display the Attribute agreement analysis dialog.

Select the Multiple columns (Unstacked data) option button.

Click the Variables button, and in the variable selection dialog, select Appraiser as the Appraiser variable, Sample as the Samples variable, Appraiser 1-5 as the Appraiser's response variables, and Attribute as the Standard(optional) variable.

Click OK in the variable selection dialog to return to the Attribute agreement analysis dialog. Change the Number of appraisers from 1 to 5. Leave the Number of trials at 1.

Now select the

Advanced

tab. Since the data are ordered, select the Categories of the attribute data are ordered check box.

Click OK to run the analysis and display the Attribute agreement results dialog.

Click the Agreement assessment graphs button to create a graph showing the percent of agreement for each appraiser with the standard and associated 95% confidence intervals.