Example 3: Gage Repeatability and Reproducibility

Ribbon bar. Select the Home tab. In the File group, click the Open arrow and on the menu, select Open Examples to display the Open a Statistica Data File dialog box. Open the Temperat.sta data file, which is located in the Datasets folder. Then, select the Statistics tab. In the Industrial Statistics group, click Process Analysis to display the Process Analysis Procedures Startup Panel.

Classic menus. On the File menu, select Open Examples to display the Open a Statistica Data File dialog box. Open the Temperat.sta data file, which is located in the Datasets folder. Then, on the Statistics - Industrial Statistics & Six Sigma submenu, select Process Analysis to display the Process Analysis Procedures Startup Panel.

Double-click Gage repeatability & reproducibility to display the Repeatability & Reproducibility Analysis - Generate design dialog box.

Now, suppose you have five engineers who are routinely involved in the production process. Those engineers will serve as your operators of the gages; thus, enter 5 in the Number of operators box.

Also, assume that within the available time frame, you can manage to run a study where each engineer measures 8 kilns (parts) three times (3 trials). Therefore, enter 8 as the Number of parts and 3 as the Number of trials in the respective boxes on the Generate design tab. The Repeatability & Reproducibility Analysis dialog box - Generate design tab will now look like this:

The design of the R & R study can be displayed (and saved) either in the Standard Statistica data file format (using the Data files tab), or the Standard gage R & R data sheet format (using the R & R data sheets tab) without any grouping or coding variables (see, Gage Repeatability and Reproducibility Overview for details).

For this example, choose the Standard Statistica data file format, i.e., use the Data files tab. In this format, the design summary can be displayed in the spreadsheet in two ways: in Randomize trials or in Standard order, which is selected by default. Leave the default option button selected, and then click the Summary: Display design button in the Standard Statistica data file format group box to create the spreadsheet.

Part of the Standard order design summary spreadsheet is shown below:

When you select the Randomize trials option button, Statistica will randomize the parts within operators and trials. It is always recommended to randomize the experiment in this manner in order to rule out any serial effects.

For example, operators may tire, and measurements taken later in the experiment may be less accurate than those taken early on. Part of the Randomize trials design spreadsheet is shown below.

As you can see, in this format there are no grouping variables (columns) in the spreadsheet; instead, each column represents the data for one part. This format of presenting the R & R experiment is often used in the applied literature (e.g., ASQC/AIAG, 1990).

Click the Cancel button in the Repeatability & Reproducibility Design dialog box to close it and return to the Repeatability & Reproducibility Analysis - Generate design dialog box. Select the Analyze data file tab.

The operator names (codes) are given in variable OPERATOR, the part numbers are given in variable PART, the trial numbers (codes) in variable TRIAL, and the measurements in variable MEASURE. Click the Variables button to display a standard variable specification dialog box and then specify those variables, as shown below.

Click the OK button to return to the Analyze data file tab. Then, click the Codes: (for operators, parts, trials) button, and in the resulting dialog box, select all codes by either clicking the All button for each variable or the Select All button to select all codes for all of the variables.

Click the OK button to return to the Analyze data file tab, which will now look like this:

Finally, click the OK button to proceed to the Gage Repeatability & Reproducibility Results dialog box.

This dialog box contains options to review the results of the analysis. To graphically view the results of this study, you can choose from several types of plots. First, click the Repeatability & reproducibility plot button on the Quick tab.

For example, operator Hill "sticks out," in that his measurements are generally below the average measurements of the same parts by other engineers. The average deviation of the respective operator's measurements is also indicated by the dashed horizontal line in each box.

After returning to the Results dialog box, select the Advanced tab. Click the Complete ANOVA table button. Two spreadsheets will be produced; the first one shows the sums of squares for all effects.

If you are not familiar with the ANOVA method or with the concept of main effects and interactions, it is recommended that you read the Introductory Overview to the ANOVA/MANOVA module, which discusses these concepts and provides examples.

It is customary in R & R studies to regard the variability due to the interactions involving the Trials factor as error variability. This assumption seems reasonable, since, for example, it is difficult to imagine how the measurement of some parts will be systematically different in successive trials, in particular when parts and trials are randomized. The next spreadsheet shows the ANOVA results, treating all interactions by Trials as error.

Looking at the results in this spreadsheet it appears that the Operator by Parts interaction is not Statistically significant. Thus, we could ignore this interaction and consider a simpler ANOVA model without this interaction (select the No 2-way (Operator-Part) interaction check box on the Advanced tab).

Now review the right-most columns of the spreadsheet shown above. In these columns you find the estimates for the variances (and standard deviations) for the components of interest. Once again, consider what you would like your ideal measurement system to look like. Ideally, if you had a perfectly repeatable and reproducible measurement system, then all operators would arrive at identical measurements regardless of trial. Therefore, there would be no variability due to operators (perfect reproducibility), no variability due to trials (perfect repeatability), but only variability due to parts.

The last column of numbers reports the variability due to different sources relative to the total variability in the measurements: Repeatability of measurements accounts for 6.5%, reproducibility across appraisers accounts for 8.1% of the total variability, the part-to-part variation accounts for 85.4%, and the combined repeatability and reproducibility variability accounts for about 14.6% of the total process variability. Thus, most of the variability in measurements is due to differences between parts, as is desirable for a reliable measurement system. Using the common guidelines for evaluating the quality of the measurement system (under 10% = OK, 10% to 30% = questionable, above 30% = needs improvement; see ASQC/AIAG, 1991, page 127), these percentages indicate that the performance of the measurement system is acceptable.

You could now proceed to use this measurement system to put a quality control system (chart) in place (use the Quality Control Charts module), to evaluate your machine capability, or to use designed experiments to improve the quality of your process (use the Experimental Design module). Now, review some additional results available from the Gage Repeatability & Reproducibility Results dialog box.

To identify outliers with regard to measurement precision, you want to chart the variability of measurements across trials. Two standard charts for controlling the variability of a process are the R chart of ranges and the S (sigma) charts of standard deviations; both can be produced by operators or by parts.

On the Descriptives/plots tab, click the Sigma chart by operator button to produce the plot shown below.

For example, as you saw in the summary plot produced earlier, engineer Miller seemed to have produced the least variability across trials; perhaps by observing how engineer Miller is using the temperature gages, you can find out how to make your measurement system even more precise.

After returning to the Descriptives/plots tab, click the Box & whisker plot button.

For each operator, this plot summarizes the range of average measurements (averaged across trials) as well as the distribution of those average measurements. In this case, for each operator the median seems to fall in the upper part of each box. The median statistic and the quartile ranges are discussed in greater detail in the Basic Statistics and Tables Introductory Overview, and in the Nonparametrics Introductory Overview. In short, each box denotes the range of values into which the center 50 percent of all measurements fall. The median itself "splits" the distribution in half; that is, it is the point below and above which 50% of all measurements fall. In a normal distribution, the mean is equal to the median and would fall in the center of each box. In our plot, however, the distributions appear to have a "long tail" towards the lower end of the measurements.

In this particular example, the quality control engineer might want to look at the distributions of measurements more closely to determine how serious is the deviation from the normal distribution. In general, unless the distribution is clearly not normal, most procedures are not seriously affected. If such cases should occur though, normality can usually be achieved by applying appropriate transformations to the measurements. For example, log transformations will "pull in" the lower tail of the distribution, etc.