Материал: [lect] Grubbs - Procedure for Detecting outlying observations in samples (1969)

Внимание! Если размещение файла нарушает Ваши авторские права, то обязательно сообщите нам
criterion S2.,/S2

10 FRANK E. GRUBBS

or with Dixon's r22. Omitting

-1.40"

and renumbering the observations, we

compute x = 1.67/14 = .119, s = .401, and

 

T4

1.01 -

.119

= 2.22

.401

 

.401

 

From Table 1, for n = 14, we find that a value as large as 2.22 would occur by chance more than 5% of the time, so we should retain the value 1.01 in further calculations. We next calculate Dixon's sample criterion:

 

x14 -

x 2

1.01

-

.48

.53

or

Xr22x-

x3

1.01

+

.24

1.25

 

 

 

 

 

 

r22 = .424

 

 

 

 

 

From Table 2 for n =

14, we see that the 5% critical value for r22is .546. Since

our calculated value

(.424) is less than the

critical value, we also retain 1.01

by Dixon's test, and no further values would be tested in this sample. (Note 3.) 4.8 We next turn to the case where we may have the two largest or the two smallest observations as probable outliers. Here, we employ a test provided by Grubbs [8] which is based on the ratio of the sample sum of squares when the two doubtful values are omitted to the sample sum of squares when the two doubtful values are included. If simplicity in calculation is the prime require- ment, then the Dixon type of test (actually omitting one observation in the sample) might be used for this case. In illustrating the test procedure, we give

the following Examples 4 and 5.

Example 4

In a comparison of strength of various plastic materials, one characteristic studied was the per cent elongation at break. Before comparison of the average elongation of the several materials, it was desirable to isolate for further study any pieces of a given material which gave very small elongation at breakage com- pared with the rest of the pieces in the sample. In this example, one might have primary interest only in outliers to the left of the mean for study, since very high readings indicate exceeding plasticity, a desirable characteristic.

Following are ten measurements of per cent elongation at break made on material No. 23: 3.73, 3.59, 3.94, 4.13, 3.04, 2.22, 3.23, 4.05, 4.11, 2.02. Arranged in ascending order of magnitude, these measurements are: 2.02, 2.22, 3.04, 3.23, 3.59, 3.73, 3.94, 4.05, 4.11, 4.13. The questionable readings are the two lowest, 2.02 and 2.22. We can test these two low readings simultaneously by using the

of Table 4. For the above measurements:

s2 =

(x ~ -)2 ~ n E

x - (C

x)2 _ 10(121.3594) - (34.06)2

 

Zi

n

-

10

 

~__~ -

 

 

 

S2 =

5.351

 

 

 

Note 3: It should be noted that in a multiplicity of tests of this kind, the final overall significancelevel will be less than that used in the individualtests, as we areofferingmorethan one chanceof acceptingthe sampleas one producedby a randomoperation.It is not ourpurpose here to cover the theory of multiple tests.

 

 

 

 

 

 

 

 

 

DETECTINGOUTLYING OBSERVATIONS IN SAMPLES

 

 

11

and

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

(n -

 

n

 

 

 

n

\2

 

 

 

 

 

si 2=

C

(x

 

-

 

 

-

2)

Ex-

 

 

(1

 

 

 

 

 

 

Cf2

 

 

(Xi

z,.l)2

 

 

i'i

 

 

 

i=3

 

 

 

 

 

 

 

 

V^

 

/?

 

 

?

\2

 

 

 

 

-

 

 

 

 

 

 

 

 

 

 

 

 

i

 

 

 

 

 

 

 

 

 

(n

2)

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

(where

 

l,2

=

 

xi/(n - 2))

 

 

_8(112.3506)

 

-

(29.82)2

 

 

 

 

 

 

 

 

 

 

 

 

 

 

8

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

S2

= 9.5724

 

=

1.197

 

 

 

 

 

 

 

 

 

 

 

 

 

 

S1.2

 

 

 

8

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

TABLE

4

 

 

 

 

 

 

 

 

 

 

 

 

 

Critical Values for S_

,.I/S2 or S, 2/S2 for Simultaneously

Testing

 

 

 

 

 

 

 

 

 

 

the Two Largest or Two Smallest Observations*

 

 

 

 

 

 

 

 

Number of

 

 

10%

 

 

 

 

5%

 

 

1%

 

 

 

 

 

 

Observations

 

Significance

 

 

Significance

 

Significance

 

 

 

 

 

 

 

n

 

 

 

 

Level

 

 

 

Level

 

 

Level

 

 

 

 

 

 

 

 

4

 

 

 

 

.0031

 

 

 

.0008

 

 

.0000

 

 

 

 

 

 

 

 

5

 

 

 

 

.0376

 

 

 

.0183

 

 

.0035

 

 

 

 

 

 

 

 

6

 

 

 

 

.0921

 

 

 

.0565

 

 

.0186

 

 

 

 

 

 

 

 

7

 

 

 

 

.1479

 

 

 

.1020

 

 

.0440

 

 

 

 

 

 

 

 

8

 

 

 

 

.1994

 

 

 

.1478

 

 

.0750

 

 

 

 

 

 

 

 

9

 

 

 

 

.2454

 

 

 

.1909

 

 

.1082

 

 

 

 

 

 

 

 

10

 

 

 

 

.2853

 

 

 

.2305

 

 

.1415

 

 

 

 

 

 

 

 

11

 

 

 

 

.3226

 

 

 

.2666

 

 

.1736

 

 

 

 

 

 

 

 

12

 

 

 

 

.3552

 

 

 

.2996

 

 

.2044

 

 

 

 

 

 

 

 

13

 

 

 

 

.3843

 

 

 

.3295

 

 

.2333

 

 

 

 

 

 

 

 

14

 

 

 

 

.4105

 

 

 

.3568

 

 

.2605

 

 

 

 

 

 

 

 

15

 

 

 

 

.4345

 

 

 

.3818

 

 

.2859

 

 

 

 

 

 

 

 

16

 

 

 

 

.4562

 

 

 

.4018

 

 

.3098

 

 

 

 

 

 

 

 

17

 

 

 

 

.4761

 

 

 

.4259

 

 

.3321

 

 

 

 

 

 

 

 

18

 

 

 

 

.4914

 

 

 

.4455

 

 

.3530

 

 

 

 

 

 

 

 

19

 

 

 

 

.5113

 

 

 

.4636

 

 

.3725

 

 

 

 

 

 

 

 

20

 

 

 

 

.5269

 

 

 

.4804

 

 

.3909

 

 

 

 

 

 

 

 

 

2

 

 

 

 

1

 

 

 

 

 

 

 

 

 

."

S2

=

 

E

(Xi

-

 

 

 

 

X

= -

 

X

 

 

 

 

XI <X2

<

 

 

 

 

x)2

 

 

 

 

n

 

 

 

 

 

 

<

<,Xn

 

 

 

-1

 

 

 

 

 

 

 

 

 

i-l

 

 

 

 

 

 

 

 

 

 

 

 

 

n

 

 

 

 

 

 

 

 

 

 

 

1

 

n

 

 

 

 

 

S.2

 

 

-=

i-3

(x,

- x12)2

 

 

 

X1,2

=-

n -

2

i3E

 

 

 

 

 

 

 

 

 

 

n-2

 

 

 

 

 

 

 

 

 

 

 

 

 

n-2

 

 

 

 

 

 

 

-=n

 

E

 

(Xi

-

Xn-l,n)

 

 

 

Xn-l

,n

=

-

 

X

 

 

 

 

 

 

 

 

 

 

 

n-2

il

 

 

 

 

 

 

 

 

i=t

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

* These

significance

levels

are taken

from Table

V of Grubbs,

Reference

[8]. An observed

ratio less than the appropriate

critical ratio in this table

calls for rejection of the null hypothesis.

12

FRANK E. GRUBBS

We find

 

 

 

 

S1S2.

1.197

-224

 

.31

 

S2

5.351

 

From Table 4 for n =

10, the 5% significance level for S~,2/S2 is .2305. Since

the calculated value is less than the critical value, we should conclude that both 2.02 and 2.22 are outliers. In a situation such as the one described in this ex- ample, where the outliers are to be isolated for further analysis, a singificance level as high as perhaps even 10% would probably be used in order to get a reasonable size of sample for additional study.

Example 5

The following ranges (horizontal distances in yards from gun muzzle to point of impact of a projectile) were obtained in firings from a weapon at a constant angle of elevation and at the same weight of charge of propellant powder:

Distances in Yards

4782

4420

4838

4803

4765

4730

4549

4833

It is desired to make a judgment on whether the projectiles exhibit uniformity in ballistic behavior or if some of the ranges are inconsistent with the others. The doubtful values are the two smallest ranges, 4420 and 4549. For testing these two suspected outliers, the statistic ST,2/S2 of Table 4 is probably the best to use. (Note 4.)

The distances arranged in increasing order of magnitude are:

4420

4782

4549

4803

4730

4833

4765

4838

The value of S2 is 158,592. Omission of the two shortest ranges, 4420 and 4549, and recalculation gives S,2 equal to 8590.8. Thus,

Si,2

 

8590.8

.054

2_

 

=

S2

-

158,592

.0

which is significant at the .01 level (See Table 4). It is thus highly unlikely that the two shortest ranges (occurring actually from excessive yaw) could have come from the same population as that represented by the other six ranges. It should be noted that the critical values in Table 4 for the 1% level of significance are smaller than those for the 5% level. So for this particular test, the calculated value is significant if it is less than the chosen critical value.

Note 4: Kudo [11] indicates

that

if the

two outliers are due to a shift in location or level, as

compared to the scale a, then the optimum

sample criterion for testing should be of the type:

min. (2x - xi - xi)/s = (2x

- xi

- x2)/s

in our Example 5.

DETECTINGOUTLYING OBSERVATIONS IN SAMPLES

13

4.9 If simplicity in calculation is very important, or if a large number of samples must be examined individually for outliers, the questionable observations may be tested with the application of Dixon's criteria. Disregarding the lowest range, 4420 we test if the next lowest range 4549 is outlying. With n = 7, we see from Table 2 that r,o is the appropriate statistic. Renumbering the ranges as x, to X7, beginning with 4549, we find

x2 -

x,

4730

-

4549

-

181

=.626

rio =. 7, -

x,

4838

-

4549

289

 

which is only a little less than the 1% critical value, .637, for n = 7. So, if the test is being conducted at any significance level greater than the 1% level, we would conclude that 4549 is an outlier. Since the lowest of the original set of ranges, 4420, is even more outlying than the one we have just tested, it can be classified as an outlier without further testing. We note here, however, that this test did not use all of the sample observations.

4.10 Rejection of Several Outliers. So far we have discussed procedures for detecting one or two outliers in the same sample, but these techniques are not generally recommended for repeated rejection, since if several outliers are present in the sample the detection of one or two spurious values may be "masked" by the presence of other anomalous observations. Outlying observations occur due to a shift in level (or mean), or a change in scale (i.e., change in variance of the observations), or both. Ferguson [6, 7] has studied the power of the various rejection rules relative to changes in level or scale. For several outliers and repeated rejection of observations, Ferguson points out that the sample coefficient of skewness

Vb = V/n I (x,

-

t)3/(n

- 1)3s: = V/n

(

(-

)3/[E

(x-

)2]

i-i

 

i,-

 

 

 

 

should be used for "one-sided" tests (change in level of several observations in the same direction), and the sample coefficient of kurtosis

 

 

n

 

 

it

 

 

hb

=

n

1)s

n

-

(x

-

 

xx)]2

 

 

(x, x)/(n

 

 

 

 

 

i-l

 

 

i=l

 

 

is recommended for "two-sided" tests (change in level to higher and lower values) and also for changes in scale (variance)*. In applying the above tests, the vb, or the b2 , or both, are computed and if their observed values exceed those for significance levels given in the following tables, then the observation farthest from the mean is rejected and the same procedure repeated until no further

sample values are judged as outliers. [As is well-known

~ib, and b, are also

used as tests of Normality].

 

4.10.1 The significance levels in the following tables

for sample sizes of

5, 10, 15 and 20 (and 25 for b2) were obtained by Ferguson on an IBM 704 Computer using a sampling experiment or "Monte Carlo" procedure. The

*In the above equations for

-/b' and b2, s is defined as used in this paper, i.e.

s

=-

 

(x,i-

'X/(n

-

1)

i,,1

 

 

 

 

 

 

 

14 FRANK E. GRUBBS

significance levels for the other sample sizes are from E. S. Pearson, "Table of

Percentage Points of

/~b and b2 in Normal Samples; a Rounding Off," Bio-

metrika (1965), Vol. 52, pp. 282-285.

 

 

 

 

 

 

 

 

 

 

SignificanceLevelsfor

b

 

 

 

 

 

 

 

 

 

n

 

 

 

 

 

Sig

5*

10*

15*

20*

25

30

35

40

50

60

Level

1%

1.34

1.31

1.20

1.11

1.06

.98

.92

.87

.79

.72

5%

1.05

.92

.84

.79

.71

.66

.62

.59

.53

.49

 

 

 

SignificanceLevelsfor bl

 

 

 

 

 

 

 

 

 

 

n

 

 

 

 

Sig

 

5*

10*

15*

 

20*

 

25*

50

75

100

Level

 

 

1%

 

3.11

4.83

5.08

 

5.23

 

5.00

4.88

4.59

4.39

5%

 

2.89

3.85

4.07

 

4.15

 

4.00

3.99

3.87

3.77

* These

values

were obtained by

Ferguson,

using

a Monte Carlo procedure. For n

= 25,

Ferguson's

Monte

Carlo values of b2 agree with

Pearson's

computed

values.

 

 

4.10.2 The V/b and b2 statistics have the optimum property of being "locally" best against one-sided and two-sided alternatives, respectively. The /b, test is good for up to 50% spurious observations in the sample for the one-sided case and the b2test is optimum in the two-sided alternatives case for up to 21% "contamination" of sample values. For only one or two outliers the sample statistics of the previous paragraphs are recommended, and Ferguson [7] discusses in detail their optimum properties of pointing out one or two outliers.

5. RECOMMENDEDCRITERIONUSING INDEPENDENT

STANDARD DEVIATION

5.1 Suppose that an independent estimate of the standard deviation is avail- able from previous data. This estimate may be from a single sample of previous similar data or may be the result of combining estimates from several such

previous sets of data. In any event, each estimate is said to have degrees of freedom equal to one less than the sample size that it is based on. The proper combined estimate is a weighted average of the several values of s2, the weights being proportional to the respective degrees of freedom. The total degrees of freedom in the combined estimate is then the sum of the individual degrees of

freedom. When one uses an independent estimate of the standard deviation, s, the test criterion recommended here for an outlier is as follows:

T, = x- x

(v = total number of degrees of freedom)

or

 

-

X

T

Sn8,

Источник: https://studfile.net/preview/16673013/