Cover |
Introduction |
1a |
1b |
1c |
1d |
2a |
2b |
2c |
2d |
3
Errors
and Omissions in Experimental Trials - 2b
CRITICISMS AND COMMENTS [cont.]
DR R. M. GRAINGER
The third review of this book in the
Australian Dental Journal, by DR R. M. GRAINGER,
Division of Dental Research, Faculty of Dentistry
University of Toronto, was as follows:
Those whose work has been so
unfairly criticized might well ask P. R. N. Sutton
if he feels his own work is proof that the
unimpeachable study can be done(26), or if he would
welcome similar scrutiny of his publications.
While we do not claim to be able to
answer every question to P R. N. Sutton's
satisfaction (or even our own), in order to help set
the record as straight as possible(27) the following
are specific comments on points raised by P. R. N.
Sutton in his discussion of the Brantford Study. No
attempt is made to rationalize why specific workers
directed or restricted their research efforts or
discussion in any areas other than to comment that
they no doubt accomplished as much as they could
under the circumstances in which they had to work.
Item 1: Reference to Hutton et al.
(1951). The numbers of children of the same age
examined in the years 1944 and 1945 were very
similar with the exception of the nine-year age
group. From the unpublished data released at annual
meetings in Brantford the number of children
examined in 1944 was 239, and in 1945, 319; making a
total of 558 (not 608). The data in Table I (Hutton
et al.) were apparently combined by pooling
the two years' results not by averaging the
averages. However, the point is rather academic(28).
Item 2 : Reference to Ontario Health
Department report. The date of water-fluoridation in
Tables I and 11 was given as 1946 through a
typographical error but was twice correctly stated
to be June, 1945, in the text referring to the
Table. The small error in percentage reduction for
seven-year-olds was also conceded. These points do
not seriously underline the usefulness of the
work(29).
Item 3: Re late commencement of
National Health and Welfare study and detection of
caries protection for young individuals born prior
to commencement of fluoridation. Despite the fact
that the Department of National Health and Welfare
began its control study nearly three years after
fluoridation began, much worthwhile information was
obtained and the effect of late commencement, if
any, was to result in underestimation of the
fluoride protection(30).
Item 4: Reason for selection of
control cities. It seems clear that Brown (1951)
gave adequate reasons for selecting Sarnia and
Brantford(31).
Item 5: Re superior dental care in
Brantford. The difference in level of dental care
between the cities is factual as recorded by Brown
(1952). This variation of numbers of teeth
classified as F. rather than D. or M. does not
fundamentally influence the DMF rate(32).
Item 6: Comparability of rates. As
stated under item 5, the dental condition of the
children in Sarnia and Brantford differed in 1948
because a lower level of dental treatment in Sarnia
resulted in higher tooth mortality. The tooth
mortality rates thus differed but it does not follow
that the DMF rates differed(33). Brown's statement
(1951) "by 1948 the Brantford data were not greatly
different from those in Sarnia" is obviously
referring to DMF rates and hence quite valid(34).
The differences in oral hygiene are also only
remotely related to the DMF rates under
discussion(35).
Item 7: Concentration of fluorides.
The fluoride content of the Brantford water supply
was raised to approximately 1 ppm in June, 1945, and
raised to 1.2 in February, 1949(36) The Stratford
water fluoride content is believed to have been in
the order of 1.3 to 1.6 ppm since 1917 when the
wells were drilled. Naturally no analysis for
fluoride was available prior to the beginning of the
interest in fluoridation and early techniques for
analysis were not as reliable as present
methods(37). These facts have been recorded in the
writings of the primary workers (Hutton et al.,
1951; and Brown et al., 1951, 1952, 1953,
1954, 1956) and the differences in amounts from
other writers night seem less "strange" if they were
merely acknowledged to be minor misquotations(38).
Item 8: Re statement by Brown et
al. (1956). The statement is substantially
correct with the exception that Brown's
observational period did not begin until 1948 hence
is less than 10 years(39). The decrease in mean df
rates for the 9-11 years group in Sarnia between
years 1948 and 1954 (Brown, 1956), did not continue
into 1955(40). There was a highly significant
decrease over the period 1948 to 1955 (2.37 to 1.93)
in Brantford and no significant decrease in Sarnia
(2.50 to 2.31)(41). In the same periods the mean df
rates for this age [in] Stratford remained nearly
equal (1.66 and 1.65) and increased for other ages
(42).
Item 9: Re Table II: Ontario
Department of Health Report. The printing of dashes
rather than percentages for the control cities was
to avoid confusing the table With "negative
reductions" and in the case of the 9 to 11 df figure
to avoid emphasising what was considered to be a
spurious decrease(43). This judgment was borne out
by the 1955 figures(44).
Item 10: Differences in reported
rates between examiners. Different examiners give
characteristically higher or lower rates upon
examining the same individuals due to differences in
skill, training the physical condition. Thus the
differences quoted are no reflection on the design
of the experiment or the care taken in the work. The
strength of the double examinations comes through
corroboration of caries trends in Brantford over the
years and not through interchangeability of
data(45).
Item 11: Significant fluctuations in
controls. The important point is that for the
controls the inter-year changes were upward trends
or mere fluctuations (even though in some cases
calculated to be beyond change), whereas in
Brantford the change took the form of a highly
significant continual downward trend(46).
Item 12: Larger percentage changes
in control. There is no definite explanation as to
why rates increased in Stratford and also in Sarnia
over the ten years, but this may be a reflection of
a general post-war increase in dental caries which
has been seen in other areas. However, it is
significant that in the various fluoridation
experiments, e.g. in Brantford, Newburgh, Grand
Rapids, etc., the shift has always been
significantly downward in the fluoridated cities
whereas the control city rates have remained about
the same or in the case of Stratford, increased(47).
Item 13: Smaller percentage decrease
after longer fluoridation. As pointed out by Sutton
himself on page 168 (middle paragraph) the fluoride
protection for permanent teeth of the children aged
six to seven seemed to occur within two or three
years after fluoridation began. Thereafter the
yearly DMF rates were subject to random fluctuation
and the differences in percentage decrease of 69 per
cent and 51 per cent are most likely a reflection of
this inter-year variation(48).
Item 14: The quotation from the
Ontario Government Report is taken out of context
from a series of summary statements. The previous
statement was to the effect that in Brantford there
had been a significant decrease of about 60 per cent
in DMF rates. In the statement following, as picked
out by Sutton, it was stated that "no change"
occurred in Stratford and Sarnia(49). It should be
clear enough from the context that the words "no
similar downward change" were inferred(50).
Item 15: Possible weighting effect.
The critic is referring to a possible shift in age
distribution within the group, e.g. a possible
sampling shift within the six to eight-year age
group so that certain years had disproportionately
higher or lower numbers of eight-year-olds and hence
higher or lower average caries scores. This is
rather remote in that selection methods used by Dr
Brown were the same each year, moreover very large
shifts in age distribution would be needed to
produce the significant differences to which P N. R.
Sutton refers(51).
Commentary on the Review by Dr R. M. Grainger
(26) Dr R. M. Grainger raises the
question as to whether "the unimpeachable study can
be done". This is, of course, unlikely. It is
precisely for this reason that all papers (and these
include my own) which set out to present new
knowledge should be examined, in order to reduce the
chance that findings which are not soundly based
will be accepted at their face value. This is
particularly necessary in those studies which may
involve the health of the public.
(27) The result of Dr Grainger's
attempt "to help set the record as straight as
possible" will be judged after considering his other
remarks.
(28) Item 1, Reference to Hutton
et al. (1951); The numbers of children of the
same age examined in the years 1944 and 1945 were
very similar with the exception of the nine-year age
group. From the unpublished data released at annual
meetings in Brantford the number of children
examined in 1944 was 239, and in 1945, 319; making a
total of 558 (not 608). The data in Table I (Hutton
et al.) were apparently combined by pooling
the two years' results not by averaging the
averages. However, the point is rather academic.
Comment. The phrase "making
total of 558 (not 608)" suggests that the figure 608
was an error in this monograph. This is not the
case, in fact this figure was not mentioned. It was
published by the authors of this study, Hutton et
al., in 1951 (Table 1, column 2). Dr Grainger,
therefore, is suggesting that the total 558 children
(derived from the unpublished figures of 239 and
319) is correct, and that the figure of 608 children
examined, published by the authors of the study, is
incorrect. It should be noted that, five years after
this figure of 608 was first published, in Table II,
column 2, of their final report Hutton et al.
(1956) again published their figure of 608. In both
the tables in which it appears it has been used in
computing the def and the DMF rates. Also, if one
accepted Dr Grainger's figure of 558 as the correct
number of nine-year-old children examined in these
two years, the impossible situation would also have
to be accepted in which the number of these children
with decayed, missing or filled teeth, which Hutton
et al. (1951) gave as 595, would exceed the
number of children examined.
(29) Item 2: Reference to Ontario
Health Department report. The date of
water-fluoridation in Tables I and II was given as
1946 through a typographical error but was twice
correctly stated to be June, 1945, in the text
referring to the Table. The small error in
percentage reduction for seven-year-olds was also
conceded. These points do not seriously undermine
the usefulness of the work.
Comment. The "small" error in
percentage reduction, which, Dr Grainger said "was
also conceded", was the showing of 51 per cent
instead of 66 per cent (p. 167). Dr Grainger does
not mention here the substitution of dashes for
figures in the two cases of reduction in the caries
rate in the control cities (pp. 4, 37, 44). Several
types of errors are present in Tables I and 11: (a)
two omissions, which Dr Grainger implied-Item 9 of
this review(43)-were made deliberately; (b) two
typographical errors; (c) two arithmetical errors
(Table I, age 7, "% Reduction Since 1944-45" in the
caries attack rates should be 66, not 51, and in
Table II, age 10, the "% Reduction Since 1944" in
the caries attack rates should be 18, not 61); and
if, as appears likely, the figures given by Dr
Grainger in Item 1, of this review(28), are
incorrect and were used, (d) four incorrect mean
figures.
Dr Grainger contends that the points
which he mentioned "do not seriously undermine the
usefulness of the work"; but the occurrence, on one
page alone, of all the errors and omissions which
have just been mentioned certainly undermines
confidence in the care taken in the preparation of
this official report by the anonymous
"statisticians" of the Division of Medical
Statistics, Ontario Department of Health.
(30) Item 3: Re late commencement of
National Health and Welfare Study and detection of
caries protection for young individuals born prior
to commencement of fluoridation. Despite the fact
that the Department of National Health and Welfare
began its control study nearly three years after
fluoridation began, much worthwhile information was
obtained and the effect of late commencement, if
any, was to result in underestimation of the
fluoride protection.
Comment. Dr Grainger does not
state the nature of this "worthwhile information"
but, whatever it was, it could not compensate for
the lack of a pre-fluoridation caries assessment in
this study. Its late commencement could be justified
only if it was known that the caries rates in
Brantford had not been affected by the ingestion of
fluorides prior to the baseline examination (p.
168). However, the results from the City Health
Department study, if taken at their face value,
indicated that there had been marked and erratic
changes: at first a considerable rise in the DMF
rates after about one year of fluoridation, followed
by a marked fall during the second year. It is
surprising, therefore, that, out of all the cities
in Canada, Brantford was chosen as the location of
two long-term studies, for it should have been
obvious that the value of the second study would be
severely limited by the fact that the very important
data showing the pre-fluoridation caries rates could
never be obtained.
(31) Item 4: Reason for selection of
control cities. It seems clear that Brown (1951)
gave adequate reasons for selecting Sarnia and
Brantford.
Comment. As Dr Grainger
notes, this paragraph refers to the selection of the
control cities, which were Sarnia and Stratford-not
"Sarnia and Brantford". The sole reference to the
selection of control cities which Brown (1951) gave
is as follows: "The Ontario Dental Division, under
Dr Frank Kohli, volunteered assistance, as did Dr G.
L. Anderson, Medical Officer of Health for Sarnia,
and Dr H. B. Kenner, Medical Officer of Health for
Stratford, and both these cities entered the study
as controls. (The water of Sarnia is fluorine-free,
and that of Stratford contains 1.3 ppm. of fluorine
from a natural source.)" Dr Grainger considers that
"Brown (1951) gave adequate reasons" for the
selection of the control cities-but few would agree
with him.
(32) Item 5: Re superior dental care
in Brantford. The difference in level of dental care
between the cities is factual as recorded by Brown
(1952). This variation of numbers of teeth
classified as F. rather than D. or M. does not
fundamentally influence the DMF rate.
Comment. Increased dental
care usually includes some prophylactic treatments
and, as noted by Doctors Blayney and Hill (18). even
regular examinations may be accompanied by "more
emphasis on the teaching of oral health." This
statement by Dr Grainger implies that he considers
that such increased dental care has no influence on
the total DMF rate.
(33) Item 6: Comparability of rates.
As stated under Item 5, the dental condition of the
children in Sarnia and Brantford differed in 1948
because a lower level of dental treatment in Sarnia
resulted in higher tooth mortality. The tooth
mortality rates thus differed but it does not follow
that the DMF rates differed.
Comment. It does not state
"that the DMF rates differed; it points out, as its
title states, the "Doubtful comparability of rates"
owing to the delay in setting up this study. Dr
Grainger's comments suggest either that he has not
understood the meaning of the first sentence of the
paragraph, or that he is seeking to distract
attention from the presence of this important
deficiency in the study-its late commencement.
(34) Brown's statement (1951) "by
1948 the Brantford data were not greatly different
from those in Sarnia" is obviously referring to DMF
rates and hence quite valid.
Comment. This quotation does
not appear in Brown (1951) but a similar statement
was made by Brown et al. in 1953 and 1954 (b)
and is given on page 169. The fact that it was
"obviously referring to DMF rates" was not
questioned. The implications of this remark were
discussed.
(35) The differences in oral hygiene
are only remotely related to the DMF rates under
discussion.
Comment. Brown et al.
(1954b) said that "marked differences in oral
hygiene as between the test and control groups might
conceivably affect the findings". Such "marked
differences" were reported-but were disregarded.
(36) Item 7: Concentration of
fluorides. The fluoride content of the Brantford
water supply was raised to approximately 1 ppm in
June, 1945, and raised to 1.2 in February, 1949
Comment. This statement of Dr
Grainger is welcome because it provides the answer
to the question: Which of the statements regarding
the concentrations of fluorides in the Brantford
water, which were reported on page forty-two, are
accurate and which ones are not?
(37) The Stratford water fluoride
content is believed to have been in the order of 1.3
to 1.6 ppm since 1917 when the wells were drilled.
Naturally no analysis for fluoride was available
prior to the beginning of the interest in
fluoridation and early techniques for analysis were
not as reliable as present methods.
Comment. This statement is
most revealing for it indicates that the wells at
Stratford have been analyzed to determine their
fluoride content only since "the beginning of the
interest in fluoridation". If this is the case, the
statements of Brown et al, (1953, 1956),
concerning the "continuous" use of water containing
fluorides in concentrations of 1.3 ppm or 1.6 ppm
since 1917 are not founded on data and are,
therefore, merely different guesses.
(38) These facts have been recorded
in the writings of the primary workers (Hutton et
al., 1951; and Brown et al., 1951, 1952,
1953, 1954, 1956) and the differences in amounts
from other writers might seem less "strange" if they
were merely acknowledged to be minor misquotations.
Comment. It was pointed out
that the "facts" regarding fluoride concentrations
were stated differently in these papers. In regard
to the concentration in the Stratford supply, a
comparison of the statements made by Brown et al.,
in 1953 and 1956 suggests that the concentration of
fluorides in this supply may have increased from 1.3
to 1.6 in this three year period. The important
admission that the fluoride concentration in
Stratford was obtained only relatively recently, is
not contained in any of the six "writings of the
primary workers" mentioned by Dr Grainger. Therefore
his statement is not correct.
As Dr Grainger suggests, it is not
unlikely that the statements regarding fluoride
concentration of the "other writers", the New
Zealand Commission of Inquiry (1957) and the Ontario
Department of Health (1956), were "minor
misquotations".
(39) Item 8 Re statement by Brown
et al (1956). The statement is substantially
correct with the exception that Brown's
observational period did not begin until 1948, hence
is less than 10 years.
Comment. Dr Grainger suggests
that the phrase "more than ten years" is incorrect.
It was inserted into the quotation of a statement by
Brown et al. (1956), but enclosed in square
brackets to indicate that it was not a part of the
quotation. However, in the sentence which
immediately precedes that quotation Brown et al.
(1956) said: "Brantford has had more than 10 years
of experience with 1 part per million of fluoride in
its water supply. During that time... "It is clear
that they were not referring to "Brown's
observational period" of about seven years, but to
the period of fluoridation in Brantford which
commenced in June 1945 (Hutton et al., 1951;
p. 173) and was, therefore, "more than 10 years".
(40) The decrease in mean df rates
for the 9-11 years group in Sarnia between the years
1948 and 1954 (Brown, 1954), did not continue into
1955.
Comment. The 1955 rate of
2.31 df was still below the 1948 and the 1951
figures of 2.50 and 2.41 respectively.
(41) There was a highly significant
decrease over the period 1948 to 1955 (2.37 to 1.93)
in Brantford and no significant decrease in Sarnia
(2.50 to 2.31).
Comment. The decrease
mentioned by Dr Grainger (2.37 to 1.93) was reported
in Brantford between 1948 and 1954 (Brown et al.,
1954b) not "over the period 1948 to 1955". In 1955
this rate rose to 1.99 (Brown et al., 1956),
and the difference between 1948 and 1955 was no
longer said to be "highly significant" (Brown, 1955)
The rates quoted by Dr Grainger for
Brantford are for the years 1948 and 1954 (see 63).
It should be noted that it was between these two
years that the maximum "decrease" was reported in
the rates in that test city (2.37 to 1.93).
Furthermore, in mentioning Sarnia, instead of giving
the figures for the same period (1948-54), 2.50 to
2.11, he cited the figures 2.50 to 2.31, which cover
a different period (1948-55) and do not reveal
(Brown, 1955) the significant "decrease", in the
rate in this control city, which was shown in the
previous report (Brown et al., 1954b). By the
use of these figures, the reviewer exaggerates the
contrast between the test city and this control.
Thus, this statement by Dr Grainger is both
inaccurate and misleading.
(42) In the same periods the mean df
rates for this age [in] Stratford remained nearly
equal (1.66 and 1.65) and increased for other ages.
Comment. The rates for the
four examinations were: 1.66, 1.76, 1.58, 1.65
(Brown, 1955). (Throughout this monograph caries
rates have been given in the form in which they
appear in the original papers although it is
recognized that, in cases such as these, the
practice of showing caries rates with two places of
decimals is, probably, not warranted.) Dr Grainger
mentions the least variable of the ten caries rates
in the control cities-that for the deciduous teeth
of children aged nine to eleven years in Stratford.
He omits to mention the DMF rates which show the
remarkable situation, in this control city, in which
each of the inter-year changes occurring in this age
group, and in five out of the six inter-year changes
in the rates of the "other ages", were said by Brown
(1955) to be statistically significant.
(43) Item 9: Re Table 11, Ontario
Department of Health Report. The printing of dashes
rather than percentages for the control cities was
to avoid confusing the table with "negative
reductions" and in the case of the 9 to 11 df figure
to avoid emphasizing what was considered to be a
spurious decrease.
Comment. This astonishing
explanation, for the printing of dashes in this
table, implies that these omissions were made
deliberately because the results did not conform to
those expected. Why should a decrease of 0.44 df (18
per cent) in the test city be accepted and
published, but a very similar one of 0.39 df (16 per
cent) in a caries rate in the control city of Sarnia
be considered "spurious" and not published-a dash
being shown in the appropriate position in the
table? By printing these dashes, the "statisticians"
of the Ontario Department of Health could have
misled their Minister into thinking that there were
no changes in these caries rates in Sarnia and
Stratford (particularly as the Summary of the report
said so) but that there had been a decrease of
eighteen per cent in the corresponding rate in
Brantford due to fluoridation.
Talk of "negative reductions" cannot
disguise the fact that nothing is more calculated to
confuse a table than (as Dr Grainger implies) the
deliberate omission by "statisticians" of figures
giving the percentage changes in rates (one of which
was said to be significant).
(44) This judgement was borne out by
the 1955 figures.
Comment. Dr Grainger tries to
justify this "judgment" by implying that these
so-called "spurious" decreases were not seen in the
1955 figures. However, small "decreases" were still
shown in that year, the "reduction" in Sarnia being
7.6 per cent.
In any case, this so-called
"judgment", regarding the omission of data, has no
place in the preparation of an unbiased report.
(45) Item 10: Differences in
reported rates between examiners. Different
examiners give characteristically higher or lower
rates upon examining the same individuals due to
differences in skill, training the physical
condition [sic]. Thus the differences quoted are no
reflection on the design of the experiment or the
care taken in the work. The strength of the double
examinations comes through corroboration of caries
trends in Brantford over the years and not through
interchangeability of data.
Comment. Dr Grainger refers
to the differences between examiners in the
assessment of caries rates. This important matter
has already been considered. The aim of the
paragraph mentioned was to show that, as the rates
obtained by the examiners in the two Brantford
studies were different, data from the City Health
Department study could not be used to decrease "the
deficiency in the data of the National Health and
Welfare study, owing to its late commencement"
The admission, which is implicit in
Dr Grainger's remark, that "interchangeability of
data" was not permissible between the two studies in
Brantford confirms the point made. The degree of
reliance which can be placed on the "corroboration
of caries trends in Brantford over the years" must
be considered in the light of the widely divergent
results obtained in these studies, which were
discussed in the second paragraph of page three.
(46) Item 11: Significant
fluctuations in controls. The important point is
that for the controls the inter-year changes were
upward trends or mere fluctuations (even though in
some cases calculated to be beyond change) [sic],
whereas in Brantford the change took the form of a
highly significant continual downward trend.
Comment. It was pointed out
that in the control city of Stratford five out of
the six comparisons made between the permanent teeth
rates of successive years were said by Brown et
al. (1954b) to be significant changes (four of
them being at the three standard error level). In
the permanent teeth rates in the other control city,
Sarnia, there were four highly significant and one
significant change in the nine comparisons made.
Brown et al. (1953, 1954b) and Brown (1955)
said that the odds relating to the occurrence by
chance or sampling variation of a difference of the
magnitude of three standard errors (such as were
reported in eight of these changes) "are 369 to I
against". Therefore, when Dr Grainger terms these
unexplained changes "mere fluctuations" he is
rejecting that remark of Brown et al. and
denying the meaning of statistical significance.
Dr Grainger neglected to mention
that the "highly significant continual downward
trend" in the caries rates in Brantford occurred
only in children who were aged twelve to fourteen
years. In the two other age groups, in both the
deciduous and the permanent dentitions, there was an
upward trend in the caries rates in the fluoridated
city during the last year of the study, the rise
from 0.44 DMF to 0.69 DMF, in the youngest age
group, being said to be a highly significant rise
(Brown, 1955). Therefore this statement by Dr
Grainger, that there was a "continual downward
trend" in Brantford, is incorrect and is misleading.
(47) Item 12. Larger percentage
changes in control. There is no definite explanation
as to why rates increased in Stratford and also in
Sarnia over the ten years but this may be a
reflection of a general post-war increase in dental
caries which has been seen in other areas. However,
it is significant that in the various fluoridation
experiments e.g. in Brantford, Newburgh, Grand
Rapids, etc., the shift has always been
significantly downward in the fluoridated cities
whereas the control city rates have remained about
the same or in the case of Stratford, increased.
Comment. Dr Grainger's
statement, that the "rates increased in Stratford
and also in Sarnia over the ten years", is
inaccurate for, as he pointed out in his Item eight
(39), "Brown's observational period did not begin
until 1948, hence is less than 10 years." Sarnia was
first examined in March 1948 and Stratford in
October of that year (Brown, 1952). Therefore the
caries rates of the children in both those towns
were known for a period of about seven years, not
one of ten years. Furthermore, in the deciduous
teeth in Sarnia, the younger age group showed a
higher rate in 1955 than in 1948, but in the older
children the final rate was lower than the initial
one. In this city the DMF rate rose between 1948 and
1953 but between that year and 1955 there was a
decrease in this rate in each of the three age
groups.
The suggestion that there has been
"a general post-war increase in dental caries" is
not supported by these studies, for such a rise it
was not seen in any of the unfluoridated control
cities considered. In Muskegon and Oak Park there
was no definite trend. At the time when (as a result
of their being fluoridated) these cities ceased to
serve as controls, the rates for the children of
some ages were higher, and for other ages they were
lower, than during the initial examination. The
trend in Kingston cannot be investigated owing to
the method of presenting the data in the Newburgh
study. The contention that "the shift has always
been significantly downward in the fluoridated
cities" can be accepted only if the many
deficiencies pointed out in this monograph are
ignored and the figures from these trials accepted
at their face value.
(48) Item 13: Smaller percentage
decrease after longer fluoridation. As pointed out
by Sutton himself on page 168 (middle paragraph) the
fluoride protection for permanent teeth of the
children aged six to seven seemed to occur within
two or three years after fluoridation began.
Thereafter the yearly DMF rates were subject to
random fluctuation and the differences in percentage
decrease of 69 per cent and 51 per cent are most
likely a reflection of this inter-year variation.
Comment. No specific mention
was made of "the children aged six to seven" when
pointing out the reductions in the DMF rates which
were reported to have occurred in the early years of
the City Health Department study (Hutton et al.,
1951). Some implications of this reported early
decrease in caries rates were discussed on pages two
and thirty-eight.
When he makes the remark that "the
differences in percentage decrease of 69 per cent
and 51 per cent are most likely a reflection of this
inter-year variation", Dr Grainger shows that he
could not have noted that the increase in the DMF
rate in these children, in the fluoridated city,
from 0.44 in 1954 to 0.69 in 1955 (so that the
"percentage decrease" dropped from 69 per cent to 51
per cent), was shown by Brown (1955) to be,
statistically speaking, a highly significant (three
standard error level) rise in the rate in the test
city during the final year of the study. Therefore,
when he makes this suggestion, he is expressing an
opinion which contradicts the notation given by
Brown (1955), in his Table IV, which indicates that
the rise in rates which produced this percentage
"decrease" is statistically significant at the three
standard error level.
Dr Grainger may be right-but if he
is, Brown's (1955) indication of statistical
significance in this case is incorrect, and the
methods used in the National Health and Welfare
study for calculating statistical significance must
be questioned. In consequence, all statements made
in the study regarding significant changes in the
caries rates, both in the test city and its
controls, become doubtful.
It is of interest to note that the
"1955 Report" from this study was released in two
different publications. The first, a booklet, was
dated November 1955, and was "prepared by H. K.
Brown . . . with the assistance of H. R. McLaren...
G. H. Josie... and Barbara J. Stewart". The second
publication is a paper by Brown, McLaren, Josie and
Stewart published in 1956 in the Canadian Journal of
Public Health, no reference being made to the
previous publication. The ten tables and the two
figures in the body of these reports are the same
and the text of both is practically identical.
However, there is one important difference: that
part of the discussion dealing with the unexplained
rise (shown as significant) in the DMF rate of
children aged six to eight years in Brantford (and
also in the control city with the "optimum"
concentration of fluorides in its water), was
omitted from the later report (Brown et al.,
1956).
The questions should be asked: Why was this very
important small section of the original report
omitted when it was published in the Journal? Why,
in both these 1955 reports, was the smaller
percentage "decrease" between the initial and the
final caries rates of the youngest age group in
Brantford not published?
(49-50) Item 14 The quotation from
the Ontario Government Report is taken out of
context from a series of summary statements. The
previous statement was to the effect that in
Brantford there had been a significant decrease of
about 60 per cent in DMF rates. In the statement
following, as picked out by Sutton, it was stated
that "no change" occurred in Stratford and
Sarnia(49). It should be clear enough from the
context at the words "no similar downward change"
were inferred(50)
Comment. The charge that the
quotation mentioned was "taken out of context" is
meaningless unless it implies that the sense of the
original statement has been altered. So that the
reader can judge this matter, the whole of paragraph
three (the "previous statement" referred to by Dr
Grainger) and paragraph four are reproduced. The
quotation given on page 176, which was the one
criticized, is shown here in italic type to
distinguish it from its context.
The evidence produced by the
investigators of the Brantford City Health
Department and of the Department of National Health
and Welfare, independently show that since the
introduction of the fluorine in the water there has
been a significant decrease, amounting to
approximately 60 per cent in the number of decayed,
missing and filled teeth suffered by Brantford
school children.
At the same time, it has been
established that there has been no change in the
already low dental caries attack rates in Stratford
which has 1.2 ppm of natural fluoride in its
domestic water, or in the relatively high rates for
Sarnia which has had virtually no fluoride in
its water."
Dr Grainger suggests that the
writers of this report, when they used the
expression "no change", really meant "no similar
downward change". However, as they were members of
the Division of Medical Statistics and, presumably,
were trained in the very precise science, of
statistics, it is unlikely that they would use such
an inexact expression. Also, the term "downward
change" is a rather clumsy substitute for the word
"decrease" which was used in the previous paragraph
(para. 3 above). Both these points suggest that Dr
Grainger's interpretation is incorrect.
(51) Item 15 Possible weighting
effect. The critic is referring to a possible shift
in age distribution within the group, e.g. a
possible sampling shift within the six to eight-year
age group so that certain years had
disproportionately higher or lower numbers of
eight-year-olds and hence higher or lower average
caries scores. This is rather remote in that
selection methods used by Dr Brown were the same
each year; moreover very large shifts in age
distribution would be needed to produce the
significant differences to which P N. R. Sutton
refers.
Comment. After reading Dr
Grainger's remarks it may be supposed that it was
said that the inter-year significant changes in
caries rates, reported from the control cities, were
due to a "weighting" effect. However, if the
original paragraph is consulted, it will be found
that it is headed "Unexplained Significant changes
in controls." A "weighting" effect was mentioned as
a possible explanation for these unexplained
changes. This suggestion was made following the
failure of the authors (Brown, 1951, 1952; Brown
et al., 1953, 1954b, 1956) to advance even a
suggestion why these changes occurred. Dr Grainger's
comment leaves these significant changes in the
caries rates of the control cities as the authors of
this study left them - unexplained.
This reviewer has made it clear that
the statements regarding the "continuous" use of
water containing fluorides in concentrations of 1.3
or 1.6 ppm since 1917 in Stratford are not based on
data (comment 37); and that the misleading omission
of figures from the Ontario Department of Health
Report (1956) was made deliberately (comment 43).
However it has been seen that, although Dr Grainger
said (27) that his aim was "to help set the record
as straight as possible", most of his comments, if
they had been accepted at their face value, would
have had the reverse effect.
MR J. FERRIS FULLER
Apart from the reviews already
quoted above, the only published criticism known to
the author is that contained in the Book Reviews
section of the January 1960 issue of the New Zealand
Dental Journal. This was contributed by MR J. FERRIS
FULLER, a member of the Dental Research Committee of
the New Zealand Medical Research Council and a
member of the Fluoridation Committee of the
Department of Health, whose submissions to the New
Zealand Commission of Inquiry (1957) are mentioned
in over twenty paragraphs of its report.
Mr Fuller's review stated:
"Everyone is out of step except our
Albert," or so the author would have us conclude.
Altogether an extraordinary book; clever but
unfortunate; skilfully contrived and yet-stripped of
its finery-rather slender. It could be ignored if
the matter rested within the Sciences; but since by
the very nature of the subject it takes us into the
public forum, some of the errors must be stated..
Part I of Fluoridation: Errors and
Omissions in Experimental Trials (Chapter 19 here)
is a reprint of a paper by Sutton and Amies (see
footnote on page 136) that appeared recently in the
Medical Journal of Australia criticising the
Brantford-Sarnia-Stratford study in Canada(52). But
the authors have omitted to read the literature(53),
and their criticisms therefore are not based on the
known facts. This is a serious matter especially
when the comments come from two critics who exalt
themselves above fellow scientists of at least
equivalent status in other parts of the world. They
accuse the Canadian workers of failing to devise a
randomisation procedure that would eliminate
bias(54), of deliberately omitting vital information
in some of the tables(55), and finally of displaying
bias in the presentation of results(56). Their
comments are based on a report of the Ontario
Department of Health (1956) to the Ontario Minister
of Health, a report obviously written in simple
abbreviated terms for public consumption(57). Sutton
and Amies failed to read two official publications
readily available(58), namely, a 51-page booklet "A
suggested methodology for fluoridation surveys in
Canada" and the 35-page detailed report of the
Department of Health and Welfare, of November, 1955
These two booklets together show that great care was
taken to introduce a well-designed randomisation
procedure(59), that examiner variability was
eliminated as far as humanly possible by the
employment of one examiner only throughout the whole
period of the study(60), and that the information
alleged to have been omitted is in fact shown in
detail in the tables in the 1955 report(61),
together with the standard error for each of the
indices used. In short, the more important
criticisms that appear so damaging are in fact
without foundation. Thus, when the authors say that
"what must be eventually a statistical study does
not appear to have been designed as such" and "no
attempt at statistical evaluation has been
considered" their comments are absurd and, indeed,
irresponsible(62). The full official report on the
Brantford study was available in New Zealand,
incidentally, when the Commission of Inquiry held
its hearings(63), and three of its tables are
included in the published report of the Commission.
In Part 2 of the book Sutton
continues in the same vein. He complains that
misleading comments are made in some reports, yet
his own book contains many misleading statements.
For example, he claims that a proper evaluation of
examination errors at Grand Rapids has not been
carried out(64), and he doubts the accuracy of
caries attack rates in test and control areas
because X-ray examinations were incomplete or
absent(65). It is significant that he omits to refer
to a report by Hayes, McAuley, and Arnold published
in the U.S. Public Health Report in December, 1956,
which is a key reference in this subject(66). This
report met the specific point that "some observers
have suggested that X-rays are essential to
determine the efficacy of caries control measures"
and an investigation was undertaken "to determine
whether or not supplementing direct observation with
X-ray examinations would affect the conclusions
based on direct observation alone." The conclusion
was that supplementary X-ray examinations supported
the clinical findings and did not change the basic
observation that substantial decreases in dental
caries occurred during the test period. The very
standard errors that Sutton demands for a proper
statistical evaluation were available in this
report(67). He quotes a subsequent (1957) paper by
McAuley that suits his book and, in the light of his
criticisms and allegations, this makes the omission
of any reference to the 1956 report more
damaging(68). To borrow his own phrase, omissions of
this nature render his work "open to doubt." Sutton
criticises his overseas colleagues for their
inability to examine children in control towns prior
to fluoridation(69). With personal experience of a
study of this nature he would appreciate that where
on the one hand the interests of a large number of
people and their local bodies and institutions are
concerned as compared with only one or two examining
personnel on the other, it is almost impossible to
operate a plan to the exactitude dreamed of at the
statistician's desk. In any event, the criticism is
rather meaningless as far as the Grand Rapids
study(70) is concerned when we realise that the
baseline examination in the control city of Muskegon
showed that caries prevalence in that city is of the
same order as in Grand Rapids.
In attacking the Evanston-Oak Park
study, Sutton bemoans the lack of information about
the design of the study and phrases such as "It is
not clear...", "It is not understood...", (It) was
not stated..." give the lead to questions and
speculations that follow. But why not adopt the
simple expedient of writing to the workers concerned
and so finding out instead of speculating? This
attitude is typical of the book(71). And typical
also is the quibbling over details that do not
detract one iota from the part that fluoridation has
played in these areas in reducing dental decay(72).
"The total tooth surfaces considered... should be
58,325, not 58,352" says the author, and also... the
mean of these values for 1946... is 150.09, not
149.76"(73). Dear me, Dr Sutton, how dreadful.
And then we come to the
Newburgh-Kingston study. Prominence is given to the
different composition of the waters at Newburgh as
compared with the control city of Kingston(74), and
this is cited as the reason why the latter is
unacceptable as a control. But once again Sutton
omits any reference to a key report, that by Dean,
Arnold, and Elvove of August, 1942, listing caries
prevalence rates in communities where the variables
in the domestic water mentioned by Sutton varied to
a greater degree than between Newburgh and Kingston
without caries prevalence being markedly
affected(75).
The author complains of bias in the
manner in which some results are presented but, as
can be seen, he displays bias himself in the choice
of articles he quotes(76) and in his omission to
read others. It is not surprising, therefore, to see
him fall into the familiar pattern of the anti-fluoridationist.
Those who question fluoridation are given the
familiar title of "eminent authorities," a
distinction not afforded anyone else(77). It is
surprising, however, to see him serve his ends by
quoting Feltman's study on the use of fluoride
tablets. This study lacks the very control that one
would expect Sutton to consider essential(78).
As one would expect, there are no
bouquets for the New Zealand Commission of Inquiry,
one complaint being that "no mention was made of the
employment of a statistician to assist its members
in evaluating numerical data." Had the author
inquired, he would have been told that the Professor
of Biochemistry on the Commission was well versed in
biometrics, and that scientific witnesses quickly
discovered that tables were unacceptable unless they
contained complete details including standard
errors, so that he could evaluate data statistically
for himself and the Commission(79).
Finally, a warning to those reading
this book, lest they be misled by the polemics and
the array of figures. Please note that Sutton's
conclusions in part 2 (which forms the greater part
of the book) are confined to variations in the
prevalence of dental decay in control cities and not
to the cities where fluoride has been added(80).
What of the places where fluoridation has been
adopted? Sutton does not dispute the fact that the
prevalence of dental decay has been substantially
reduced in the fluoridation cities of Grand Rapids,
Newburgh, Brantford, and Evanston(81), nor does he
mention that these good results have been confirmed
by several independent studies in the U.S.A., and
also in Tasmania, Brazil, Japan, Germany, Sweden,
and at Hastings in New Zealand(82). The validity of
the results from Hastings, incidentally, has been
checked by the Applied Mathematics Laboratory of the
New Zealand Department of Scientific and Industrial
Research(83).
The anti-fluoridationists will
rejoice with fresh ammunition to replenish their
stocks; but it is unlikely that this work will serve
any useful purpose in scientific circles despite the
author's rather pretentious hopes. The performance
is almost as old as Time: "The mountains are in
labour, there will be born a ridiculous mouse," said
the ancient poet.
|
No comments:
Post a Comment