Errors
and Omissions in Experimental Trials - 2a
PART THREE
CRITICISMS AND COMMENTS
Soon after the publication of the
first edition of this monograph in September 1959,
the author was informed by Dr K. T. Adamson,
President of the Australian Dental Association, that
copies had been sent to "all of the men who are in
charge of the experiments" asking them for comments
(personal communication). As a result the reviews
published in the February 1960 issue of the
Australian Dental Journal were contributed by Dr
Donald Galagan, Dr J. R. Blayney and Dr I. N. Hill,
and by Dr R. M. Grainger. The review in the New
Zealand Dental Journal of January 1960 was written
by Mr J. Ferris Fuller.
The aim of this monograph is to
attempt to clarify some aspects of five crucial
trials of artificial fluoridation, those conducted
in Grand Rapids, Evanston, and Newburgh, U.S.A., and
the two trials held in Brantford, Canada. Therefore,
in this part, all these critical reviews will be
quoted in full; and comments will be made on the
points raised and indicated by the figures in
brackets.
In order to allow the reader to
appraise the criticisms more easily it will be
necessary to refer to the statements made in the
text, therefore page references are indicated for
Parts One and Two of this volume.
DR DONALD GALAGAN
DR DONALD GALAGAN
The first review of this book in the
Australian Dental Journal, by Dr Donald Galagan,
Assistant Chief, Division of Dental Public Health,
Public Health Service, Washington 25, D.C., was as
follows:
My comments will be limited to the
general qualities of Dr Sutton's treatise and the
conclusions he has reached. Individuals associated
with the several fluoridation projects which he has
purported to "analyse" will have provided their
specific reactions to his "analysis" of their
findings.
Although it is nothing new to see an
accredited scientist mix fact and fancy, near truth
with truth, and emotion with reason it is always
shocking to realise that an intelligent individual
in a responsible position can so baldly misinterpret
scientific data. Actually, it would be extremely
difficult for an objective scientist who knows
anything at all about the data characterizing the
relationship between dental caries and exposure to
fluorides to reach the conclusion that the
effectiveness of controlled water fluoridation in
reducing dental caries has not been proved. This
conclusion could only be reached by an armchair
statistician who has chosen to ignore or does not
know the great mass of information on the subject.
Thus, first and foremost among the
fundamental errors(1) which Dr Sutton makes is
expressed in the statement in the second paragraph
of the monograph which says that "proposals to
fluoridate domestic water are almost entirely based
on the results of the Brantford, Grand Rapids,
Newburgh, and Evanston projects". Using this
premise, the balance of the document is devoted to
efforts to describe "errors and omissions" in these
four projects. These errors are supposed to negate
the whole fluoride-caries hypotheses(2), or at least
to throw serious doubt on the fluoridation of water
as a caries preventive.
Actually, the scientific basis on
which this public health measure rests was
established solidly before any of the above
mentioned projects were started. The preventive
effect of long term exposure to water-borne fluoride
on caries experience was observed in literally
thousands of children residing in many different
communities where the water consumed had picked up
the element as it coursed over or through the
earth's crust. The long series of investigations
documenting the relationship are considered to be
classic examples of good epidemiological method(3) -
so much so that they are used as case studies in
teaching the science of epidemiology in our schools
of public health.
The fact is that the projects at
Brantford, Grand Rapids, Newburgh and Evanston were
designed primarily to evaluate the technical,
financial and administrative problems associated
with the controlled addition of fluorides to a
municipal water supply, and, secondarily, to
demonstrate the effectiveness of the procedure(4) to
the profession and the public. To be sure, it was
necessary to show that the procedure would reduce
the caries attack rate in resident children, but as
soon as the trend toward reduction was observed and
corroborated, the "experimental" portions of the
projects were completed for all practical purposes.
The principal point, however, is that these projects
emphatically are not the sole basis on which the
widespread use (in the United States) of this
procedure rests(5).
In short, it is preposterous to
attempt to conclude that the basis for community
water fluoridation is faulty because of some real or
imaginary defects in the planning, execution and
evaluation of data from the four community projects.
The contents of the monograph, therefore, represent
no more than an exercise in semantic and scientific
dilettantism designed to serve some other purpose.
There are a good many other
specific, but less important errors in judgment
which the author has made, such as his suggestion
that the variability of examiners in diagnosing
dental caries has been overlooked(6). The truth is
that this variability is well known, and is
discussed at length by specialists in the fields of
epidemiology and caries diagnosis. Because of this
"human error" calibration of examiners is practised
as a matter of course. Calibration reduces the
variability among examiners, but even if it did not,
the difference between the caries experience of
children exposed and not exposed to fluoride is so
great that even Dr Sutton could recognise it(7).
There are several other examples of
errors of judgment in the arguments contained in the
monograph, indicating the author's serious lack of
understanding of the principles of statistics and
epidemiology. For instance, the rather amateurish
interpretation of adequate community "controls" for
evaluating the effect of fluoride-bearing water on
caries indicates that the author does not really
know the manner in which caries occurs in a
population(8).
The author's use of innuendo to make
a point is contained in his reference to some very
questionable work of Feltman which indicated that
the eruption of deciduous teeth might occur later in
children exposed to fluorides. The author implies
that it is likely that retarded tooth eruption in
the children residing in the fluoride communities
reduces their exposure to caries attack, and thus
their caries rate naturally is lower(9). Had Dr
Sutton been familiar with the literature, he would
have known that this was one of the first
possibilities thought responsible for the low caries
rate in children exposed to fluoride-bearing water
(10). He would have known about the rather
exhaustive report of Short on the relation between
fluoride in domestic waters and tooth eruption which
showed that fluoride in concentrations around the
optimum used for caries control does not influence
the eruption pattern of permanent teeth (11).
However, one suspects that further
analysis of the details contained in the monograph
will not yield much of value. From reading the
document and from hearing him present part of it as
a paper last February at your Adelaide Congress, I
can only conclude that Dr Sutton has an intense and
emotional drive to oppose fluoridation. Why he feels
this way is not clear, but it seems likely to come
from some motive other than a sincere concern for
the statistical or scientific validity of the
concept(12).
Commentary on the Review by Dr Donald Galagan
Commentary on the Review by Dr Donald Galagan
(1) Dr Galagan makes the charge that
the "first and foremost among the fundamental
errors" was made in the second paragraph of the
monograph (p. 136). This paragraph stated: "Apart
from these considerations, an examination reveals
that there are aspects that call for a very careful
appraisal of the figures presented in the reports of
the experimental trials which have been conducted in
Brantford, Canada, and in Grand Rapids, Newburgh and
Evanston, U.S.A., and upon the results of which
proposals to fluoridate domestic water are almost
entirely based." Such proposals are based on two
different sets of results- those reported from areas
where fluorides occur naturally in the water
supplies, and those from trials of the mechanical
addition of fluorides to waters in which the
fluoride content is very low or absent. The former
reports are not considered in this investigation,
but Dr Galagan says that they were gathered in such
a way that they "are considered to be classic
examples of good epidemiological method". In any
case, they are of practical importance only if it is
known that the results reported will be obtained
when commercially available fluorides are
mechanically added to water supplies.
This knowledge can be obtained only
by the use of experimental trials the results of
which must then be considered to be of outstanding
importance-unless it is accepted that it can be
established, on theoretical grounds, that the
results of artificial fluoridation will be identical
with those seen in areas where fluorides are found
naturally. If the latter case is accepted, the early
artificial schemes which are considered here were
held merely, as Dr Galagan suggests, "to demonstrate
the effectiveness of the procedure" and cannot be
considered to be true experimental trials.
The question to be answered is this:
Were these trials mere demonstrations or were they
set up as genuine experimental studies? If they were
only demonstrations Dr Galagan's charge is
justified, but if these trials were conducted to
determine the outcome of the process of mechanical
fluoridation, then his accusation is without
foundation and must be discredited.
In 1951 a report was issued by the
ad hoc Committee on Fluoridation of Water Supplies
of the National (U.S.A.) Academy of Sciences,
National Research Council (Maxcy, Appleton, Bibby,
Dean, Harvey, Heyroth, Johnson, Whittaker and
Wolman, 1952). One of the members of this Committee
was Dr H. Trendley Dean, who was closely associated
with much of the earlier work concerning fluorides
and was, at that time, the Director of the National
Institute of Dental Research. The report included
this statement: "In 1945, studies were begun to
ascertain whether the adjustment of the fluoride
content of a public water supply to the optimal
level with commercially available fluorides would
confer the same caries-inhibitory effects as do
waters which carry the same concentrations of
fluoride naturally."
This statement makes it clear that
Dr Galagan's contention, that these trials were only
demonstrations, is not correct. However, because of
the importance of this matter and in case it is
suggested that that Committee was misinformed
concerning the intention of these trials, a
quotation will be given from each of the five
studies considered. Dr Galagan calls the process
used in these test cities "the controlled addition
of fluorides to a municipal water supply", but the
original term used by Dean et al. (1950) and
by Brown (1951, 1952), "artificially fluoridated"
drinking water, is preferable as it is free from
ambiguity.
(a) Brantford, City Health
Department Study, Hutton et al., (1951): "It
was recognized that fluorine in the public water
supply was not a proven method for the prevention of
dental caries, and that it might take ten years to
prove or disprove its preventive value."
(b) Brantford, National Health and
Welfare Study, Brown et al., (1954b): "The
Brantford Fluoridation Caries Study was undertaken
with a view to finding out whether or not the
raising of the fluoride content of a previously
fluoride-free water supply to part per million, by
the mechanical addition of sodium fluoride, would
reduce the incidence of dental caries to that which
obtains where water supplies derive about 1 part per
million of fluoride from deposits in the earth"
(c) Grand Rapids, Dean et al.
(1950): "In 1945, three studies to determine the
caries prophylactic value of artificially
fluoridated drinking water were started in the
United States and Canada."
(d) Newburgh, Ast et al.
(1950): "In 1943 it was proposed to determine
whether we can translate the conclusions derived
from the epidemiological studies in fluoride areas
to a practical application in fluoride-free areas
where the communal water supplies may lend
themselves to treatment."
(e) Evanston, Blayney and Tucker
(1948): "After further deliberation of the project,
both professional groups recommended to the
Commissioner of Health that a carefully controlled
study be developed to determine whether or not the
addition of fluorine in minute quantities to the
communal water supply would reduce the incidence of
dental caries in Evanston and Skokie children."
Blayney and Tucker (1948) also said: "It was
carefully explained to these Evanston citizens that
nothing could be promised regarding the ultimate
value in the control of tooth decay; that if such a
program was to be undertaken it must be in the
nature of an exhaustive study; and that it would be
several years before data would be available which
would even indicate the trend which we might
expect."
These statements, by the authors of
all of the five studies considered, establish beyond
question that in every case the studies were not
designed "to demonstrate the effectiveness of the
procedure" but to determine whether or not
artificial fluoridation would be efficacious.
Therefore Dr Galagan's opinion is incorrect, and the
statement made in Part One, which he termed the
"first and foremost among the fundamental errors",
is a correct description of the situation.
(2) His suggestion that an attempt
has been made to "negate the whole ''
fluoride-caries hypotheses" is without foundation.
The only reference to this matter is contained in
the quotation of a statement. made in 1949, by the
American Water Works Association regarding the
experimental verification of "the fluoride-dental
caries hypothesis" - that is, to the "fluoridation
hypothesis". Statements made in the preface to the
first edition and on the first page of Part One and
the first and last pages of Part Two show that
consideration has been given only to five
experimental trials of artificial fluoridation
produced by mechanical means. The data from
epidemiological studies in "naturally fluoridated"
areas, on which the fluorine-dental caries
hypothesis is based, have not been considered.
(5) Dr Galagan states that "The
principal point, however, is that these projects
emphatically are not the sole basis on which the
widespread use (in the United States) of this
procedure rests." It can be seen, by reading the
"free" quotation from paragraph two, page 136, given
by this reviewer (1) and pages 140, 189, 190 and
196, that the word "sole" was not used nor implied.
No comment can be made on the other
"fundamental errors" which Dr Galagan says (1) are
present-for he has neglected to state their nature.
Instead he continues:
(6) There are a good many other
specific, but less important errors in judgment
which the author has made, such as his suggestion
that the variability of examiners in diagnosing
dental caries has been overlooked.
Comment. This suggestion was
not made. As Dr Galagan points out in his next
sentence (7), "this variability is well known".
Therefore it is most unlikely that such a matter
would be "overlooked" in studies employing
statisticians. Indeed, attention was drawn, on page
180, to the fact that the importance of examiner
variability was recognized by Ast et al. in
1950. However, in conducting the clinical
examinations, in all the studies considered, this
matter was ignored. Therefore, when speaking of this
phenomenon, terms such as "not assessed" and "not
estimated" were used.
(7) The truth is that this
variability is well known, and is discussed at
length by specialists in the fields of epidemiology
and caries diagnosis. Because of this "human error"
calibration of examiners is practised as a matter of
course. Calibration reduces the variability among
examiners, but even if it did not, the difference
between the caries experience of children exposed
and not exposed to fluoride is so great that even Dr
Sutton could recognise it.
Comment. As Dr Galagan says,
this (examiner) variability is well known. It is
precisely this fact that makes it so surprising that
this factor was not assessed in these studies. It
will be recalled that reference was made, on page
nine, to two papers which investigated the matter of
examiner variability in caries diagnosis.
The claim is made by the reviewer
that "Calibration reduces the variability among
examiners", but he does not suggest that this
process eliminates between-examiner
variability-therefore it should have been taken into
account. Of course, the use of the method of the
"calibration" of the subjective judgment of several
examiners with the subjective judgment of another
is, to say the least, a poor substitute for a
standard rigorous statistical procedure.
(8) There are several other examples
of errors of judgment in the arguments contained in
the monograph, indicating the author's serious lack
of understanding of the principles of statistics and
epidemiology. For instance, the rather amateurish
interpretation of adequate community "controls" for
evaluating the effect of fluoride-bearing water on
caries indicates that the author does not really
know the manner in which caries occurs in a
population.
Comment. The question may be
asked: Who does?
(9) The author's use of innuendo to
make a point is contained in his reference to some
very questionable work of Feldman (p. 192) which
indicated that the eruption of deciduous teeth might
occur later in children exposed to fluorides. The
author implies that it is likely that retarded tooth
eruption in the children residing in the fluoride
communities reduces their exposure to caries attack,
and thus their caries rate naturally is lower.
Comment. On page 192,
reference was not made to the work; of Feldman but
to that of Dr Reuben Feltman who was an associate of
Doctors D. E. Gardner and F. A. Smith, whose
publications on fluorides are well known, and of
Doctors H. C. Hodge and D. E. Overton, who were
closely associated with the Newburgh trial (Gardner,
Smith, Hodge, Overton and Feltman, 1952). Dr
Feltman's earlier (1951) work with fluoride tablets
was referred to by the New Zealand Commission of
Inquiry (1957) as "promising". His 1956 paper, which
was quoted (p.192), was a brief "progress report"
only. Therefore the results mentioned in it were
treated with reserve, the statement being made that
"Of course, if fluoridation results in the eruption
rate of teeth being retarded ....".
(10) Had Dr Sutton been familiar
with the literature, he would have known that this
was one of the first possibilities thought
responsible for the low caries rate in children
exposed to fluoride-bearing water.
Comment. Dr Galagan has
avoided the main point which was discussed in this
paragraph. His insistence that one of the first
possibilities considered was that the ingestion of
fluoride-bearing water may retard tooth eruption,
makes it even more strange that in only the
Newburgh-Kingston study was mention made that this
important matter had been investigated. Even in that
trial, the only study of tooth eruption rate
published was conducted after four years of
fluoridation (Ast et al., 1951) in children
who were six to twelve years of age, so that none of
the subjects studied had been ingesting artificially
fluoridated water throughout their lives.
(11) He would have known about the
rather exhaustive report of Short on the relation
between fluoride in domestic waters and tooth
eruption which showed that fluoride in
concentrations around the optimum used for caries
control does not influence the eruption pattern of
permanent teeth.
Comment. It is highly
probable that Dr Galagan is referring to the paper
by E. M. Short (1944) which is well known to those
interested in fluorides, for this is the only paper
concerning fluorides and tooth eruption listed under
that name in the Index to Dental Literature in the
English Language (published by the American Dental
Association), the Quarterly Cumulative Index Medicus
(to December 1956), or the issues of the Current
List of Medical Literature which cover the
subsequent period.
This "rather exhaustive report of
Short" does not show whether or not the ingestion of
fluorides at the "optimum" level has any influence
on the eruption pattern of permanent teeth. This
report was made on "selected 12 14 year old white
school children" (Short, 1944) in whom almost all
the permanent teeth had erupted. The data deal only
with the total number of erupted permanent teeth
and, despite Dr Galagan's remark, do not give any
information regarding their eruption pattern-a
factor which could be of considerable importance in
the development of the DMF rate. Short's Tables II
and III show that, in all except three of the 4,453
children examined, at least twelve of the permanent
teeth had erupted, the minimum number of erupted
permanent teeth, for the remaining three children,
being ten. His Table I (which excluded third molars)
shows that, out of the possible twenty-eight teeth,
the mean number of erupted permanent teeth per child
in the various cities was between 25.22 and 26.81.
In fact, in fifty-five per cent of the children all
of the twenty-eight teeth had erupted.
Therefore, this study of Short
(1944) gives no information regarding the ages at
which the first ten permanent teeth erupted in these
children, certainly none regarding the first
permanent molars which, presumably, even in the
youngest of these children, erupted about five or
six years prior to the study. These molars are of
outstanding importance in regard to the DMF rate,
particularly in young children. Ast et al.
(1956) said: "The first permanent molars are
frequently used as an index of caries experience
among children because this tooth accounts for the
major incidence of caries in this group."
It should be noted that neither Dr
Galagan nor (as will be seen in their review of this
book) Doctors Blayney and Hill, authors of the
Evanston study, have commented on the suggestion of
a decline in the eruption rate of first permanent
molars in Evanston, between 1946 and 1951, which
followed the introduction of fluoridation and which
is depicted in Figure 4 (p. 139). Neither have they
explained why, after 1951, whilst continuing to
publish this type of data for the older children,
they ceased publishing it for the younger
(12) However, one suspects that
further analysis of the details contained in the
monograph will not yield much of value. From reading
the document and from hearing him present part of it
as a paper last February at your Adelaide Congress,
I can only conclude that Dr Sutton has an intense
and emotional drive to oppose fluoridation. Why he
feels this way is not clear, but it seems likely to
come from some motive other than a sincere concern
for the statistical or scientific validity of the
concept.
Comment. Dr Galagan's
questioning of the motive behind this study should
be considered in relation to statements which he
made during his lecture to the Adelaide Congress (Galagan,
1959). He called those who questioned fluoridation
"the opposition" and said that this group "seems to
be composed of four distinct kinds of people." These
he termed: "the hatemonger, the pseudo-health
believer, and the person who opposes fluoridation
for personal notoriety" and "the fourth, or rugged
individualist, group". As these are Dr Galagan's
views, it is not surprising that he doubts the
sincerity of this attempt to investigate "the
statistical or scientific validity" of these
fluoridation findings.
DR J. R. BLAYNEY and DR I. N. HILL
DR J. R. BLAYNEY and DR I. N. HILL
The second review of this book in
the Australian Dental Journal, by DR J. R. BLAYNEY,
Director, Dental Caries Study, Evanston, and DR I.
N. HILL, Zoller Memorial Dental Clinic, University
of Chicago, was as follows:
Dr Sutton has much to say regarding
the lack of comparability(13) of the study and the
control areas and the manner of selection of the
children in each area to be examined. Oak Park is
the suburb immediately to the west of Chicago and
Evanston lies immediately to the north. Each
community draws its water supply from Lake Michigan.
Standard analyses for composition of tlw water are
frequently run. A spectro-chemical analysis for 26
trace elements has been run on Lake Michigan water
for a comparison of a similar analysis of water
obtained from fluoride areas (unpublished data). Oak
Park and Evanston receive their food from the same
wholesale markets, each is chiefly residential and
free from heavy industry. Each is composed of the
same socio-economic level, as borne out by the
United States census of population for 1950. In
Evanston, 47,395 persons 21 years or over were
native born. In Oak Park the figure was 42,454. In
Evanston there were 6,049 foreign born persons 21
years or older while in Oak Park the figure was
5,081. In Evanston 41.2 per cent of the occupied
dwelling units were owner occupied. In Oak Park 50.8
per cent were occupied by the owner. The median
value of a one dwelling unit structure in Evenston
was 19,499 dollars and in Oak Park the value was
16,259 dollars. The median value of gross monthly
rentals in renter occupied dwelling units in
Evanston was 72.53 dollars while in Oak Park the
median value was 66.86 dollars. Both areas have
comparable, climatic conditions and both are
subjected to the same radio and television
commercials regarding oral hygiene and dentifrices.
Finally, the majority of the dental practitioners in
the study and control areas are graduates of one of
the three Chicago dental colleges.
For the baseline examination in
Evanston we were committed to examine all
school children within the selected age range
regardless of the length of time they had resided in
Evanston. Although an effort was made to include
them, the Oak Park Parochial Schools did not find it
convenient for us to examine their pupils(14). When
we compared the caries prevalence rates of the two
towns we found a difference. It is indeed fortunate
that our records showed the school that each child
attended. In this manner we could first eliminate
the Parochial School group and then the Negro group
from the total Evanston data. It is well known that
coloured people have less dental caries than whites
living in the same population centres(15). The
caries rates for the Evanston and Oak Park Public
School white children compared favourably(16). All
of this clearly indicates how hidden variables may
exist in areas which otherwise appear to be
comparable, and how important it is to be certain
that comparisons are made between like groups(17).
Dr Sutton expressed astonishment
that in 1955 the six and seven-year-old Evanston
children had a lower caries prevalence rate than the
Aurora children of like ages. We, likewise, did not
anticipate this. However, the same critical
evaluation both clinical and roentgenological, was
made of every case. This difference was due to
something other than fluoride. Possibly the presence
of the dental team in the school, year after year,
has stimulated the classroom teachers and the school
nurses to place more emphasis on the teaching of
oral health. Some unknown hidden variation not
related to fluorides must account for the
difference(18).
Dr Sutton was concerned that the
control group was not examined annually. Neither we
nor our advisers could see a reason to require an
examination of the control group other than at the
beginning and near the close of the study. This
provides the baseline from which to measure the
trend of the dental caries rate during the time
interval (1947-1956). Should the rate in the last
examination (1956) deviate materially from that of
the initial baseline period (1947) that figure could
be used as a correction factor on the Evanston
findings. In fact, if we had not desired to measure
the yearly decrement in the rate of dental caries
under fluoridation and evaluate other factors only
two examinations in Evanston would have been
necessary, the first in 1946 (before fluoridation),
the second and final in 1961(19).
Much has been made of the variations
reported in the number of children examined. The
baseline examination of 4,375 Evanston children and
of 2,493 Oak Park children are correct. However, the
data from those children who had not used Lake
Michigan water all of their lives had to be
excluded. It was also observed that some children
below the age of 67 months and above the maximum of
174 months had been examined. Therefore these out of
range children were not considered in the final
determination of the caries rates. This explains the
discrepancy between the Evanston 4,375 and 3,692 and
the Oak Park 2,493 - 2,051 figures(20). It should be
noted that when dental caries experience rates were
compared, the same number of examinations, that is
1,991 for the six to eight-year-old children and
1,701 for the 12 to 14-year-old children, were used
throughout the reports for the combined Evanston
school groups (Public White, Parochial and Public
Negro)(21). Dr Sutton on p. 167 of his report (fig.
5, Statement C) calls attention to Evanston Dental
Caries Study Report Number XVII. Here he points out
that only 1,754 six to eight-year-old children and
1,556 12 to 14-yearold children were listed. This
particular report was primarily concerned with
differences in sex and race, as they influenced
caries, rather than the effect of fluorides.
Therefore a comparison was made of the dental caries
experience rates of white girls to white boys; Negro
girls to Negro boys; white girls to Negro girls;
white boys to Negro boys; and white children as a
group to Negro children as a group. The children
were classified into male white, female white, male
Negro and female Negro for comparison(22). As Dr
Sutton calls attention to the difference in the
number of examinations made in this report when
compared with other reports, we wish to point out,
in explanation, that it was necessary to not only
insure that children of correct age be included but
also it was necessary that every examination be
clearly classified according to race and sex There
were 236 six to eight-year-old children and 245 12
to 14-year-old children excluded from this report
because they did not fulfil the requirements for
this comparison. These children were included in
other reports as no distinction except age was
made(23). It is also pointed out that this report,
Evanston No. XVII as noted above, was primarily
concerned with comparison of the caries rates of the
coloured and white children and not with the effect
of fluoridation on dental caries rates. Therefore in
this light, this report, No. XVII, should not have
been listed under the general heading of
fluoridation as Dr Sutton has it listed in his
critique(24).
It is true that a discrepancy in
figures published in our paper XVI, Table I and in
paper XVIII, Table I are at variance. This is due to
the operator of the tabulating machine providing the
wrong figures for the number of seven and
eight-year-old children examined. This error was
discovered after manuscript XVI was in press and
therefore the corrections could only be made
manually in the reprints supplied to readers who
requested them(25).
Commentary on the Review by Dr
J. R. Blaynex and Dr L N. Hill
(13) Reference to Part Two will show
that no suggestion was made in it that there was a
"lack of comparability" between Evanston and Oak
Park; it was merely pointed out that the manner in
which Oak Park resembled Evanston was not stated.
Therefore this detailed exposition
of the similarity of the two cities is welcome,
although it is unfortunate that, when speaking of
rental and dwelling values, the "mean" values were
not given as well as the "median" ones. It will be
realized that the housing picture may be very
different in two towns and yet the "median" (that
is, the middle) values of the rentals and dwellings
can be the same. In view of the data shown here,
regarding housing and the training of the dentists,
it is surprising that the members of the United
Kingdom Mission (1953) should have singled out
Evanston for comment, remarking on its high economic
level and its "outstandingly good" dental care (p.
149).
These data also show how reasonable
was the assumption of Dr Blayney before assessing
the caries rates in Oak Park-that that city was "the
ideal control community" for Evanston (Blayney and
Tucker, 1948;). They also emphasize how strange it
is that such gross differences should be found
between the initial caries rates of the children
aged six to eight years in the two cities, and
reported-after a delay of ten years-by these workers
(p. 153; Fig. 3, p. 154). The fact that such gross
differences can be found in the caries rates
prevalent in two cities which were so similar that
one was termed "the ideal control community" for the
other (Blayney and Tucker, 1948), confirms the
necessity for pre-fluoridation examinations in both
test and control cities. Unfortunately, as pointed
out in Part Two this was not done in any of these
studies.
(14) For the baseline examination in
Evanston we were committed to examine all school
children within the selected age range, regardless
of the length of time they had resided in Evanston.
Although an effort was made to include them the Oak
Park Parochial Schools did not ford it convenient
for us to examine their pupils.
Comment. The latter remark is
welcome for it explains the absence of data from the
parochial schools in Oak Park.
(15) When we compared the caries
prevalence rates of the two towns we found a
difference. It is indeed fortunate that our records
showed the school that each child attended. In this
manner we could first eliminate the Parochial School
group and then the Negro group from the total
Evanston data. It is well known that coloured people
have less dental caries than whites living in the
same population centres.
Comment. Despite their
statement that "It is well knowp that coloured
people have less dental caries than whites living in
the same population centres", when conducting the
initial examination in Evanston the authors combined
the data of the Negro children with those of the
white children. It is now clear, as deduced on page
eighteen, that the racial and school groups were
taken into account only after it was found that
there was "a lower caries rate for school children
of the control area" (Hill et al., 1951).
Thereafter, when comparing the test and control
cities, the data of both the Negro and the parochial
school children were excluded from the Evanston
data. No reason has been given for this exclusion of
the data of parochial school children in
Evanston-who had a high caries rate (Hill et al.)
- from the data of the main body of white children
in that city. This could not be attributed to the
fording of a similarly high caries rate in the
parochial school children in the control city of Oak
Park for they were not examined and, therefore,
their caries rates were unknown (see 14).
Neither has an explanation been
offered for the extraordinary reversal of this
policy (the exclusion of the data of Negro and
parochial school children in Evanston) when
compiling the XVIII Report (Hill et al.,
1958). This report published, for the first time,
the initial caries rates for the permanent teeth of
the children aged six to eight years which were
obtained, ten years earlier, in the control city of
Oak Park.
The rates for the deciduous teeth,
which were obtained at that time, still have not
been published. This report provided the first
opportunity to compare the initial caries rates of
the younger children in the test city and its
control (p. 153).
(16) The caries rates for the
Evanston and Oak Park Public School white children
compared favourably.
Comment. This statement is
interesting-for the caries rates for children aged
six to eight years in each of the three school
groups have not been published (p.151). It will be
recalled that the mean caries rates for the six,
seven, and eight-year-old children in Evanston in
1946 were very much higher than the mean rates for
children of those ages obtained during the initial
examination in Oak Park. For the children aged six
years the rate in Evanston was 46.85, but it was
only 26.89 in Oak Park (Hill et al., 1958).
Only 0.1 per cent of the Oak Park children were
Negro (Hill et al., 1951), but exclusion of
the data of Negro children (who have a relatively
low caries rate, see 15) from the Evanston data
would increase the rate of the remaining (white
public and parochial school) children so that in the
six-year-old children, it would be higher than
46.85. Therefore the difference between this rate
and the Oak Park rate of 26.89, for children of that
age, would be increased.
Hill et al. did not say how
many of the younger age group of children attended
each type of school, but only twenty-two per cent of
the twelve to fourteen-year-old children, shown in
their 1957a report as examined in Evanston in 1946,
attended parochial schools. Therefore, the
proportion of children aged six, seven, and eight
years who were attending parochial schools in
Evanston in 1946, and their caries rate, must have
been very high to permit Doctors Blayney and Hill to
state that "The caries rates for the Evanston and
Oak Park Public School white children compared
favourably." Of course speculation is no substitute
for data-and this still has not been published
(17) All of this clearly indicates
how hidden variables may exist in areas which
otherwise appear to be comparable, and how important
it is to be certain that comparisons are made
between like groups.
Comment. The latter phrase is
a reiteration of remarks made in the 1957a report
from this study, that it is necessary "to make
comparisons of like groups." Why then, having
realized this necessity, did Hill et al.
ignore it in their 1958 report (p. 152)? In this
report the data shown, for the year 1946, combined
not only that of the white children attending both
public and parochial schools, but the data of the
Negro children as well. The resultant rate was then
compared with that of children in Oak Park
comprising, almost entirely, white children
attending public schools. By ignoring the opinion
they expressed in the previous year (1957a)-which
they now reiterate-and comparing "unlike" groups of
children, a more favourable degree of comparability
was obtained between the initial caries rates of
children in the test and the control cities.
(18) Dr Sutton expressed
astonishment that in 1955 the six and seven-year-old
Evanston children had a lower caries prevalence rate
than the Aurora children of like ages. We, likewise,
did not anticipate this. However, the same critical
evaluation both clinical and roentgenological, was
made of every case. This difference was due to
something other than fluoride. Possibly the presence
of the dental team in the school, year after year,
has stimulated the classroom teachers and the school
nurses to place more emphasis on the teaching of
oral health. Some unknown hidden variation not
related to fluorides must account for the
difference.
Comment. It is pleasing that
Doctors Blayney and Hill should support the view
expressed on page 187 (para. 4) that regular dental
examinations may stimulate interest in the teeth and
thus lead to improved oral health. In advancing the
suggestion that "something other than fluoride" can
affect the caries rates, they recognize the great
importance which factors other than the fluoride
concentration of the water supply may have on the
caries rates. This extremely important matter was
practically ignored by the authors of all these
studies when preparing their reports.
(19) Dr Sutton was concerned that
the control group was not examined annually. Neither
we nor our advisers could see a reason to require an
examination of the control group other than at the
beginning and near the close of the study. This
provides the baseline from which to measure the
trend of the dental caries rate during the time
interval (1947 1956). Should the rate in the last
examination (1956) deviate materially from that of
the initial baseline period (1947) that figure could
be used as a correction factor in the Evanston
findings. In fact, if we had not desired to measure
the yearly decrement in the rate of dental caries
under fluoridation and evaluate other factors, only
two examinations in Evanston would have been
necessary, the first in 1946 (before fluoridation),
the second and final in 1961.
Comment. This statement makes
two things clear. The first is that, at the
commencement of the study, neither the workers nor
their advisers could have considered the
possibility, which they now acknowledge(18), that
"the presence of the dental team in the school, year
after year" might have had a stimulating effect "on
the teaching of oral health." It is obvious that, if
this effect is possible, not only the test town but
also its control should have been examined "year
after year". The other point which is indicated by
this statement(19) of Doctors Blayney and Hill is
that despite their remark in 1950, the importance of
random variation was not and, seemingly still is not
recognized.
(20) Much has been made of the
variations reported in the number of children
examined. The baseline examination of 4,375 Evanston
children and of 2,493 Oak Park children are correct.
However, the data from those children who had not
used Lake Michigan water all of their lives had to
be excluded. It was also observed that some children
below the age of 67 months and above the maximum of
174 months had been examined. Therefore these out of
range children were not considered in the final
determination of the caries rates. This explains the
discrepancy between the Evanston 4,375 and 3,692 and
the Oak Park 2,493 - 2,051 figures.
Comment. This explanation of
the difference between these sample sizes in
Evanston and Oak Park is welcome. It might have been
deduced if the decision to exclude "the data from
those children who had not used Lake Michigan water
all of their lives" had been announced in one of the
five reports giving caries rates, issued prior to
1955 (p. 163).
In the XIX Report (Hill et al.,
1959), the sample sizes shown for the two age groups
in Oak Park in 1947 (1,022 and 1,032) are almost the
same as those shown (1,020 and 1,031) in statement
"E" of Figure 5. Comparison with the statements for
Evanston cannot be made for this (XIX) report
considered only "public school white children".
(21) It should be noted that when
dental caries experience rates were compared, the
same number of examinations, that is 1,991 for the
six to eight-year-old children and 1,701 for the 12
to 14-year-old children were used throughout the
reports for the combined Evanston school groups
(Public White Parochial and Public Negro).
Comment. It is surprising
that the suggestion was made that this statement
should be noted-for it is not correct. The number of
twelve, thirteen and fourteen-year-old children
examined in Evanston in 1946 was given in Tables
III, V, VI, VII, VIII, IX and X of the XV Report
(Hill et al., 1957a) as 418, 688 and 595, a
total of 1,701. However, in Tables XI and XII of the
same paper different sample sizes for these ages
were shown: 414, 692 and 617, a total of 1,723. The
same discrepancies were noted between different
tables in the XI Report (Hill et al., 1955)
Therefore the figure 1,701 was not "used throughout
the reports for the combined Evanston school
groups".
(22) Dr Sutton on p. 167 of his
report (fig. 5, Statement C) calls attention to
Evanston Dental Caries Study Report Number XVII.
Here he points out that only 1,754 six to eight-year
old children and 1,556 12 to 14-yearold children
were listed. This particular report was primarily
concerned with differences in sex and race, as they
influenced caries, rather than the effect of
fluorides. Therefore a comparison was made of the
dental caries experience rates of white girls to
white boys; Negro girls to Negro boys; white girls
to Negro girls; white boys to Negro boys; and white
children as a group to Negro children as a group.
The children were classified into male white, female
white, male Negro and female Negro for comparison.
Comment. A curious feature of
this XVII Report is that although care was taken in
regard to the age, race, and sex of the subjects, no
attempt was made to "limit the examinations to
continuous resident children" (Hill et al.,
1957b). Thus it is reasonable to assume that some
children were examined who were not "continuous"
residents. Therefore the comparisons mentioned by
Doctors Blayney and Hill were made on mixed samples
of children some of whom had not ingested
fluoridated water, those examined in 1946, and
others who had done so for varying periods of up to
about eight years. This disregard of the possible
effect of the ingestion of fluorides on the caries
rates, of some of the children examined, is
inexplicable.
(23) As Dr Sutton calls attention to
the difference in the number of examinations made in
this report when compared with other reports, we
wish to point out, in explanation, that it was
necessary to not only insure that children of
correct age be included but also it was necessary
that every examination be clearly classified
according to race and sex. There were 236 six to
eight-year-old children and 245 12 to 14-year old
children excluded from this report because they did
not fulfil the requirements for this comparison.
These children were included in other reports as no
distinction except age was made.
Comment. This explanation why
the sample sizes from Evanston shown in the XVII
Report (1,754 six to eight-year-old children and
1,556 twelve to fourteen-year-old children) do not
agree with those shown in other reports at first
appears to be a reasonable one. However, before it
is accepted, consideration should be given to two
observations. Firstly, if the figures depicted in
statements "B" and "C" of Figure 5 which were
originally given by Hill et al. and are now
confirmed by Doctors Blayney and Hill (20, 22), are
accepted as correct, the numbers of children
excluded in the two age groups (1,991-1,754 and
1,701-1,556) were 237 and 145, not 236 and 245 as
stated by Doctors Blayney and Hill. It is possible
that these errors could have arisen in typing the
manuscript, but this could not be the case in regard
to the second observation.
This second observation is as
follows: In the XVII Report (Hill et al.,
1957b) it was stated that "in this report no attempt
has been made to limit the examinations to
continuous resident children." Thus it is almost
certain that data from both "continuous" and
"non-continuous" resident children are included in
the total of 3,310 subjects mentioned in that Report
as examined in Evanston in 1946. Therefore, to
determine the number of children who were excluded
from the XVII Report, comparison must be made, not
with the number of "continuous" residents of correct
age that were examined in 1946 (3,692, see 20) and
"were included in other reports", but with the total
number of children ("continuous" and
"non-continuous" residents and "out of range") that
were examined in 1946, that is, 4,375.
If this is done, it can be seen that
1,065 of these children (4,375-3,310) were excluded
from the XVII Report. This figure includes some "out
of range" children, for it was stated that "some
children below the age of 67 months and above the
maximum of 174 months had been examined" (20) in the
baseline examination of 4,375 Evanston children.
Nevertheless, unless there were as many as 584 "out
of range" children (1,065 = 584 + 481), the actual
number of children excluded, because they were not
of correct age or could not "be clearly classified
according to race and sex", must have been larger
than the figure of 481 given here (236 + 245) by
Doctors Blayney and Hill.
(24) It is also pointed out that
this report, Evanston No. XVII as noted above, was
primarily concerned with comparison of the caries
rates of the coloured and white children and not
with the effect of fluoridation on dental caries
rates. Therefore in this light, this report, No.
XVII, should not have been listed under the general
heading of fluoridation as Dr Sutton has it listed
in his critique.
Comment. The XVII Report from
Evanston was not listed as a fluoridation study; it
appears in the list of references (as do other
papers not specifically concerned with fluoridation)
because it was mentioned in the text. It was
consulted in an attempt to investigate the confusing
matter of the differences in the sample sizes for
1946 and 1947 shown in the various reports.
(25) It is true that a discrepancy
in figures published in our paper XVI, Table I and
in paper XVIII, Table I are at variance This is due
to the operator of the tabulating machine providing
the wrong figures for the number of seven and eight
year old children examined. This error was
discovered after manuscript XVI was in press and
therefore, the corrections could only be made
manually in the reprints supplied to the readers who
requested them.
Comment. If the errors
contained in the XVI Report had been pointed out by
providing an additional footnote to Table I in the
XVIII Report (which was the next report in which
this type of data was published) the reason for the
difference between the two sets of figures would
have been obvious. It should be noted that, although
the source of the errors in the 1955 rates in Table
1, XVI Report was given, no mention has been made of
the fact that, in the same table, there are errors
in computing the rates for the six to eightyear-old
age group in the years 1946 and 1948.Both of these
errors were of long standing as they were shown,
four years earlier, in the X Report (Hill et al.,
1952). These errors were still contained in the XV
Report (Hill et al., 1957a).
It can be seen that Doctors Blayney
and Hill devoted a considerable part of their review
to two matters. The first was the "comparability of
the study and the control areas"-which was not
questioned (see comment 13). The second was a
lengthy description of the comparisons which they
made between different groups of children in
obtaining the data for the XVII Report. This
information was given in almost the same words in
that report, which, they stated (24), was not
primarily concerned with the effect of fluoridation
on dental caries rates.
However, they have not mentioned
most of the matters which do directly concern
fluoridation and caries prevalence and which were
questioned. In fact their comments have touched on
matters mentioned in only about a third of the
sub-headings used in considering their study. Their
meagre explanations have accounted for the presence
of some of the errors in one table .... and have
supplied a reason for the differences between the
sample sizes for the year 1947 in Oak Park, and for
the disparity between two of the three sample sizes
for 1946 in Evanston (p 165) However, most of the
matters mentioned in considering the Evanston study
were ignored, even those illustrated by Figures 3
and 4.
It should be noted, therefore, that
Doctors Blayney and Hill have not commented on the
majority of the errors, omissions and mis-statements
mentioned in considering the Evanston study, and
almost all of them remain unexplained.
... Continued next page
No comments:
Post a Comment