Errors and Omissions in
Experimental Trials - 2a
PART THREE
CRITICISMS AND COMMENTS
Soon
after the publication of the first edition of this monograph in September 1959,
the author was informed by Dr K. T. Adamson, President of the Australian Dental
Association, that copies had been sent to "all of the men who are in
charge of the experiments" asking them for
comments (personal
communication). As a result the reviews published in the February 1960 issue of
the Australian Dental Journal were contributed by Dr Donald Galagan, Dr J. R.
Blayney and Dr I. N. Hill, and by Dr R. M. Grainger. The review in the New
Zealand Dental Journal of January 1960 was written by Mr J. Ferris Fuller.
The
aim of this monograph is to attempt to clarify some aspects of five crucial
trials of artificial fluoridation, those conducted in Grand Rapids, Evanston,
and Newburgh, U.S.A., and the two trials held in Brantford, Canada. Therefore,
in this part, all these critical reviews will be quoted in full; and comments
will be made on the points raised and indicated by the figures in brackets.
In
order to allow the reader to appraise the criticisms more easily it will be
necessary to refer to the statements made in the text, therefore page
references are indicated for Parts One and Two of this volume.
DR DONALD GALAGAN
DR DONALD GALAGAN
The
first review of this book in the Australian Dental Journal, by Dr Donald
Galagan, Assistant Chief, Division of Dental Public Health, Public Health
Service, Washington 25, D.C., was as follows:
My
comments will be limited to the general qualities of Dr Sutton's treatise and
the conclusions he has reached. Individuals associated with the several
fluoridation projects which he has purported to "analyse" will have
provided their specific reactions to his "analysis" of their
findings.
Although
it is nothing new to see an accredited scientist mix fact and fancy, near truth
with truth, and emotion with reason it is always shocking to realise that an
intelligent individual in a responsible position can so baldly misinterpret
scientific data. Actually, it would be extremely difficult for an objective
scientist who knows anything at all about the data characterizing the
relationship between dental caries and exposure to fluorides to reach the
conclusion that the effectiveness of controlled water fluoridation in reducing
dental caries has not been proved. This conclusion could only be reached by an
armchair statistician who has chosen to ignore or does not know the great mass
of information on the subject.
Thus,
first and foremost among the fundamental errors(1) which Dr Sutton makes is
expressed in the statement in the second paragraph of the monograph which says
that "proposals to fluoridate domestic water are almost entirely based on
the results of the Brantford, Grand Rapids, Newburgh, and Evanston
projects". Using this premise, the balance of the document is devoted to
efforts to describe "errors and omissions" in these four projects.
These errors are supposed to negate the whole fluoride-caries hypotheses(2), or
at least to throw serious doubt on the fluoridation of water as a caries
preventive.
Actually,
the scientific basis on which this public health measure rests was established
solidly before any of the above mentioned projects were started. The preventive
effect of long term exposure to water-borne fluoride on caries experience was
observed in literally thousands of children residing in many different
communities where the water consumed had picked up the element as it coursed
over or through the earth's crust. The long series of investigations
documenting the relationship are considered to be classic examples of good
epidemiological method(3) - so much so that they are used as case studies in
teaching the science of epidemiology in our schools of public health.
The
fact is that the projects at Brantford, Grand Rapids, Newburgh and Evanston
were designed primarily to evaluate the technical, financial and administrative
problems associated with the controlled addition of fluorides to a municipal
water supply, and, secondarily, to demonstrate the effectiveness of the
procedure(4) to the profession and the public. To be sure, it was necessary to
show that the procedure would reduce the caries attack rate in resident
children, but as soon as the trend toward reduction was observed and
corroborated, the "experimental" portions of the projects were completed
for all practical purposes. The principal point, however, is that these
projects emphatically are not the sole basis on which the widespread use (in
the United States) of this procedure rests(5).
In
short, it is preposterous to attempt to conclude that the basis for community
water fluoridation is faulty because of some real or imaginary defects in the
planning, execution and evaluation of data from the four community projects.
The contents of the monograph, therefore, represent no more than an exercise in
semantic and scientific dilettantism designed to serve some other purpose.
There
are a good many other specific, but less important errors in judgment which the
author has made, such as his suggestion that the variability of examiners in
diagnosing dental caries has been overlooked(6). The truth is that this
variability is well known, and is discussed at length by specialists in the
fields of epidemiology and caries diagnosis. Because of this "human
error" calibration of examiners is practised as a matter of course.
Calibration reduces the variability among examiners, but even if it did not,
the difference between the caries experience of children exposed and not
exposed to fluoride is so great that even Dr Sutton could recognise it(7).
There
are several other examples of errors of judgment in the arguments contained in
the monograph, indicating the author's serious lack of understanding of the
principles of statistics and epidemiology. For instance, the rather amateurish
interpretation of adequate community "controls" for evaluating the
effect of fluoride-bearing water on caries indicates that the author does not
really know the manner in which caries occurs in a population(8).
The
author's use of innuendo to make a point is contained in his reference to some
very questionable work of Feltman which indicated that the eruption of
deciduous teeth might occur later in children exposed to fluorides. The author
implies that it is likely that retarded tooth eruption in the children residing
in the fluoride communities reduces their exposure to caries attack, and thus
their caries rate naturally is lower(9). Had Dr Sutton been familiar with the
literature, he would have known that this was one of the first possibilities
thought responsible for the low caries rate in children exposed to
fluoride-bearing water (10). He would have known about the rather exhaustive
report of Short on the relation between fluoride in domestic waters and tooth
eruption which showed that fluoride in concentrations around the optimum used
for caries control does not influence the eruption pattern of permanent teeth
(11).
However,
one suspects that further analysis of the details contained in the monograph
will not yield much of value. From reading the document and from hearing him
present part of it as a paper last February at your Adelaide Congress, I can
only conclude that Dr Sutton has an intense and emotional drive to oppose
fluoridation. Why he feels this way is not clear, but it seems likely to come
from some motive other than a sincere concern for the statistical or scientific
validity of the concept(12).
Commentary on the Review by Dr Donald Galagan
Commentary on the Review by Dr Donald Galagan
(1)
Dr Galagan makes the charge that the "first and foremost among the
fundamental errors" was made in the second paragraph of the monograph (p.
136). This paragraph stated: "Apart from these considerations, an
examination reveals that there are aspects that call for a very careful
appraisal of the figures presented in the reports of the experimental trials
which have been conducted in Brantford, Canada, and in Grand Rapids, Newburgh
and Evanston, U.S.A., and upon the results of which proposals to fluoridate
domestic water are almost entirely based." Such proposals are based on two
different sets of results- those reported from areas where fluorides occur
naturally in the water supplies, and those from trials of the mechanical
addition of fluorides to waters in which the fluoride content is very low or
absent. The former reports are not considered in this investigation, but Dr
Galagan says that they were gathered in such a way that they "are
considered to be classic examples of good epidemiological method". In any
case, they are of practical importance only if it is known that the results
reported will be obtained when commercially available fluorides are
mechanically added to water supplies.
This
knowledge can be obtained only by the use of experimental trials the results of
which must then be considered to be of outstanding importance-unless it is
accepted that it can be established, on theoretical grounds, that the results
of artificial fluoridation will be identical with those seen in areas where
fluorides are found naturally. If the latter case is accepted, the early
artificial schemes which are considered here were held merely, as Dr Galagan
suggests, "to demonstrate the effectiveness of the procedure" and
cannot be considered to be true experimental trials.
The
question to be answered is this: Were these trials mere demonstrations or were
they set up as genuine experimental studies? If they were only demonstrations
Dr Galagan's charge is justified, but if these trials were conducted to
determine the outcome of the process of mechanical fluoridation, then his
accusation is without foundation and must be discredited.
In
1951 a report was issued by the ad hoc Committee on Fluoridation of Water
Supplies of the National (U.S.A.) Academy of Sciences, National Research
Council (Maxcy, Appleton, Bibby, Dean, Harvey, Heyroth, Johnson, Whittaker and
Wolman, 1952). One of the members of this Committee was Dr H. Trendley Dean,
who was closely associated with much of the earlier work concerning fluorides
and was, at that time, the Director of the National Institute of Dental
Research. The report included this statement: "In 1945, studies were begun
to ascertain whether the adjustment of the fluoride content of a public water
supply to the optimal level with commercially available fluorides would confer
the same caries-inhibitory effects as do waters which carry the same
concentrations of fluoride naturally."
This
statement makes it clear that Dr Galagan's contention, that these trials were
only demonstrations, is not correct. However, because of the importance of this
matter and in case it is suggested that that Committee was misinformed
concerning the intention of these trials, a quotation will be given from each
of the five studies considered. Dr Galagan calls the process used in these test
cities "the controlled addition of fluorides to a municipal water
supply", but the original term used by Dean et al. (1950) and by Brown (1951, 1952), "artificially
fluoridated" drinking water, is preferable as it is free from ambiguity.
(a)
Brantford, City Health Department Study, Hutton et al., (1951): "It was recognized that fluorine in the public
water supply was not a proven method for the prevention of dental caries, and
that it might take ten years to prove or disprove its preventive value."
(b)
Brantford, National Health and Welfare Study, Brown et al., (1954b): "The Brantford Fluoridation Caries Study was
undertaken with a view to finding out whether or not the raising of the
fluoride content of a previously fluoride-free water supply to part per
million, by the mechanical addition of sodium fluoride, would reduce the
incidence of dental caries to that which obtains where water supplies derive
about 1 part per million of fluoride from deposits in the earth"
(c)
Grand Rapids, Dean et al. (1950):
"In 1945, three studies to determine the caries prophylactic value of
artificially fluoridated drinking water were started in the United States and
Canada."
(d)
Newburgh, Ast et al. (1950): "In
1943 it was proposed to determine whether we can translate the conclusions
derived from the epidemiological studies in fluoride areas to a practical
application in fluoride-free areas where the communal water supplies may lend
themselves to treatment."
(e)
Evanston, Blayney and Tucker (1948): "After further deliberation of the
project, both professional groups recommended to the Commissioner of Health
that a carefully controlled study be developed to determine whether or not the
addition of fluorine in minute quantities to the communal water supply would
reduce the incidence of dental caries in Evanston and Skokie children."
Blayney and Tucker (1948) also said: "It was carefully explained to these Evanston
citizens that nothing could be promised regarding the ultimate value in the
control of tooth decay; that if such a program was to be undertaken it must be
in the nature of an exhaustive study; and that it would be several years before
data would be available which would even indicate the trend which we might
expect."
These
statements, by the authors of all of the five studies considered, establish
beyond question that in every case the studies were not designed "to
demonstrate the effectiveness of the procedure" but to determine whether
or not artificial fluoridation would be efficacious. Therefore Dr Galagan's
opinion is incorrect, and the statement made in Part One, which he termed the
"first and foremost among the fundamental errors", is a correct
description of the situation.
(2)
His suggestion that an attempt has been made to "negate the whole ''
fluoride-caries hypotheses" is without foundation. The only reference to
this matter is contained in the quotation of a statement. made in 1949, by the
American Water Works Association regarding the experimental verification of
"the fluoride-dental caries hypothesis" - that is, to the
"fluoridation hypothesis". Statements made in the preface to the
first edition and on the first page of Part One and the first and last pages of
Part Two show that consideration has been given only to five experimental
trials of artificial fluoridation produced by mechanical means. The data from
epidemiological studies in "naturally fluoridated" areas, on which the
fluorine-dental caries hypothesis is based, have not been considered.
(5)
Dr Galagan states that "The principal point, however, is that these
projects emphatically are not the sole basis on which the widespread use (in
the United States) of this procedure rests." It can be seen, by reading
the "free" quotation from paragraph two, page 136, given by this
reviewer (1) and pages 140, 189, 190 and 196, that the word "sole"
was not used nor implied.
No
comment can be made on the other "fundamental errors" which Dr Galagan
says (1) are present-for he has neglected to state their nature. Instead he
continues:
(6)
There are a good many other specific, but less important errors in judgment
which the author has made, such as his suggestion that the variability of
examiners in diagnosing dental caries has been overlooked.
Comment. This suggestion was not made. As
Dr Galagan points out in his next sentence (7), "this variability is well
known". Therefore it is most unlikely that such a matter would be
"overlooked" in studies employing statisticians. Indeed, attention
was drawn, on page 180, to the fact that the importance of examiner variability
was recognized by Ast et al. in 1950.
However, in conducting the clinical examinations, in all the studies
considered, this matter was ignored. Therefore, when speaking of this
phenomenon, terms such as "not assessed" and "not
estimated" were used.
(7)
The truth is that this variability is well known, and is discussed at length by
specialists in the fields of epidemiology and caries diagnosis. Because of this
"human error" calibration of examiners is practised as a matter of
course. Calibration reduces the variability among examiners, but even if it did
not, the difference between the caries experience of children exposed and not
exposed to fluoride is so great that even Dr Sutton could recognise it.
Comment. As Dr Galagan says, this
(examiner) variability is well known. It is precisely this fact that makes it
so surprising that this factor was not assessed in these studies. It will be
recalled that reference was made, on page nine, to two papers which
investigated the matter of examiner variability in caries diagnosis.
The
claim is made by the reviewer that "Calibration reduces the variability
among examiners", but he does not suggest that this process eliminates
between-examiner variability-therefore it should have been taken into account.
Of course, the use of the method of the "calibration" of the
subjective judgment of several examiners with the subjective judgment of
another is, to say the least, a poor substitute for a standard rigorous
statistical procedure.
(8)
There are several other examples of errors of judgment in the arguments
contained in the monograph, indicating the author's serious lack of
understanding of the principles of statistics and epidemiology. For instance,
the rather amateurish interpretation of adequate community "controls"
for evaluating the effect of fluoride-bearing water on caries indicates that
the author does not really know the manner in which caries occurs in a
population.
Comment. The question may be asked: Who
does?
(9)
The author's use of innuendo to make a point is contained in his reference to
some very questionable work of Feldman (p. 192) which indicated that the
eruption of deciduous teeth might occur later in children exposed to fluorides.
The author implies that it is likely that retarded tooth eruption in the
children residing in the fluoride communities reduces their exposure to caries
attack, and thus their caries rate naturally is lower.
Comment. On page 192, reference was not
made to the work; of Feldman but to that of Dr Reuben Feltman who was an
associate of Doctors D. E. Gardner and F. A. Smith, whose publications on
fluorides are well known, and of Doctors H. C. Hodge and D. E. Overton, who were
closely associated with the Newburgh trial (Gardner, Smith, Hodge, Overton and
Feltman, 1952). Dr Feltman's earlier (1951) work with fluoride tablets was
referred to by the New Zealand Commission of Inquiry (1957) as
"promising". His 1956 paper, which was quoted (p.192), was a brief
"progress report" only. Therefore the results mentioned in it were
treated with reserve, the statement being made that "Of course, if
fluoridation results in the eruption rate of teeth being retarded ....".
(10)
Had Dr Sutton been familiar with the literature, he would have known that this
was one of the first possibilities thought responsible for the low caries rate
in children exposed to fluoride-bearing water.
Comment. Dr Galagan has avoided the main
point which was discussed in this paragraph. His insistence that one of the
first possibilities considered was that the ingestion of fluoride-bearing water
may retard tooth eruption, makes it even more strange that in only the
Newburgh-Kingston study was mention made that this important matter had been
investigated. Even in that trial, the only study of tooth eruption rate
published was conducted after four years of fluoridation (Ast et al., 1951) in children who were six
to twelve years of age, so that none of the subjects studied had been ingesting
artificially fluoridated water throughout their lives.
(11)
He would have known about the rather exhaustive report of Short on the relation
between fluoride in domestic waters and tooth eruption which showed that
fluoride in concentrations around the optimum used for caries control does not
influence the eruption pattern of permanent teeth.
Comment. It is highly probable that Dr
Galagan is referring to the paper by E. M. Short (1944) which is well known to
those interested in fluorides, for this is the only paper concerning fluorides
and tooth eruption listed under that name in the Index to Dental Literature in
the English Language (published by the American Dental Association), the
Quarterly Cumulative Index Medicus (to December 1956), or the issues of the
Current List of Medical Literature which cover the subsequent period.
This
"rather exhaustive report of Short" does not show whether or not the
ingestion of fluorides at the "optimum" level has any influence on
the eruption pattern of permanent teeth. This report was made on "selected
12 14 year old white school children" (Short, 1944) in whom almost all the
permanent teeth had erupted. The data deal only with the total number of
erupted permanent teeth and, despite Dr Galagan's remark, do not give any
information regarding their eruption pattern-a factor which could be of
considerable importance in the development of the DMF rate. Short's Tables II
and III show that, in all except three of the 4,453 children examined, at least
twelve of the permanent teeth had erupted, the minimum number of erupted
permanent teeth, for the remaining three children, being ten. His Table I
(which excluded third molars) shows that, out of the possible twenty-eight
teeth, the mean number of erupted permanent teeth per child in the various
cities was between 25.22 and 26.81. In fact, in fifty-five per cent of the
children all of the twenty-eight teeth had erupted.
Therefore,
this study of Short (1944) gives no information regarding the ages at which the
first ten permanent teeth erupted in these children, certainly none regarding
the first permanent molars which, presumably, even in the youngest of these
children, erupted about five or six years prior to the study. These molars are
of outstanding importance in regard to the DMF rate, particularly in young
children. Ast et al. (1956) said:
"The first permanent molars are frequently used as an index of caries
experience among children because this tooth accounts for the major incidence
of caries in this group."
It
should be noted that neither Dr Galagan nor (as will be seen in their review of
this book) Doctors Blayney and Hill, authors of the Evanston study, have
commented on the suggestion of a decline in the eruption rate of first
permanent molars in Evanston, between 1946 and 1951, which followed the
introduction of fluoridation and which is depicted in Figure 4 (p. 139).
Neither have they explained why, after 1951, whilst continuing to publish this
type of data for the older children, they ceased publishing it for the younger
(12)
However, one suspects that further analysis of the details contained in the
monograph will not yield much of value. From reading the document and from
hearing him present part of it as a paper last February at your Adelaide
Congress, I can only conclude that Dr Sutton has an intense and emotional drive
to oppose fluoridation. Why he feels this way is not clear, but it seems likely
to come from some motive other than a sincere concern for the statistical or
scientific validity of the concept.
Comment. Dr Galagan's questioning of the
motive behind this study should be considered in relation to statements which
he made during his lecture to the Adelaide Congress (Galagan, 1959). He called
those who questioned fluoridation "the opposition" and said that this
group "seems to be composed of four distinct kinds of people." These
he termed: "the hatemonger, the pseudo-health believer, and the person who
opposes fluoridation for personal notoriety" and "the fourth, or rugged
individualist, group". As these are Dr Galagan's views, it is not
surprising that he doubts the sincerity of this attempt to investigate
"the statistical or scientific validity" of these fluoridation
findings.
DR J. R. BLAYNEY and DR I. N. HILL
DR J. R. BLAYNEY and DR I. N. HILL
The
second review of this book in the Australian Dental Journal, by DR J. R.
BLAYNEY, Director, Dental Caries Study, Evanston, and DR I. N. HILL, Zoller
Memorial Dental Clinic, University of Chicago, was as follows:
Dr
Sutton has much to say regarding the lack of comparability(13) of the study and
the control areas and the manner of selection of the children in each area to
be examined. Oak Park is the suburb immediately to the west of Chicago and
Evanston lies immediately to the north. Each community draws its water supply
from Lake Michigan. Standard analyses for composition of tlw water are
frequently run. A spectro-chemical analysis for 26 trace elements has been run
on Lake Michigan water for a comparison of a similar analysis of water obtained
from fluoride areas (unpublished data). Oak Park and Evanston receive their
food from the same wholesale markets, each is chiefly residential and free from
heavy industry. Each is composed of the same socio-economic level, as borne out
by the United States census of population for 1950. In Evanston, 47,395 persons
21 years or over were native born. In Oak Park the figure was 42,454. In
Evanston there were 6,049 foreign born persons 21 years or older while in Oak
Park the figure was 5,081. In Evanston 41.2 per cent of the occupied dwelling
units were owner occupied. In Oak Park 50.8 per cent were occupied by the
owner. The median value of a one dwelling unit structure in Evenston was 19,499
dollars and in Oak Park the value was 16,259 dollars. The median value of gross
monthly rentals in renter occupied dwelling units in Evanston was 72.53 dollars
while in Oak Park the median value was 66.86 dollars. Both areas have
comparable, climatic conditions and both are subjected to the same radio and
television commercials regarding oral hygiene and dentifrices. Finally, the
majority of the dental practitioners in the study and control areas are
graduates of one of the three Chicago dental colleges.
For
the baseline examination in Evanston we were committed to examine all school children within the selected
age range regardless of the length of time they had resided in Evanston.
Although an effort was made to include them, the Oak Park Parochial Schools did
not find it convenient for us to examine their pupils(14). When we compared the
caries prevalence rates of the two towns we found a difference. It is indeed
fortunate that our records showed the school that each child attended. In this
manner we could first eliminate the Parochial School group and then the Negro
group from the total Evanston data. It is well known that coloured people have
less dental caries than whites living in the same population centres(15). The
caries rates for the Evanston and Oak Park Public School white children
compared favourably(16). All of this clearly indicates how hidden variables may
exist in areas which otherwise appear to be comparable, and how important it is
to be certain that comparisons are made between like groups(17).
Dr
Sutton expressed astonishment that in 1955 the six and seven-year-old Evanston
children had a lower caries prevalence rate than the Aurora children of like
ages. We, likewise, did not anticipate this. However, the same critical
evaluation both clinical and roentgenological, was made of every case. This
difference was due to something other than fluoride. Possibly the presence of
the dental team in the school, year after year, has stimulated the classroom
teachers and the school nurses to place more emphasis on the teaching of oral
health. Some unknown hidden variation not related to fluorides must account for
the difference(18).
Dr
Sutton was concerned that the control group was not examined annually. Neither
we nor our advisers could see a reason to require an examination of the control
group other than at the beginning and near the close of the study. This
provides the baseline from which to measure the trend of the dental caries rate
during the time interval (1947-1956). Should the rate in the last examination
(1956) deviate materially from that of the initial baseline period (1947) that
figure could be used as a correction factor on the Evanston findings. In fact,
if we had not desired to measure the yearly decrement in the rate of dental
caries under fluoridation and evaluate other factors only two examinations in
Evanston would have been necessary, the first in 1946 (before fluoridation),
the second and final in 1961(19).
Much
has been made of the variations reported in the number of children examined.
The baseline examination of 4,375 Evanston children and of 2,493 Oak Park
children are correct. However, the data from those children who had not used
Lake Michigan water all of their lives had to be excluded. It was also observed
that some children below the age of 67 months and above the maximum of 174
months had been examined. Therefore these out of range children were not
considered in the final determination of the caries rates. This explains the
discrepancy between the Evanston 4,375 and 3,692 and the Oak Park 2,493 - 2,051
figures(20). It should be noted that when dental caries experience rates were compared,
the same number of examinations, that is 1,991 for the six to eight-year-old
children and 1,701 for the 12 to 14-year-old children, were used throughout the
reports for the combined Evanston school groups (Public White, Parochial and
Public Negro)(21). Dr Sutton on p. 167 of his report (fig. 5, Statement C)
calls attention to Evanston Dental Caries Study Report Number XVII. Here he
points out that only 1,754 six to eight-year-old children and 1,556 12 to
14-yearold children were listed. This particular report was primarily concerned
with differences in sex and race, as they influenced caries, rather than the
effect of fluorides. Therefore a comparison was made of the dental caries
experience rates of white girls to white boys; Negro girls to Negro boys; white
girls to Negro girls; white boys to Negro boys; and white children as a group
to Negro children as a group. The children were classified into male white,
female white, male Negro and female Negro for comparison(22). As Dr Sutton
calls attention to the difference in the number of examinations made in this
report when compared with other reports, we wish to point out, in explanation,
that it was necessary to not only insure that children of correct age be
included but also it was necessary that every examination be clearly classified
according to race and sex There were 236 six to eight-year-old children and 245
12 to 14-year-old children excluded from this report because they did not
fulfil the requirements for this comparison. These children were included in
other reports as no distinction except age was made(23). It is also pointed out
that this report, Evanston No. XVII as noted above, was primarily concerned
with comparison of the caries rates of the coloured and white children and not
with the effect of fluoridation on dental caries rates. Therefore in this
light, this report, No. XVII, should not have been listed under the general
heading of fluoridation as Dr Sutton has it listed in his critique(24).
It
is true that a discrepancy in figures published in our paper XVI, Table I and
in paper XVIII, Table I are at variance. This is due to the operator of the
tabulating machine providing the wrong figures for the number of seven and
eight-year-old children examined. This error was discovered after manuscript
XVI was in press and therefore the corrections could only be made manually in
the reprints supplied to readers who requested them(25).
Commentary on the Review by Dr J.
R. Blaynex and Dr L N. Hill
(13)
Reference to Part Two will show that no suggestion was made in it that there
was a "lack of comparability" between Evanston and Oak Park; it was
merely pointed out that the manner in which Oak Park resembled Evanston was not
stated.
Therefore
this detailed exposition of the similarity of the two cities is welcome,
although it is unfortunate that, when speaking of rental and dwelling values,
the "mean" values were not given as well as the "median"
ones. It will be realized that the housing picture may be very different in two
towns and yet the "median" (that is, the middle) values of the
rentals and dwellings can be the same. In view of the data shown here,
regarding housing and the training of the dentists, it is surprising that the
members of the United Kingdom Mission (1953) should have singled out Evanston
for comment, remarking on its high economic level and its "outstandingly
good" dental care (p. 149).
These
data also show how reasonable was the assumption of Dr Blayney before assessing
the caries rates in Oak Park-that that city was "the ideal control
community" for Evanston (Blayney and Tucker, 1948;). They also emphasize
how strange it is that such gross differences should be found between the
initial caries rates of the children aged six to eight years in the two cities,
and reported-after a delay of ten years-by these workers (p. 153; Fig. 3, p.
154). The fact that such gross differences can be found in the caries rates
prevalent in two cities which were so similar that one was termed "the
ideal control community" for the other (Blayney and Tucker, 1948),
confirms the necessity for pre-fluoridation examinations in both test and
control cities. Unfortunately, as pointed out in Part Two this was not done in
any of these studies.
(14)
For the baseline examination in Evanston we were committed to examine all
school children within the selected age range, regardless of the length of time
they had resided in Evanston. Although an effort was made to include them the
Oak Park Parochial Schools did not ford it convenient for us to examine their
pupils.
Comment. The latter remark is welcome for
it explains the absence of data from the parochial schools in Oak Park.
(15)
When we compared the caries prevalence rates of the two towns we found a
difference. It is indeed fortunate that our records showed the school that each
child attended. In this manner we could first eliminate the Parochial School
group and then the Negro group from the total Evanston data. It is well known
that coloured people have less dental caries than whites living in the same
population centres.
Comment. Despite their statement that
"It is well knowp that coloured people have less dental caries than whites
living in the same population centres", when conducting the initial
examination in Evanston the authors combined the data of the Negro children
with those of the white children. It is now clear, as deduced on page eighteen,
that the racial and school groups were taken into account only after it was
found that there was "a lower caries rate for school children of the
control area" (Hill et al.,
1951). Thereafter, when comparing the test and control cities, the data of both
the Negro and the parochial school children were excluded from the Evanston
data. No reason has been given for this exclusion of the data of parochial
school children in Evanston-who had a high caries rate (Hill et al.) - from the data of the main body
of white children in that city. This could not be attributed to the fording of
a similarly high caries rate in the parochial school children in the control
city of Oak Park for they were not examined and, therefore, their caries rates
were unknown (see 14).
Neither
has an explanation been offered for the extraordinary reversal of this policy
(the exclusion of the data of Negro and parochial school children in Evanston)
when compiling the XVIII Report (Hill et
al., 1958). This report published, for the first time, the initial caries
rates for the permanent teeth of the children aged six to eight years which
were obtained, ten years earlier, in the control city of Oak Park.
The
rates for the deciduous teeth, which were obtained at that time, still have not
been published. This report provided the first opportunity to compare the
initial caries rates of the younger children in the test city and its control
(p. 153).
(16)
The caries rates for the Evanston and Oak Park Public School white children
compared favourably.
Comment. This statement is interesting-for
the caries rates for children aged six to eight years in each of the three
school groups have not been published (p.151). It will be recalled that the
mean caries rates for the six, seven, and eight-year-old children in Evanston
in 1946 were very much higher than the mean rates for children of those ages
obtained during the initial examination in Oak Park. For the children aged six
years the rate in Evanston was 46.85, but it was only 26.89 in Oak Park (Hill et al., 1958). Only 0.1 per cent of the
Oak Park children were Negro (Hill et al.,
1951), but exclusion of the data of Negro children (who have a relatively low
caries rate, see 15) from the Evanston data would increase the rate of the
remaining (white public and parochial school) children so that in the
six-year-old children, it would be higher than 46.85. Therefore the difference
between this rate and the Oak Park rate of 26.89, for children of that age,
would be increased.
Hill
et al. did not say how many of the
younger age group of children attended each type of school, but only twenty-two
per cent of the twelve to fourteen-year-old children, shown in their 1957a
report as examined in Evanston in 1946, attended parochial schools. Therefore,
the proportion of children aged six, seven, and eight years who were attending
parochial schools in Evanston in 1946, and their caries rate, must have been
very high to permit Doctors Blayney and Hill to state that "The caries
rates for the Evanston and Oak Park Public School white children compared
favourably." Of course speculation is no substitute for data-and this
still has not been published
(17)
All of this clearly indicates how hidden variables may exist in areas which
otherwise appear to be comparable, and how important it is to be certain that
comparisons are made between like groups.
Comment. The latter phrase is a
reiteration of remarks made in the 1957a report from this study, that it is
necessary "to make comparisons of like groups." Why then, having
realized this necessity, did Hill et al.
ignore it in their 1958 report (p. 152)? In this report the data shown, for the
year 1946, combined not only that of the white children attending both public
and parochial schools, but the data of the Negro children as well. The
resultant rate was then compared with that of children in Oak Park comprising,
almost entirely, white children attending public schools. By ignoring the
opinion they expressed in the previous year (1957a)-which they now
reiterate-and comparing "unlike" groups of children, a more
favourable degree of comparability was obtained between the initial caries
rates of children in the test and the control cities.
(18)
Dr Sutton expressed astonishment that in 1955 the six and seven-year-old
Evanston children had a lower caries prevalence rate than the Aurora children
of like ages. We, likewise, did not anticipate this. However, the same critical
evaluation both clinical and roentgenological, was made of every case. This
difference was due to something other than fluoride. Possibly the presence of
the dental team in the school, year after year, has stimulated the classroom
teachers and the school nurses to place more emphasis on the teaching of oral
health. Some unknown hidden variation not related to fluorides must account for
the difference.
Comment. It is pleasing that Doctors
Blayney and Hill should support the view expressed on page 187 (para. 4) that
regular dental examinations may stimulate interest in the teeth and thus lead
to improved oral health. In advancing the suggestion that "something other
than fluoride" can affect the caries rates, they recognize the great
importance which factors other than the fluoride concentration of the water
supply may have on the caries rates. This extremely important matter was
practically ignored by the authors of all these studies when preparing their
reports.
(19)
Dr Sutton was concerned that the control group was not examined annually. Neither
we nor our advisers could see a reason to require an examination of the control
group other than at the beginning and near the close of the study. This
provides the baseline from which to measure the trend of the dental caries rate
during the time interval (1947 1956). Should the rate in the last examination
(1956) deviate materially from that of the initial baseline period (1947) that
figure could be used as a correction factor in the Evanston findings. In fact,
if we had not desired to measure the yearly decrement in the rate of dental
caries under fluoridation and evaluate other factors, only two examinations in
Evanston would have been necessary, the first in 1946 (before fluoridation),
the second and final in 1961.
Comment. This statement makes two things
clear. The first is that, at the commencement of the study, neither the workers
nor their advisers could have considered the possibility, which they now
acknowledge(18), that "the presence of the dental team in the school, year
after year" might have had a stimulating effect "on the teaching of
oral health." It is obvious that, if this effect is possible, not only the
test town but also its control should have been examined "year after
year". The other point which is indicated by this statement(19) of Doctors
Blayney and Hill is that despite their remark in 1950, the importance of random
variation was not and, seemingly still is not recognized.
(20)
Much has been made of the variations reported in the number of children
examined. The baseline examination of 4,375 Evanston children and of 2,493 Oak
Park children are correct. However, the data from those children who had not
used Lake Michigan water all of their lives had to be excluded. It was also
observed that some children below the age of 67 months and above the maximum of
174 months had been examined. Therefore these out of range children were not
considered in the final determination of the caries rates. This explains the
discrepancy between the Evanston 4,375 and 3,692 and the Oak Park 2,493 - 2,051
figures.
Comment. This explanation of the
difference between these sample sizes in Evanston and Oak Park is welcome. It
might have been deduced if the decision to exclude "the data from those
children who had not used Lake Michigan water all of their lives" had been
announced in one of the five reports giving caries rates, issued prior to 1955
(p. 163).
In
the XIX Report (Hill et al., 1959),
the sample sizes shown for the two age groups in Oak Park in 1947 (1,022 and
1,032) are almost the same as those shown (1,020 and 1,031) in statement
"E" of Figure 5. Comparison with the statements for Evanston cannot
be made for this (XIX) report considered only "public school white
children".
(21)
It should be noted that when dental caries experience rates were compared, the
same number of examinations, that is 1,991 for the six to eight-year-old
children and 1,701 for the 12 to 14-year-old children were used throughout the
reports for the combined Evanston school groups (Public White Parochial and
Public Negro).
Comment. It is surprising that the
suggestion was made that this statement should be noted-for it is not correct.
The number of twelve, thirteen and fourteen-year-old children examined in
Evanston in 1946 was given in Tables III, V, VI, VII, VIII, IX and X of the XV
Report (Hill et al., 1957a) as 418,
688 and 595, a total of 1,701. However, in Tables XI and XII of the same paper
different sample sizes for these ages were shown: 414, 692 and 617, a total of
1,723. The same discrepancies were noted between different tables in the XI
Report (Hill et al., 1955) Therefore
the figure 1,701 was not "used throughout the reports for the combined
Evanston school groups".
(22)
Dr Sutton on p. 167 of his report (fig. 5, Statement C) calls attention to
Evanston Dental Caries Study Report Number XVII. Here he points out that only
1,754 six to eight-year old children and 1,556 12 to 14-yearold children were
listed. This particular report was primarily concerned with differences in sex
and race, as they influenced caries, rather than the effect of fluorides.
Therefore a comparison was made of the dental caries experience rates of white
girls to white boys; Negro girls to Negro boys; white girls to Negro girls;
white boys to Negro boys; and white children as a group to Negro children as a
group. The children were classified into male white, female white, male Negro
and female Negro for comparison.
Comment. A curious feature of this XVII
Report is that although care was taken in regard to the age, race, and sex of
the subjects, no attempt was made to "limit the examinations to continuous
resident children" (Hill et al.,
1957b). Thus it is reasonable to assume that some children were examined who
were not "continuous" residents. Therefore the comparisons mentioned
by Doctors Blayney and Hill were made on mixed samples of children some of whom
had not ingested fluoridated water, those examined in 1946, and others who had
done so for varying periods of up to about eight years. This disregard of the
possible effect of the ingestion of fluorides on the caries rates, of some of
the children examined, is inexplicable.
(23)
As Dr Sutton calls attention to the difference in the number of examinations
made in this report when compared with other reports, we wish to point out, in
explanation, that it was necessary to not only insure that children of correct
age be included but also it was necessary that every examination be clearly
classified according to race and sex. There were 236 six to eight-year-old
children and 245 12 to 14-year old children excluded from this report because
they did not fulfil the requirements for this comparison. These children were
included in other reports as no distinction except age was made.
Comment. This explanation why the sample
sizes from Evanston shown in the XVII Report (1,754 six to eight-year-old
children and 1,556 twelve to fourteen-year-old children) do not agree with
those shown in other reports at first appears to be a reasonable one. However,
before it is accepted, consideration should be given to two observations.
Firstly, if the figures depicted in statements "B" and "C"
of Figure 5 which were originally given by Hill et al. and are now confirmed by Doctors Blayney and Hill (20, 22),
are accepted as correct, the numbers of children excluded in the two age groups
(1,991-1,754 and 1,701-1,556) were 237 and 145, not 236 and 245 as stated by
Doctors Blayney and Hill. It is possible that these errors could have arisen in
typing the manuscript, but this could not be the case in regard to the second
observation.
This
second observation is as follows: In the XVII Report (Hill et al., 1957b) it was stated that "in this report no attempt
has been made to limit the examinations to continuous resident children."
Thus it is almost certain that data from both "continuous" and
"non-continuous" resident children are included in the total of 3,310
subjects mentioned in that Report as examined in Evanston in 1946. Therefore,
to determine the number of children who were excluded from the XVII Report,
comparison must be made, not with the number of "continuous"
residents of correct age that were examined in 1946 (3,692, see 20) and
"were included in other reports", but with the total number of
children ("continuous" and "non-continuous" residents and
"out of range") that were examined in 1946, that is, 4,375.
If
this is done, it can be seen that 1,065 of these children (4,375-3,310) were
excluded from the XVII Report. This figure includes some "out of
range" children, for it was stated that "some children below the age
of 67 months and above the maximum of 174 months had been examined" (20)
in the baseline examination of 4,375 Evanston children. Nevertheless, unless
there were as many as 584 "out of range" children (1,065 = 584 +
481), the actual number of children excluded, because they were not of correct
age or could not "be clearly classified according to race and sex",
must have been larger than the figure of 481 given here (236 + 245) by Doctors
Blayney and Hill.
(24)
It is also pointed out that this report, Evanston No. XVII as noted above, was
primarily concerned with comparison of the caries rates of the coloured and
white children and not with the effect of fluoridation on dental caries rates.
Therefore in this light, this report, No. XVII, should not have been listed
under the general heading of fluoridation as Dr Sutton has it listed in his
critique.
Comment. The XVII Report from Evanston was
not listed as a fluoridation study; it appears in the list of references (as do
other papers not specifically concerned with fluoridation) because it was
mentioned in the text. It was consulted in an attempt to investigate the
confusing matter of the differences in the sample sizes for 1946 and 1947 shown
in the various reports.
(25)
It is true that a discrepancy in figures published in our paper XVI, Table I
and in paper XVIII, Table I are at variance This is due to the operator of the
tabulating machine providing the wrong figures for the number of seven and
eight year old children examined. This error was discovered after manuscript
XVI was in press and therefore, the corrections could only be made manually in
the reprints supplied to the readers who requested them.
Comment. If the errors contained in the
XVI Report had been pointed out by providing an additional footnote to Table I
in the XVIII Report (which was the next report in which this type of data was
published) the reason for the difference between the two sets of figures would
have been obvious. It should be noted that, although the source of the errors
in the 1955 rates in Table 1, XVI Report was given, no mention has been made of
the fact that, in the same table, there are errors in computing the rates for
the six to eightyear-old age group in the years 1946 and 1948.Both of these
errors were of long standing as they were shown, four years earlier, in the X
Report (Hill et al., 1952). These
errors were still contained in the XV Report (Hill et al., 1957a).
It
can be seen that Doctors Blayney and Hill devoted a considerable part of their
review to two matters. The first was the "comparability of the study and
the control areas"-which was not questioned (see comment 13). The second
was a lengthy description of the comparisons which they made between different
groups of children in obtaining the data for the XVII Report. This information
was given in almost the same words in that report, which, they stated (24), was
not primarily concerned with the effect of fluoridation on dental caries rates.
However,
they have not mentioned most of the matters which do directly concern fluoridation
and caries prevalence and which were questioned. In fact their comments have
touched on matters mentioned in only about a third of the sub-headings used in
considering their study. Their meagre explanations have accounted for the
presence of some of the errors in one table .... and have supplied a reason for
the differences between the sample sizes for the year 1947 in Oak Park, and for
the disparity between two of the three sample sizes for 1946 in Evanston (p
165) However, most of the matters mentioned in considering the Evanston study
were ignored, even those illustrated by Figures 3 and 4.
It
should be noted, therefore, that Doctors Blayney and Hill have not commented on
the majority of the errors, omissions and mis-statements mentioned in considering
the Evanston study, and almost all of them remain unexplained.
... Continued next page
No comments:
Post a Comment