Errors
and Omissions in Experimental Trials - 2a
PART
THREE
CRITICISMS
AND COMMENTS
Soon after the publication of the first edition of
this monograph in September 1959, the author was informed by Dr K. T. Adamson, President
of the Australian Dental Association, that copies had been sent to "all of
the men who are in charge of the experiments" asking them for comments
(personal communication). As a result the reviews published in the February
1960 issue of the Australian Dental Journal were contributed by Dr Donald
Galagan, Dr J. R. Blayney and Dr I. N. Hill, and by Dr R. M. Grainger. The
review in the New Zealand Dental Journal of January 1960 was written by Mr J.
Ferris Fuller.
The aim of this monograph is to attempt to clarify
some aspects of five crucial trials of artificial fluoridation, those conducted
in Grand Rapids, Evanston, and Newburgh, U.S.A., and the two trials held in
Brantford, Canada. Therefore, in this part, all these critical reviews will be
quoted in full; and comments will be made on the points raised and indicated by
the figures in brackets.
In order to allow the reader to appraise the
criticisms more easily it will be necessary to refer to the statements made in
the text, therefore page references are indicated for Parts One and Two of this
volume.
DR DONALD GALAGAN
The first review of this book in the Australian
Dental Journal, by Dr Donald Galagan, Assistant Chief, Division of Dental
Public Health, Public Health Service, Washington 25, D.C., was as follows:
My comments will be limited to the general
qualities of Dr Sutton's treatise and the conclusions he has reached.
Individuals associated with the several fluoridation projects which he has
purported to "analyse" will have provided their specific reactions to
his "analysis" of their findings.
Although it is nothing new to see an accredited
scientist mix fact and fancy, near truth with truth, and emotion with reason it
is always shocking to realise that an intelligent individual in a responsible
position can so baldly misinterpret scientific data. Actually, it would be
extremely difficult for an objective scientist who knows anything at all about
the data characterizing the relationship between dental caries and exposure to
fluorides to reach the conclusion that the effectiveness of controlled water
fluoridation in reducing dental caries has not been proved. This conclusion
could only be reached by an armchair statistician who has chosen to ignore or
does not know the great mass of information on the subject.
Thus, first and foremost among the fundamental
errors(1) which Dr Sutton makes is expressed in the statement in the second
paragraph of the monograph which says that "proposals to fluoridate
domestic water are almost entirely based on the results of the Brantford, Grand
Rapids, Newburgh, and Evanston projects". Using this premise, the balance
of the document is devoted to efforts to describe "errors and
omissions" in these four projects. These errors are supposed to negate the
whole fluoride-caries hypotheses(2), or at least to throw serious doubt on the
fluoridation of water as a caries preventive.
Actually, the scientific basis on which this public
health measure rests was established solidly before any of the above mentioned
projects were started. The preventive effect of long term exposure to
water-borne fluoride on caries experience was observed in literally thousands
of children residing in many different communities where the water consumed had
picked up the element as it coursed over or through the earth's crust. The long
series of investigations documenting the relationship are considered to be
classic examples of good epidemiological method(3) - so much so that they are
used as case studies in teaching the science of epidemiology in our schools of
public health.
The fact is that the projects at Brantford, Grand
Rapids, Newburgh and Evanston were designed primarily to evaluate the
technical, financial and administrative problems associated with the controlled
addition of fluorides to a municipal water supply, and, secondarily, to
demonstrate the effectiveness of the procedure(4) to the profession and the
public. To be sure, it was necessary to show that the procedure would reduce
the caries attack rate in resident children, but as soon as the trend toward
reduction was observed and corroborated, the "experimental" portions
of the projects were completed for all practical purposes. The principal point,
however, is that these projects emphatically are not the sole basis on which
the widespread use (in the United States) of this procedure rests(5).
In short, it is preposterous to attempt to conclude
that the basis for community water fluoridation is faulty because of some real
or imaginary defects in the planning, execution and evaluation of data from the
four community projects. The contents of the monograph, therefore, represent no
more than an exercise in semantic and scientific dilettantism designed to serve
some other purpose.
There are a good many other specific, but less
important errors in judgment which the author has made, such as his suggestion
that the variability of examiners in diagnosing dental caries has been
overlooked(6). The truth is that this variability is well known, and is
discussed at length by specialists in the fields of epidemiology and caries
diagnosis. Because of this "human error" calibration of examiners is
practised as a matter of course. Calibration reduces the variability among
examiners, but even if it did not, the difference between the caries experience
of children exposed and not exposed to fluoride is so great that even Dr Sutton
could recognise it(7).
There are several other examples of errors of
judgment in the arguments contained in the monograph, indicating the author's
serious lack of understanding of the principles of statistics and epidemiology.
For instance, the rather amateurish interpretation of adequate community
"controls" for evaluating the effect of fluoride-bearing water on
caries indicates that the author does not really know the manner in which
caries occurs in a population(8).
The author's use of innuendo to make a point is
contained in his reference to some very questionable work of Feltman which indicated
that the eruption of deciduous teeth might occur later in children exposed to
fluorides. The author implies that it is likely that retarded tooth eruption in
the children residing in the fluoride communities reduces their exposure to
caries attack, and thus their caries rate naturally is lower(9). Had Dr Sutton
been familiar with the literature, he would have known that this was one of the
first possibilities thought responsible for the low caries rate in children
exposed to fluoride-bearing water (10). He would have known about the rather
exhaustive report of Short on the relation between fluoride in domestic waters
and tooth eruption which showed that fluoride in concentrations around the
optimum used for caries control does not influence the eruption pattern of
permanent teeth (11).
However, one suspects that further analysis of the
details contained in the monograph will not yield much of value. From reading
the document and from hearing him present part of it as a paper last February
at your Adelaide Congress, I can only conclude that Dr Sutton has an intense
and emotional drive to oppose fluoridation. Why he feels this way is not clear,
but it seems likely to come from some motive other than a sincere concern for
the statistical or scientific validity of the concept(12).
Commentary on the Review by Dr Donald Galagan
Commentary on the Review by Dr Donald Galagan
(1) Dr Galagan makes the charge that the
"first and foremost among the fundamental errors" was made in the
second paragraph of the monograph (p. 136). This paragraph stated: "Apart
from these considerations, an examination reveals that there are aspects that
call for a very careful appraisal of the figures presented in the reports of
the experimental trials which have been conducted in Brantford, Canada, and in
Grand Rapids, Newburgh and Evanston, U.S.A., and upon the results of which
proposals to fluoridate domestic water are almost entirely based." Such
proposals are based on two different sets of results- those reported from areas
where fluorides occur naturally in the water supplies, and those from trials of
the mechanical addition of fluorides to waters in which the fluoride content is
very low or absent. The former reports are not considered in this
investigation, but Dr Galagan says that they were gathered in such a way that
they "are considered to be classic examples of good epidemiological
method". In any case, they are of practical importance only if it is known
that the results reported will be obtained when commercially available
fluorides are mechanically added to water supplies.
This knowledge can be obtained only by the use of
experimental trials the results of which must then be considered to be of
outstanding importance-unless it is accepted that it can be established, on
theoretical grounds, that the results of artificial fluoridation will be
identical with those seen in areas where fluorides are found naturally. If the
latter case is accepted, the early artificial schemes which are considered here
were held merely, as Dr Galagan suggests, "to demonstrate the effectiveness
of the procedure" and cannot be considered to be true experimental trials.
The question to be answered is this: Were these
trials mere demonstrations or were they set up as genuine experimental studies?
If they were only demonstrations Dr Galagan's charge is justified, but if these
trials were conducted to determine the outcome of the process of mechanical
fluoridation, then his accusation is without foundation and must be
discredited.
In 1951 a report was issued by the ad hoc Committee
on Fluoridation of Water Supplies of the National (U.S.A.) Academy of Sciences,
National Research Council (Maxcy, Appleton, Bibby, Dean, Harvey, Heyroth,
Johnson, Whittaker and Wolman, 1952). One of the members of this Committee was
Dr H. Trendley Dean, who was closely associated with much of the earlier work
concerning fluorides and was, at that time, the Director of the National
Institute of Dental Research. The report included this statement: "In
1945, studies were begun to ascertain whether the adjustment of the fluoride content
of a public water supply to the optimal level with commercially available
fluorides would confer the same caries-inhibitory effects as do waters which
carry the same concentrations of fluoride naturally."
This statement makes it clear that Dr Galagan's
contention, that these trials were only demonstrations, is not correct.
However, because of the importance of this matter and in case it is suggested
that that Committee was misinformed concerning the intention of these trials, a
quotation will be given from each of the five studies considered. Dr Galagan
calls the process used in these test cities "the controlled addition of
fluorides to a municipal water supply", but the original term used by Dean
et al. (1950) and by Brown (1951,
1952), "artificially fluoridated" drinking water, is preferable as it
is free from ambiguity.
(a) Brantford, City Health Department Study, Hutton
et al., (1951): "It was
recognized that fluorine in the public water supply was not a proven method for
the prevention of dental caries, and that it might take ten years to prove or
disprove its preventive value."
(b) Brantford, National Health and Welfare Study,
Brown et al., (1954b): "The
Brantford Fluoridation Caries Study was undertaken with a view to finding out
whether or not the raising of the fluoride content of a previously
fluoride-free water supply to part per million, by the mechanical addition of
sodium fluoride, would reduce the incidence of dental caries to that which
obtains where water supplies derive about 1 part per million of fluoride from
deposits in the earth"
(c) Grand Rapids, Dean et al. (1950): "In 1945, three studies to determine the caries
prophylactic value of artificially fluoridated drinking water were started in
the United States and Canada."
(d) Newburgh, Ast et al. (1950): "In 1943 it was proposed to determine whether
we can translate the conclusions derived from the epidemiological studies in
fluoride areas to a practical application in fluoride-free areas where the
communal water supplies may lend themselves to treatment."
(e) Evanston, Blayney and Tucker (1948):
"After further deliberation of the project, both professional groups
recommended to the Commissioner of Health that a carefully controlled study be
developed to determine whether or not the addition of fluorine in minute
quantities to the communal water supply would reduce the incidence of dental
caries in Evanston and Skokie children." Blayney and Tucker (1948) also
said: "It was carefully explained to these Evanston citizens that nothing
could be promised regarding the ultimate value in the control of tooth decay;
that if such a program was to be undertaken it must be in the nature of an
exhaustive study; and that it would be several years before data would be
available which would even indicate the trend which we might expect."
These statements, by the authors of all of the five
studies considered, establish beyond question that in every case the studies
were not designed "to demonstrate the effectiveness of the procedure"
but to determine whether or not artificial fluoridation would be efficacious.
Therefore Dr Galagan's opinion is incorrect, and the statement made in Part
One, which he termed the "first and foremost among the fundamental
errors", is a correct description of the situation.
(2) His suggestion that an attempt has been made to
"negate the whole '' fluoride-caries hypotheses" is without
foundation. The only reference to this matter is contained in the quotation of
a statement. made in 1949, by the American Water Works Association regarding
the experimental verification of "the fluoride-dental caries
hypothesis" - that is, to the "fluoridation hypothesis".
Statements made in the preface to the first edition and on the first page of
Part One and the first and last pages of Part Two show that consideration has
been given only to five experimental trials of artificial fluoridation produced
by mechanical means. The data from epidemiological studies in "naturally
fluoridated" areas, on which the fluorine-dental caries hypothesis is
based, have not been considered.
(5) Dr Galagan states that "The principal
point, however, is that these projects emphatically are not the sole basis on
which the widespread use (in the United States) of this procedure rests."
It can be seen, by reading the "free" quotation from paragraph two,
page 136, given by this reviewer (1) and pages 140, 189, 190 and 196, that the
word "sole" was not used nor implied.
No comment can be made on the other
"fundamental errors" which Dr Galagan says (1) are present-for he has
neglected to state their nature. Instead he continues:
(6) There are a good many other specific, but less
important errors in judgment which the author has made, such as his suggestion
that the variability of examiners in diagnosing dental caries has been
overlooked.
Comment. This suggestion was not made. As Dr Galagan points
out in his next sentence (7), "this variability is well known".
Therefore it is most unlikely that such a matter would be
"overlooked" in studies employing statisticians. Indeed, attention
was drawn, on page 180, to the fact that the importance of examiner variability
was recognized by Ast et al. in 1950.
However, in conducting the clinical examinations, in all the studies
considered, this matter was ignored. Therefore, when speaking of this
phenomenon, terms such as "not assessed" and "not
estimated" were used.
(7) The truth is that this variability is well
known, and is discussed at length by specialists in the fields of epidemiology
and caries diagnosis. Because of this "human error" calibration of
examiners is practised as a matter of course. Calibration reduces the
variability among examiners, but even if it did not, the difference between the
caries experience of children exposed and not exposed to fluoride is so great
that even Dr Sutton could recognise it.
Comment. As Dr Galagan says, this (examiner) variability is
well known. It is precisely this fact that makes it so surprising that this
factor was not assessed in these studies. It will be recalled that reference
was made, on page nine, to two papers which investigated the matter of examiner
variability in caries diagnosis.
The claim is made by the reviewer that
"Calibration reduces the variability among examiners", but he does
not suggest that this process eliminates between-examiner variability-therefore
it should have been taken into account. Of course, the use of the method of the
"calibration" of the subjective judgment of several examiners with
the subjective judgment of another is, to say the least, a poor substitute for
a standard rigorous statistical procedure.
(8) There are several other examples of errors of
judgment in the arguments contained in the monograph, indicating the author's
serious lack of understanding of the principles of statistics and epidemiology.
For instance, the rather amateurish interpretation of adequate community
"controls" for evaluating the effect of fluoride-bearing water on
caries indicates that the author does not really know the manner in which
caries occurs in a population.
Comment. The question may be asked: Who does?
(9) The author's use of innuendo to make a point is
contained in his reference to some very questionable work of Feldman (p. 192)
which indicated that the eruption of deciduous teeth might occur later in
children exposed to fluorides. The author implies that it is likely that
retarded tooth eruption in the children residing in the fluoride communities
reduces their exposure to caries attack, and thus their caries rate naturally
is lower.
Comment. On page 192, reference was not made to the work;
of Feldman but to that of Dr Reuben Feltman who was an associate of Doctors D.
E. Gardner and F. A. Smith, whose publications on fluorides are well known, and
of Doctors H. C. Hodge and D. E. Overton, who were closely associated with the
Newburgh trial (Gardner, Smith, Hodge, Overton and Feltman, 1952). Dr Feltman's
earlier (1951) work with fluoride tablets was referred to by the New Zealand
Commission of Inquiry (1957) as "promising". His 1956 paper, which
was quoted (p.192), was a brief "progress report" only. Therefore the
results mentioned in it were treated with reserve, the statement being made
that "Of course, if fluoridation results in the eruption rate of teeth
being retarded ....".
(10) Had Dr Sutton been familiar with the literature,
he would have known that this was one of the first possibilities thought
responsible for the low caries rate in children exposed to fluoride-bearing
water.
Comment. Dr Galagan has avoided the main point which was
discussed in this paragraph. His insistence that one of the first possibilities
considered was that the ingestion of fluoride-bearing water may retard tooth
eruption, makes it even more strange that in only the Newburgh-Kingston study
was mention made that this important matter had been investigated. Even in that
trial, the only study of tooth eruption rate published was conducted after four
years of fluoridation (Ast et al.,
1951) in children who were six to twelve years of age, so that none of the
subjects studied had been ingesting artificially fluoridated water throughout
their lives.
(11) He would have known about the rather
exhaustive report of Short on the relation between fluoride in domestic waters
and tooth eruption which showed that fluoride in concentrations around the
optimum used for caries control does not influence the eruption pattern of
permanent teeth.
Comment. It is highly probable that Dr Galagan is referring
to the paper by E. M. Short (1944) which is well known to those interested in
fluorides, for this is the only paper concerning fluorides and tooth eruption
listed under that name in the Index to Dental Literature in the English
Language (published by the American Dental Association), the Quarterly
Cumulative Index Medicus (to December 1956), or the issues of the Current List
of Medical Literature which cover the subsequent period.
This "rather exhaustive report of Short"
does not show whether or not the ingestion of fluorides at the
"optimum" level has any influence on the eruption pattern of
permanent teeth. This report was made on "selected 12 14 year old white
school children" (Short, 1944) in whom almost all the permanent teeth had
erupted. The data deal only with the total number of erupted permanent teeth
and, despite Dr Galagan's remark, do not give any information regarding their
eruption pattern-a factor which could be of considerable importance in the
development of the DMF rate. Short's Tables II and III show that, in all except
three of the 4,453 children examined, at least twelve of the permanent teeth had
erupted, the minimum number of erupted permanent teeth, for the remaining three
children, being ten. His Table I (which excluded third molars) shows that, out
of the possible twenty-eight teeth, the mean number of erupted permanent teeth
per child in the various cities was between 25.22 and 26.81. In fact, in
fifty-five per cent of the children all of the twenty-eight teeth had erupted.
Therefore, this study of Short (1944) gives no
information regarding the ages at which the first ten permanent teeth erupted
in these children, certainly none regarding the first permanent molars which,
presumably, even in the youngest of these children, erupted about five or six
years prior to the study. These molars are of outstanding importance in regard
to the DMF rate, particularly in young children. Ast et al. (1956) said: "The first permanent molars are frequently
used as an index of caries experience among children because this tooth
accounts for the major incidence of caries in this group."
It should be noted that neither Dr Galagan nor (as
will be seen in their review of this book) Doctors Blayney and Hill, authors of
the Evanston study, have commented on the suggestion of a decline in the
eruption rate of first permanent molars in Evanston, between 1946 and 1951, which
followed the introduction of fluoridation and which is depicted in Figure 4 (p.
139). Neither have they explained why, after 1951, whilst continuing to publish
this type of data for the older children, they ceased publishing it for the
younger
(12) However, one suspects that further analysis of
the details contained in the monograph will not yield much of value. From
reading the document and from hearing him present part of it as a paper last
February at your Adelaide Congress, I can only conclude that Dr Sutton has an
intense and emotional drive to oppose fluoridation. Why he feels this way is
not clear, but it seems likely to come from some motive other than a sincere
concern for the statistical or scientific validity of the concept.
Comment. Dr Galagan's questioning of the motive behind this
study should be considered in relation to statements which he made during his
lecture to the Adelaide Congress (Galagan, 1959). He called those who
questioned fluoridation "the opposition" and said that this group
"seems to be composed of four distinct kinds of people." These he
termed: "the hatemonger, the pseudo-health believer, and the person who
opposes fluoridation for personal notoriety" and "the fourth, or
rugged individualist, group". As these are Dr Galagan's views, it is not
surprising that he doubts the sincerity of this attempt to investigate
"the statistical or scientific validity" of these fluoridation
findings.
DR J. R. BLAYNEY and DR I. N. HILL
DR J. R. BLAYNEY and DR I. N. HILL
The second review of this book in the Australian
Dental Journal, by DR J. R. BLAYNEY, Director, Dental Caries Study, Evanston,
and DR I. N. HILL, Zoller Memorial Dental Clinic, University of Chicago, was as
follows:
Dr Sutton has much to say regarding the lack of
comparability(13) of the study and the control areas and the manner of
selection of the children in each area to be examined. Oak Park is the suburb
immediately to the west of Chicago and Evanston lies immediately to the north.
Each community draws its water supply from Lake Michigan. Standard analyses for
composition of tlw water are frequently run. A spectro-chemical analysis for 26
trace elements has been run on Lake Michigan water for a comparison of a
similar analysis of water obtained from fluoride areas (unpublished data). Oak
Park and Evanston receive their food from the same wholesale markets, each is
chiefly residential and free from heavy industry. Each is composed of the same
socio-economic level, as borne out by the United States census of population
for 1950. In Evanston, 47,395 persons 21 years or over were native born. In Oak
Park the figure was 42,454. In Evanston there were 6,049 foreign born persons
21 years or older while in Oak Park the figure was 5,081. In Evanston 41.2 per
cent of the occupied dwelling units were owner occupied. In Oak Park 50.8 per
cent were occupied by the owner. The median value of a one dwelling unit
structure in Evenston was 19,499 dollars and in Oak Park the value was 16,259
dollars. The median value of gross monthly rentals in renter occupied dwelling
units in Evanston was 72.53 dollars while in Oak Park the median value was
66.86 dollars. Both areas have comparable, climatic conditions and both are
subjected to the same radio and television commercials regarding oral hygiene
and dentifrices. Finally, the majority of the dental practitioners in the study
and control areas are graduates of one of the three Chicago dental colleges.
For the baseline examination in Evanston we were
committed to examine all school
children within the selected age range regardless of the length of time they
had resided in Evanston. Although an effort was made to include them, the Oak
Park Parochial Schools did not find it convenient for us to examine their
pupils(14). When we compared the caries prevalence rates of the two towns we
found a difference. It is indeed fortunate that our records showed the school
that each child attended. In this manner we could first eliminate the Parochial
School group and then the Negro group from the total Evanston data. It is well
known that coloured people have less dental caries than whites living in the
same population centres(15). The caries rates for the Evanston and Oak Park
Public School white children compared favourably(16). All of this clearly
indicates how hidden variables may exist in areas which otherwise appear to be
comparable, and how important it is to be certain that comparisons are made
between like groups(17).
Dr Sutton expressed astonishment that in 1955 the
six and seven-year-old Evanston children had a lower caries prevalence rate
than the Aurora children of like ages. We, likewise, did not anticipate this.
However, the same critical evaluation both clinical and roentgenological, was
made of every case. This difference was due to something other than fluoride.
Possibly the presence of the dental team in the school, year after year, has
stimulated the classroom teachers and the school nurses to place more emphasis
on the teaching of oral health. Some unknown hidden variation not related to
fluorides must account for the difference(18).
Dr Sutton was concerned that the control group was
not examined annually. Neither we nor our advisers could see a reason to
require an examination of the control group other than at the beginning and
near the close of the study. This provides the baseline from which to measure
the trend of the dental caries rate during the time interval (1947-1956).
Should the rate in the last examination (1956) deviate materially from that of
the initial baseline period (1947) that figure could be used as a correction
factor on the Evanston findings. In fact, if we had not desired to measure the
yearly decrement in the rate of dental caries under fluoridation and evaluate
other factors only two examinations in Evanston would have been necessary, the
first in 1946 (before fluoridation), the second and final in 1961(19).
Much has been made of the variations reported in
the number of children examined. The baseline examination of 4,375 Evanston
children and of 2,493 Oak Park children are correct. However, the data from
those children who had not used Lake Michigan water all of their lives had to
be excluded. It was also observed that some children below the age of 67 months
and above the maximum of 174 months had been examined. Therefore these out of
range children were not considered in the final determination of the caries
rates. This explains the discrepancy between the Evanston 4,375 and 3,692 and
the Oak Park 2,493 - 2,051 figures(20). It should be noted that when dental
caries experience rates were compared, the same number of examinations, that is
1,991 for the six to eight-year-old children and 1,701 for the 12 to
14-year-old children, were used throughout the reports for the combined
Evanston school groups (Public White, Parochial and Public Negro)(21). Dr
Sutton on p. 167 of his report (fig. 5, Statement C) calls attention to
Evanston Dental Caries Study Report Number XVII. Here he points out that only
1,754 six to eight-year-old children and 1,556 12 to 14-yearold children were
listed. This particular report was primarily concerned with differences in sex
and race, as they influenced caries, rather than the effect of fluorides.
Therefore a comparison was made of the dental caries experience rates of white
girls to white boys; Negro girls to Negro boys; white girls to Negro girls;
white boys to Negro boys; and white children as a group to Negro children as a
group. The children were classified into male white, female white, male Negro
and female Negro for comparison(22). As Dr Sutton calls attention to the
difference in the number of examinations made in this report when compared with
other reports, we wish to point out, in explanation, that it was necessary to
not only insure that children of correct age be included but also it was
necessary that every examination be clearly classified according to race and
sex There were 236 six to eight-year-old children and 245 12 to 14-year-old
children excluded from this report because they did not fulfil the requirements
for this comparison. These children were included in other reports as no
distinction except age was made(23). It is also pointed out that this report,
Evanston No. XVII as noted above, was primarily concerned with comparison of
the caries rates of the coloured and white children and not with the effect of
fluoridation on dental caries rates. Therefore in this light, this report, No.
XVII, should not have been listed under the general heading of fluoridation as
Dr Sutton has it listed in his critique(24).
It is true that a discrepancy in figures published
in our paper XVI, Table I and in paper XVIII, Table I are at variance. This is
due to the operator of the tabulating machine providing the wrong figures for
the number of seven and eight-year-old children examined. This error was
discovered after manuscript XVI was in press and therefore the corrections
could only be made manually in the reprints supplied to readers who requested
them(25).
Commentary
on the Review by Dr J. R. Blaynex and Dr L N. Hill
(13) Reference to Part Two will show that no
suggestion was made in it that there was a "lack of comparability"
between Evanston and Oak Park; it was merely pointed out that the manner in
which Oak Park resembled Evanston was not stated.
Therefore this detailed exposition of the
similarity of the two cities is welcome, although it is unfortunate that, when
speaking of rental and dwelling values, the "mean" values were not
given as well as the "median" ones. It will be realized that the
housing picture may be very different in two towns and yet the
"median" (that is, the middle) values of the rentals and dwellings
can be the same. In view of the data shown here, regarding housing and the
training of the dentists, it is surprising that the members of the United
Kingdom Mission (1953) should have singled out Evanston for comment, remarking
on its high economic level and its "outstandingly good" dental care
(p. 149).
These data also show how reasonable was the
assumption of Dr Blayney before assessing the caries rates in Oak Park-that
that city was "the ideal control community" for Evanston (Blayney and
Tucker, 1948;). They also emphasize how strange it is that such gross
differences should be found between the initial caries rates of the children
aged six to eight years in the two cities, and reported-after a delay of ten
years-by these workers (p. 153; Fig. 3, p. 154). The fact that such gross
differences can be found in the caries rates prevalent in two cities which were
so similar that one was termed "the ideal control community" for the
other (Blayney and Tucker, 1948), confirms the necessity for pre-fluoridation
examinations in both test and control cities. Unfortunately, as pointed out in
Part Two this was not done in any of these studies.
(14) For the baseline examination in Evanston we
were committed to examine all school children within the selected age range,
regardless of the length of time they had resided in Evanston. Although an
effort was made to include them the Oak Park Parochial Schools did not ford it
convenient for us to examine their pupils.
Comment. The latter remark is welcome for it explains the
absence of data from the parochial schools in Oak Park.
(15) When we compared the caries prevalence rates
of the two towns we found a difference. It is indeed fortunate that our records
showed the school that each child attended. In this manner we could first
eliminate the Parochial School group and then the Negro group from the total
Evanston data. It is well known that coloured people have less dental caries
than whites living in the same population centres.
Comment. Despite their statement that "It is well
knowp that coloured people have less dental caries than whites living in the
same population centres", when conducting the initial examination in
Evanston the authors combined the data of the Negro children with those of the
white children. It is now clear, as deduced on page eighteen, that the racial
and school groups were taken into account only after it was found that there
was "a lower caries rate for school children of the control area" (Hill
et al., 1951). Thereafter, when
comparing the test and control cities, the data of both the Negro and the
parochial school children were excluded from the Evanston data. No reason has
been given for this exclusion of the data of parochial school children in Evanston-who
had a high caries rate (Hill et al.)
- from the data of the main body of white children in that city. This could not
be attributed to the fording of a similarly high caries rate in the parochial
school children in the control city of Oak Park for they were not examined and,
therefore, their caries rates were unknown (see 14).
Neither has an explanation been offered for the
extraordinary reversal of this policy (the exclusion of the data of Negro and
parochial school children in Evanston) when compiling the XVIII Report (Hill et al., 1958). This report published,
for the first time, the initial caries rates for the permanent teeth of the
children aged six to eight years which were obtained, ten years earlier, in the
control city of Oak Park.
The rates for the deciduous teeth, which were
obtained at that time, still have not been published. This report provided the
first opportunity to compare the initial caries rates of the younger children
in the test city and its control (p. 153).
(16) The caries rates for the Evanston and Oak Park
Public School white children compared favourably.
Comment. This statement is interesting-for the caries rates
for children aged six to eight years in each of the three school groups have
not been published (p.151). It will be recalled that the mean caries rates for
the six, seven, and eight-year-old children in Evanston in 1946 were very much
higher than the mean rates for children of those ages obtained during the
initial examination in Oak Park. For the children aged six years the rate in
Evanston was 46.85, but it was only 26.89 in Oak Park (Hill et al., 1958). Only 0.1 per cent of the
Oak Park children were Negro (Hill et al.,
1951), but exclusion of the data of Negro children (who have a relatively low
caries rate, see 15) from the Evanston data would increase the rate of the
remaining (white public and parochial school) children so that in the
six-year-old children, it would be higher than 46.85. Therefore the difference
between this rate and the Oak Park rate of 26.89, for children of that age,
would be increased.
Hill et al.
did not say how many of the younger age group of children attended each type of
school, but only twenty-two per cent of the twelve to fourteen-year-old
children, shown in their 1957a report as examined in Evanston in 1946, attended
parochial schools. Therefore, the proportion of children aged six, seven, and
eight years who were attending parochial schools in Evanston in 1946, and their
caries rate, must have been very high to permit Doctors Blayney and Hill to
state that "The caries rates for the Evanston and Oak Park Public School
white children compared favourably." Of course speculation is no
substitute for data-and this still has not been published
(17) All of this clearly indicates how hidden variables
may exist in areas which otherwise appear to be comparable, and how important
it is to be certain that comparisons are made between like groups.
Comment. The latter phrase is a reiteration of remarks made
in the 1957a report from this study, that it is necessary "to make
comparisons of like groups." Why then, having realized this necessity, did
Hill et al. ignore it in their 1958
report (p. 152)? In this report the data shown, for the year 1946, combined not
only that of the white children attending both public and parochial schools,
but the data of the Negro children as well. The resultant rate was then
compared with that of children in Oak Park comprising, almost entirely, white
children attending public schools. By ignoring the opinion they expressed in
the previous year (1957a)-which they now reiterate-and comparing
"unlike" groups of children, a more favourable degree of
comparability was obtained between the initial caries rates of children in the
test and the control cities.
(18) Dr Sutton expressed astonishment that in 1955
the six and seven-year-old Evanston children had a lower caries prevalence rate
than the Aurora children of like ages. We, likewise, did not anticipate this.
However, the same critical evaluation both clinical and roentgenological, was
made of every case. This difference was due to something other than fluoride.
Possibly the presence of the dental team in the school, year after year, has
stimulated the classroom teachers and the school nurses to place more emphasis
on the teaching of oral health. Some unknown hidden variation not related to
fluorides must account for the difference.
Comment. It is pleasing that Doctors Blayney and Hill
should support the view expressed on page 187 (para. 4) that regular dental
examinations may stimulate interest in the teeth and thus lead to improved oral
health. In advancing the suggestion that "something other than
fluoride" can affect the caries rates, they recognize the great importance
which factors other than the fluoride concentration of the water supply may
have on the caries rates. This extremely important matter was practically
ignored by the authors of all these studies when preparing their reports.
(19) Dr Sutton was concerned that the control group
was not examined annually. Neither we nor our advisers could see a reason to
require an examination of the control group other than at the beginning and
near the close of the study. This provides the baseline from which to measure
the trend of the dental caries rate during the time interval (1947 1956).
Should the rate in the last examination (1956) deviate materially from that of
the initial baseline period (1947) that figure could be used as a correction
factor in the Evanston findings. In fact, if we had not desired to measure the
yearly decrement in the rate of dental caries under fluoridation and evaluate
other factors, only two examinations in Evanston would have been necessary, the
first in 1946 (before fluoridation), the second and final in 1961.
Comment. This statement makes two things clear. The first
is that, at the commencement of the study, neither the workers nor their
advisers could have considered the possibility, which they now acknowledge(18),
that "the presence of the dental team in the school, year after year"
might have had a stimulating effect "on the teaching of oral health."
It is obvious that, if this effect is possible, not only the test town but also
its control should have been examined "year after year". The other
point which is indicated by this statement(19) of Doctors Blayney and Hill is
that despite their remark in 1950, the importance of random variation was not
and, seemingly still is not recognized.
(20) Much has been made of the variations reported
in the number of children examined. The baseline examination of 4,375 Evanston
children and of 2,493 Oak Park children are correct. However, the data from
those children who had not used Lake Michigan water all of their lives had to
be excluded. It was also observed that some children below the age of 67 months
and above the maximum of 174 months had been examined. Therefore these out of
range children were not considered in the final determination of the caries
rates. This explains the discrepancy between the Evanston 4,375 and 3,692 and
the Oak Park 2,493 - 2,051 figures.
Comment. This explanation of the difference between these
sample sizes in Evanston and Oak Park is welcome. It might have been deduced if
the decision to exclude "the data from those children who had not used
Lake Michigan water all of their lives" had been announced in one of the
five reports giving caries rates, issued prior to 1955 (p. 163).
In the XIX Report (Hill et al., 1959), the sample sizes shown for the two age groups in Oak
Park in 1947 (1,022 and 1,032) are almost the same as those shown (1,020 and
1,031) in statement "E" of Figure 5. Comparison with the statements
for Evanston cannot be made for this (XIX) report considered only "public
school white children".
(21) It should be noted that when dental caries
experience rates were compared, the same number of examinations, that is 1,991
for the six to eight-year-old children and 1,701 for the 12 to 14-year-old
children were used throughout the reports for the combined Evanston school
groups (Public White Parochial and Public Negro).
Comment. It is surprising that the suggestion was made that
this statement should be noted-for it is not correct. The number of twelve,
thirteen and fourteen-year-old children examined in Evanston in 1946 was given
in Tables III, V, VI, VII, VIII, IX and X of the XV Report (Hill et al., 1957a) as 418, 688 and 595, a
total of 1,701. However, in Tables XI and XII of the same paper different
sample sizes for these ages were shown: 414, 692 and 617, a total of 1,723. The
same discrepancies were noted between different tables in the XI Report (Hill et al., 1955) Therefore the figure 1,701
was not "used throughout the reports for the combined Evanston school
groups".
(22) Dr Sutton on p. 167 of his report (fig. 5,
Statement C) calls attention to Evanston Dental Caries Study Report Number
XVII. Here he points out that only 1,754 six to eight-year old children and
1,556 12 to 14-yearold children were listed. This particular report was
primarily concerned with differences in sex and race, as they influenced
caries, rather than the effect of fluorides. Therefore a comparison was made of
the dental caries experience rates of white girls to white boys; Negro girls to
Negro boys; white girls to Negro girls; white boys to Negro boys; and white
children as a group to Negro children as a group. The children were classified
into male white, female white, male Negro and female Negro for comparison.
Comment. A curious feature of this XVII Report is that
although care was taken in regard to the age, race, and sex of the subjects, no
attempt was made to "limit the examinations to continuous resident
children" (Hill et al., 1957b).
Thus it is reasonable to assume that some children were examined who were not
"continuous" residents. Therefore the comparisons mentioned by
Doctors Blayney and Hill were made on mixed samples of children some of whom
had not ingested fluoridated water, those examined in 1946, and others who had
done so for varying periods of up to about eight years. This disregard of the
possible effect of the ingestion of fluorides on the caries rates, of some of
the children examined, is inexplicable.
(23) As Dr Sutton calls attention to the difference
in the number of examinations made in this report when compared with other
reports, we wish to point out, in explanation, that it was necessary to not
only insure that children of correct age be included but also it was necessary
that every examination be clearly classified according to race and sex. There
were 236 six to eight-year-old children and 245 12 to 14-year old children
excluded from this report because they did not fulfil the requirements for this
comparison. These children were included in other reports as no distinction
except age was made.
Comment. This explanation why the sample sizes from
Evanston shown in the XVII Report (1,754 six to eight-year-old children and
1,556 twelve to fourteen-year-old children) do not agree with those shown in
other reports at first appears to be a reasonable one. However, before it is accepted,
consideration should be given to two observations. Firstly, if the figures
depicted in statements "B" and "C" of Figure 5 which were
originally given by Hill et al. and
are now confirmed by Doctors Blayney and Hill (20, 22), are accepted as
correct, the numbers of children excluded in the two age groups (1,991-1,754
and 1,701-1,556) were 237 and 145, not 236 and 245 as stated by Doctors Blayney
and Hill. It is possible that these errors could have arisen in typing the
manuscript, but this could not be the case in regard to the second observation.
This second observation is as follows: In the XVII
Report (Hill et al., 1957b) it was
stated that "in this report no attempt has been made to limit the
examinations to continuous resident children." Thus it is almost certain
that data from both "continuous" and "non-continuous"
resident children are included in the total of 3,310 subjects mentioned in that
Report as examined in Evanston in 1946. Therefore, to determine the number of
children who were excluded from the XVII Report, comparison must be made, not
with the number of "continuous" residents of correct age that were
examined in 1946 (3,692, see 20) and "were included in other
reports", but with the total number of children ("continuous"
and "non-continuous" residents and "out of range") that
were examined in 1946, that is, 4,375.
If this is done, it can be seen that 1,065 of these
children (4,375-3,310) were excluded from the XVII Report. This figure includes
some "out of range" children, for it was stated that "some
children below the age of 67 months and above the maximum of 174 months had
been examined" (20) in the baseline examination of 4,375 Evanston
children. Nevertheless, unless there were as many as 584 "out of
range" children (1,065 = 584 + 481), the actual number of children
excluded, because they were not of correct age or could not "be clearly
classified according to race and sex", must have been larger than the
figure of 481 given here (236 + 245) by Doctors Blayney and Hill.
(24) It is also pointed out that this report,
Evanston No. XVII as noted above, was primarily concerned with comparison of
the caries rates of the coloured and white children and not with the effect of
fluoridation on dental caries rates. Therefore in this light, this report, No.
XVII, should not have been listed under the general heading of fluoridation as
Dr Sutton has it listed in his critique.
Comment. If the errors contained in the XVI Report had been
pointed out by providing an additional footnote to Table I in the XVIII Report
(which was the next report in which this type of data was published) the reason
for the difference between the two sets of figures would have been obvious. It
should be noted that, although the source of the errors in the 1955 rates in
Table 1, XVI Report was given, no mention has been made of the fact that, in
the same table, there are errors in computing the rates for the six to
eightyear-old age group in the years 1946 and 1948.Both of these errors were of
long standing as they were shown, four years earlier, in the X Report (Hill et al., 1952). These errors were still
contained in the XV Report (Hill et al.,
1957a).
It can be seen that Doctors Blayney and Hill
devoted a considerable part of their review to two matters. The first was the
"comparability of the study and the control areas"-which was not
questioned (see comment 13). The second was a lengthy description of the
comparisons which they made between different groups of children in obtaining
the data for the XVII Report. This information was given in almost the same
words in that report, which, they stated (24), was not primarily concerned with
the effect of fluoridation on dental caries rates.
However, they have not mentioned most of the
matters which do directly concern fluoridation and caries prevalence and which
were questioned. In fact their comments have touched on matters mentioned in
only about a third of the sub-headings used in considering their study. Their
meagre explanations have accounted for the presence of some of the errors in
one table .... and have supplied a reason for the differences between the
sample sizes for the year 1947 in Oak Park, and for the disparity between two
of the three sample sizes for 1946 in Evanston (p 165) However, most of the
matters mentioned in considering the Evanston study were ignored, even those
illustrated by Figures 3 and 4.
It should be noted, therefore, that Doctors Blayney
and Hill have not commented on the majority of the errors, omissions and
mis-statements mentioned in considering the Evanston study, and almost all of
them remain unexplained.
No comments:
Post a Comment