Errors
and Omissions in Experimental Trials - 2a
PART
THREE
CRITICISMS
AND COMMENTS
Soon after the publication of the first
edition of this monograph in September 1959, the author was informed by Dr K.
T. Adamson, President of the Australian Dental Association, that copies had
been sent to "all of the men who are in charge of the experiments"
asking them for comments (personal communication). As a result the reviews
published in the February 1960 issue of the Australian Dental Journal were
contributed by Dr Donald Galagan, Dr J. R. Blayney and Dr I. N. Hill, and by Dr
R. M. Grainger. The review in the New Zealand Dental Journal of January 1960
was written by Mr J. Ferris Fuller.
The aim of this monograph is to attempt
to clarify some aspects of five crucial trials of artificial fluoridation,
those conducted in Grand Rapids, Evanston, and Newburgh, U.S.A., and the two
trials held in Brantford, Canada. Therefore, in this part, all these critical
reviews will be quoted in full; and comments will be made on the points raised
and indicated by the figures in brackets.
In order to allow the reader to
appraise the criticisms more easily it will be necessary to refer to the
statements made in the text, therefore page references are indicated for Parts
One and Two of this volume.
DR DONALD GALAGAN
DR DONALD GALAGAN
The first review of this book in the
Australian Dental Journal, by Dr Donald Galagan, Assistant Chief, Division of
Dental Public Health, Public Health Service, Washington 25, D.C., was as
follows:
My comments will be limited to the general
qualities of Dr Sutton's treatise and the conclusions he has reached.
Individuals associated with the several fluoridation projects which he has
purported to "analyse" will have provided their specific reactions to
his "analysis" of their findings.
Although it is nothing new to see an
accredited scientist mix fact and fancy, near truth with truth, and emotion
with reason it is always shocking to realise that an intelligent individual in
a responsible position can so baldly misinterpret scientific data. Actually, it
would be extremely difficult for an objective scientist who knows anything at
all about the data characterizing the relationship between dental caries and
exposure to fluorides to reach the conclusion that the effectiveness of
controlled water fluoridation in reducing dental caries has not been proved.
This conclusion could only be reached by an armchair statistician who has
chosen to ignore or does not know the great mass of information on the subject.
Thus, first and foremost among the fundamental
errors(1) which Dr Sutton makes is expressed in the statement in the second
paragraph of the monograph which says that "proposals to fluoridate
domestic water are almost entirely based on the results of the Brantford, Grand
Rapids, Newburgh, and Evanston projects". Using this premise, the balance
of the document is devoted to efforts to describe "errors and
omissions" in these four projects. These errors are supposed to negate the
whole fluoride-caries hypotheses(2), or at least to throw serious doubt on the
fluoridation of water as a caries preventive.
Actually, the scientific basis on which
this public health measure rests was established solidly before any of the
above mentioned projects were started. The preventive effect of long term
exposure to water-borne fluoride on caries experience was observed in literally
thousands of children residing in many different communities where the water
consumed had picked up the element as it coursed over or through the earth's
crust. The long series of investigations documenting the relationship are
considered to be classic examples of good epidemiological method(3) - so much
so that they are used as case studies in teaching the science of epidemiology
in our schools of public health.
The fact is that the projects at
Brantford, Grand Rapids, Newburgh and Evanston were designed primarily to
evaluate the technical, financial and administrative problems associated with
the controlled addition of fluorides to a municipal water supply, and,
secondarily, to demonstrate the effectiveness of the procedure(4) to the
profession and the public. To be sure, it was necessary to show that the
procedure would reduce the caries attack rate in resident children, but as soon
as the trend toward reduction was observed and corroborated, the
"experimental" portions of the projects were completed for all
practical purposes. The principal point, however, is that these projects
emphatically are not the sole basis on which the widespread use (in the United
States) of this procedure rests(5).
In short, it is preposterous to attempt
to conclude that the basis for community water fluoridation is faulty because
of some real or imaginary defects in the planning, execution and evaluation of
data from the four community projects. The contents of the monograph,
therefore, represent no more than an exercise in semantic and scientific
dilettantism designed to serve some other purpose.
There are a good many other specific,
but less important errors in judgment which the author has made, such as his suggestion
that the variability of examiners in diagnosing dental caries has been
overlooked(6). The truth is that this variability is well known, and is
discussed at length by specialists in the fields of epidemiology and caries
diagnosis. Because of this "human error" calibration of examiners is
practised as a matter of course. Calibration reduces the variability among
examiners, but even if it did not, the difference between the caries experience
of children exposed and not exposed to fluoride is so great that even Dr Sutton
could recognise it(7).
There are several other examples of
errors of judgment in the arguments contained in the monograph, indicating the
author's serious lack of understanding of the principles of statistics and epidemiology.
For instance, the rather amateurish interpretation of adequate community
"controls" for evaluating the effect of fluoride-bearing water on
caries indicates that the author does not really know the manner in which
caries occurs in a population(8).
The author's use of innuendo to make a
point is contained in his reference to some very questionable work of Feltman
which indicated that the eruption of deciduous teeth might occur later in
children exposed to fluorides. The author implies that it is likely that
retarded tooth eruption in the children residing in the fluoride communities
reduces their exposure to caries attack, and thus their caries rate naturally
is lower(9). Had Dr Sutton been familiar with the literature, he would have
known that this was one of the first possibilities thought responsible for the
low caries rate in children exposed to fluoride-bearing water (10). He would
have known about the rather exhaustive report of Short on the relation between
fluoride in domestic waters and tooth eruption which showed that fluoride in
concentrations around the optimum used for caries control does not influence
the eruption pattern of permanent teeth (11).
However, one suspects that further
analysis of the details contained in the monograph will not yield much of
value. From reading the document and from hearing him present part of it as a
paper last February at your Adelaide Congress, I can only conclude that Dr
Sutton has an intense and emotional drive to oppose fluoridation. Why he feels
this way is not clear, but it seems likely to come from some motive other than
a sincere concern for the statistical or scientific validity of the
concept(12).
Commentary on the Review by Dr Donald Galagan
Commentary on the Review by Dr Donald Galagan
(1) Dr Galagan makes the charge that
the "first and foremost among the fundamental errors" was made in the
second paragraph of the monograph (p. 136). This paragraph stated: "Apart
from these considerations, an examination reveals that there are aspects that
call for a very careful appraisal of the figures presented in the reports of
the experimental trials which have been conducted in Brantford, Canada, and in
Grand Rapids, Newburgh and Evanston, U.S.A., and upon the results of which
proposals to fluoridate domestic water are almost entirely based." Such
proposals are based on two different sets of results- those reported from areas
where fluorides occur naturally in the water supplies, and those from trials of
the mechanical addition of fluorides to waters in which the fluoride content is
very low or absent. The former reports are not considered in this
investigation, but Dr Galagan says that they were gathered in such a way that
they "are considered to be classic examples of good epidemiological
method". In any case, they are of practical importance only if it is known
that the results reported will be obtained when commercially available
fluorides are mechanically added to water supplies.
This knowledge can be obtained only by
the use of experimental trials the results of which must then be considered to
be of outstanding importance-unless it is accepted that it can be established,
on theoretical grounds, that the results of artificial fluoridation will be
identical with those seen in areas where fluorides are found naturally. If the
latter case is accepted, the early artificial schemes which are considered here
were held merely, as Dr Galagan suggests, "to demonstrate the
effectiveness of the procedure" and cannot be considered to be true
experimental trials.
The question to be answered is this:
Were these trials mere demonstrations or were they set up as genuine
experimental studies? If they were only demonstrations Dr Galagan's charge is
justified, but if these trials were conducted to determine the outcome of the
process of mechanical fluoridation, then his accusation is without foundation
and must be discredited.
In 1951 a report was issued by the ad
hoc Committee on Fluoridation of Water Supplies of the National (U.S.A.)
Academy of Sciences, National Research Council (Maxcy, Appleton, Bibby, Dean,
Harvey, Heyroth, Johnson, Whittaker and Wolman, 1952). One of the members of
this Committee was Dr H. Trendley Dean, who was closely associated with much of
the earlier work concerning fluorides and was, at that time, the Director of
the National Institute of Dental Research. The report included this statement:
"In 1945, studies were begun to ascertain whether the adjustment of the
fluoride content of a public water supply to the optimal level with
commercially available fluorides would confer the same caries-inhibitory
effects as do waters which carry the same concentrations of fluoride
naturally."
This statement makes it clear that Dr
Galagan's contention, that these trials were only demonstrations, is not
correct. However, because of the importance of this matter and in case it is
suggested that that Committee was misinformed concerning the intention of these
trials, a quotation will be given from each of the five studies considered. Dr
Galagan calls the process used in these test cities "the controlled addition
of fluorides to a municipal water supply", but the original term used by
Dean et al. (1950) and by Brown
(1951, 1952), "artificially fluoridated" drinking water, is
preferable as it is free from ambiguity.
(a) Brantford, City Health Department
Study, Hutton et al., (1951):
"It was recognized that fluorine in the public water supply was not a
proven method for the prevention of dental caries, and that it might take ten
years to prove or disprove its preventive value."
(b) Brantford, National Health and
Welfare Study, Brown et al., (1954b):
"The Brantford Fluoridation Caries Study was undertaken with a view to
finding out whether or not the raising of the fluoride content of a previously
fluoride-free water supply to part per million, by the mechanical addition of
sodium fluoride, would reduce the incidence of dental caries to that which
obtains where water supplies derive about 1 part per million of fluoride from
deposits in the earth"
(c) Grand Rapids, Dean et al. (1950): "In 1945, three
studies to determine the caries prophylactic value of artificially fluoridated
drinking water were started in the United States and Canada."
(d) Newburgh, Ast et al. (1950): "In 1943 it was proposed to determine whether
we can translate the conclusions derived from the epidemiological studies in
fluoride areas to a practical application in fluoride-free areas where the
communal water supplies may lend themselves to treatment."
(e) Evanston, Blayney and Tucker
(1948): "After further deliberation of the project, both professional
groups recommended to the Commissioner of Health that a carefully controlled
study be developed to determine whether or not the addition of fluorine in
minute quantities to the communal water supply would reduce the incidence of
dental caries in Evanston and Skokie children." Blayney and Tucker (1948)
also said: "It was carefully explained to these Evanston citizens that
nothing could be promised regarding the ultimate value in the control of tooth
decay; that if such a program was to be undertaken it must be in the nature of
an exhaustive study; and that it would be several years before data would be
available which would even indicate the trend which we might expect."
These statements, by the authors of all
of the five studies considered, establish beyond question that in every case
the studies were not designed "to demonstrate the effectiveness of the
procedure" but to determine whether or not artificial fluoridation would
be efficacious. Therefore Dr Galagan's opinion is incorrect, and the statement
made in Part One, which he termed the "first and foremost among the
fundamental errors", is a correct description of the situation.
(2) His suggestion that an attempt has
been made to "negate the whole '' fluoride-caries hypotheses" is
without foundation. The only reference to this matter is contained in the
quotation of a statement. made in 1949, by the American Water Works Association
regarding the experimental verification of "the fluoride-dental caries
hypothesis" - that is, to the "fluoridation hypothesis".
Statements made in the preface to the first edition and on the first page of
Part One and the first and last pages of Part Two show that consideration has
been given only to five experimental trials of artificial fluoridation produced
by mechanical means. The data from epidemiological studies in "naturally
fluoridated" areas, on which the fluorine-dental caries hypothesis is
based, have not been considered.
(5) Dr Galagan states that "The
principal point, however, is that these projects emphatically are not the sole
basis on which the widespread use (in the United States) of this procedure
rests." It can be seen, by reading the "free" quotation from
paragraph two, page 136, given by this reviewer (1) and pages 140, 189, 190 and
196, that the word "sole" was not used nor implied.
No comment can be made on the other
"fundamental errors" which Dr Galagan says (1) are present-for he has
neglected to state their nature. Instead he continues:
(6) There are a good many other
specific, but less important errors in judgment which the author has made, such
as his suggestion that the variability of examiners in diagnosing dental caries
has been overlooked.
Comment.
This suggestion was not made. As Dr Galagan points out in his next sentence
(7), "this variability is well known". Therefore it is most unlikely
that such a matter would be "overlooked" in studies employing
statisticians. Indeed, attention was drawn, on page 180, to the fact that the
importance of examiner variability was recognized by Ast et al. in 1950. However, in conducting the clinical examinations,
in all the studies considered, this matter was ignored. Therefore, when
speaking of this phenomenon, terms such as "not assessed" and
"not estimated" were used.
(7) The truth is that this variability
is well known, and is discussed at length by specialists in the fields of
epidemiology and caries diagnosis. Because of this "human error"
calibration of examiners is practised as a matter of course. Calibration
reduces the variability among examiners, but even if it did not, the difference
between the caries experience of children exposed and not exposed to fluoride
is so great that even Dr Sutton could recognise it.
Comment.
As Dr Galagan says, this (examiner) variability is well known. It is precisely
this fact that makes it so surprising that this factor was not assessed in
these studies. It will be recalled that reference was made, on page nine, to
two papers which investigated the matter of examiner variability in caries
diagnosis.
The claim is made by the reviewer that
"Calibration reduces the variability among examiners", but he does
not suggest that this process eliminates between-examiner variability-therefore
it should have been taken into account. Of course, the use of the method of the
"calibration" of the subjective judgment of several examiners with
the subjective judgment of another is, to say the least, a poor substitute for
a standard rigorous statistical procedure.
(8) There are several other examples of
errors of judgment in the arguments contained in the monograph, indicating the
author's serious lack of understanding of the principles of statistics and
epidemiology. For instance, the rather amateurish interpretation of adequate
community "controls" for evaluating the effect of fluoride-bearing
water on caries indicates that the author does not really know the manner in
which caries occurs in a population.
Comment.
The question may be asked: Who does?
(9) The author's use of innuendo to
make a point is contained in his reference to some very questionable work of
Feldman (p. 192) which indicated that the eruption of deciduous teeth might
occur later in children exposed to fluorides. The author implies that it is
likely that retarded tooth eruption in the children residing in the fluoride
communities reduces their exposure to caries attack, and thus their caries rate
naturally is lower.
Comment.
On page 192, reference was not made to the work; of Feldman but to that of Dr
Reuben Feltman who was an associate of Doctors D. E. Gardner and F. A. Smith,
whose publications on fluorides are well known, and of Doctors H. C. Hodge and
D. E. Overton, who were closely associated with the Newburgh trial (Gardner,
Smith, Hodge, Overton and Feltman, 1952). Dr Feltman's earlier (1951) work with
fluoride tablets was referred to by the New Zealand Commission of Inquiry
(1957) as "promising". His 1956 paper, which was quoted (p.192), was
a brief "progress report" only. Therefore the results mentioned in it
were treated with reserve, the statement being made that "Of course, if
fluoridation results in the eruption rate of teeth being retarded ....".
(10) Had Dr Sutton been familiar with
the literature, he would have known that this was one of the first
possibilities thought responsible for the low caries rate in children exposed
to fluoride-bearing water.
Comment.
Dr Galagan has avoided the main point which was discussed in this paragraph.
His insistence that one of the first possibilities considered was that the
ingestion of fluoride-bearing water may retard tooth eruption, makes it even
more strange that in only the Newburgh-Kingston study was mention made that
this important matter had been investigated. Even in that trial, the only study
of tooth eruption rate published was conducted after four years of fluoridation
(Ast et al., 1951) in children who
were six to twelve years of age, so that none of the subjects studied had been
ingesting artificially fluoridated water throughout their lives.
(11) He would have known about the
rather exhaustive report of Short on the relation between fluoride in domestic
waters and tooth eruption which showed that fluoride in concentrations around
the optimum used for caries control does not influence the eruption pattern of
permanent teeth.
Comment.
It is highly probable that Dr Galagan is referring to the paper by E. M. Short
(1944) which is well known to those interested in fluorides, for this is the
only paper concerning fluorides and tooth eruption listed under that name in
the Index to Dental Literature in the English Language (published by the
American Dental Association), the Quarterly Cumulative Index Medicus (to
December 1956), or the issues of the Current List of Medical Literature which
cover the subsequent period.
This "rather exhaustive report of
Short" does not show whether or not the ingestion of fluorides at the
"optimum" level has any influence on the eruption pattern of
permanent teeth. This report was made on "selected 12 14 year old white
school children" (Short, 1944) in whom almost all the permanent teeth had
erupted. The data deal only with the total number of erupted permanent teeth
and, despite Dr Galagan's remark, do not give any information regarding their
eruption pattern-a factor which could be of considerable importance in the
development of the DMF rate. Short's Tables II and III show that, in all except
three of the 4,453 children examined, at least twelve of the permanent teeth
had erupted, the minimum number of erupted permanent teeth, for the remaining
three children, being ten. His Table I (which excluded third molars) shows
that, out of the possible twenty-eight teeth, the mean number of erupted
permanent teeth per child in the various cities was between 25.22 and 26.81. In
fact, in fifty-five per cent of the children all of the twenty-eight teeth had
erupted.
Therefore, this study of Short (1944)
gives no information regarding the ages at which the first ten permanent teeth
erupted in these children, certainly none regarding the first permanent molars
which, presumably, even in the youngest of these children, erupted about five
or six years prior to the study. These molars are of outstanding importance in
regard to the DMF rate, particularly in young children. Ast et al. (1956) said: "The first
permanent molars are frequently used as an index of caries experience among
children because this tooth accounts for the major incidence of caries in this
group."
It should be noted that neither Dr
Galagan nor (as will be seen in their review of this book) Doctors Blayney and
Hill, authors of the Evanston study, have commented on the suggestion of a
decline in the eruption rate of first permanent molars in Evanston, between
1946 and 1951, which followed the introduction of fluoridation and which is
depicted in Figure 4 (p. 139). Neither have they explained why, after 1951,
whilst continuing to publish this type of data for the older children, they
ceased publishing it for the younger
(12) However, one suspects that further
analysis of the details contained in the monograph will not yield much of value.
From reading the document and from hearing him present part of it as a paper
last February at your Adelaide Congress, I can only conclude that Dr Sutton has
an intense and emotional drive to oppose fluoridation. Why he feels this way is
not clear, but it seems likely to come from some motive other than a sincere
concern for the statistical or scientific validity of the concept.
Comment.
Dr Galagan's questioning of the motive behind this study should be considered
in relation to statements which he made during his lecture to the Adelaide
Congress (Galagan, 1959). He called those who questioned fluoridation "the
opposition" and said that this group "seems to be composed of four
distinct kinds of people." These he termed: "the hatemonger, the
pseudo-health believer, and the person who opposes fluoridation for personal
notoriety" and "the fourth, or rugged individualist, group". As
these are Dr Galagan's views, it is not surprising that he doubts the sincerity
of this attempt to investigate "the statistical or scientific
validity" of these fluoridation findings.
DR J. R. BLAYNEY and DR I. N. HILL
DR J. R. BLAYNEY and DR I. N. HILL
The second review of this book in the
Australian Dental Journal, by DR J. R. BLAYNEY, Director, Dental Caries Study,
Evanston, and DR I. N. HILL, Zoller Memorial Dental Clinic, University of
Chicago, was as follows:
Dr Sutton has much to say regarding the
lack of comparability(13) of the study and the control areas and the manner of
selection of the children in each area to be examined. Oak Park is the suburb
immediately to the west of Chicago and Evanston lies immediately to the north.
Each community draws its water supply from Lake Michigan. Standard analyses for
composition of tlw water are frequently run. A spectro-chemical analysis for 26
trace elements has been run on Lake Michigan water for a comparison of a
similar analysis of water obtained from fluoride areas (unpublished data). Oak
Park and Evanston receive their food from the same wholesale markets, each is
chiefly residential and free from heavy industry. Each is composed of the same
socio-economic level, as borne out by the United States census of population
for 1950. In Evanston, 47,395 persons 21 years or over were native born. In Oak
Park the figure was 42,454. In Evanston there were 6,049 foreign born persons
21 years or older while in Oak Park the figure was 5,081. In Evanston 41.2 per
cent of the occupied dwelling units were owner occupied. In Oak Park 50.8 per
cent were occupied by the owner. The median value of a one dwelling unit
structure in Evenston was 19,499 dollars and in Oak Park the value was 16,259
dollars. The median value of gross monthly rentals in renter occupied dwelling
units in Evanston was 72.53 dollars while in Oak Park the median value was
66.86 dollars. Both areas have comparable, climatic conditions and both are
subjected to the same radio and television commercials regarding oral hygiene
and dentifrices. Finally, the majority of the dental practitioners in the study
and control areas are graduates of one of the three Chicago dental colleges.
For the baseline examination in
Evanston we were committed to examine all
school children within the selected age range regardless of the length of time
they had resided in Evanston. Although an effort was made to include them, the
Oak Park Parochial Schools did not find it convenient for us to examine their
pupils(14). When we compared the caries prevalence rates of the two towns we
found a difference. It is indeed fortunate that our records showed the school
that each child attended. In this manner we could first eliminate the Parochial
School group and then the Negro group from the total Evanston data. It is well
known that coloured people have less dental caries than whites living in the
same population centres(15). The caries rates for the Evanston and Oak Park
Public School white children compared favourably(16). All of this clearly
indicates how hidden variables may exist in areas which otherwise appear to be
comparable, and how important it is to be certain that comparisons are made between
like groups(17).
Dr Sutton expressed astonishment that
in 1955 the six and seven-year-old Evanston children had a lower caries
prevalence rate than the Aurora children of like ages. We, likewise, did not
anticipate this. However, the same critical evaluation both clinical and
roentgenological, was made of every case. This difference was due to something
other than fluoride. Possibly the presence of the dental team in the school,
year after year, has stimulated the classroom teachers and the school nurses to
place more emphasis on the teaching of oral health. Some unknown hidden
variation not related to fluorides must account for the difference(18).
Dr Sutton was concerned that the
control group was not examined annually. Neither we nor our advisers could see
a reason to require an examination of the control group other than at the
beginning and near the close of the study. This provides the baseline from
which to measure the trend of the dental caries rate during the time interval
(1947-1956). Should the rate in the last examination (1956) deviate materially
from that of the initial baseline period (1947) that figure could be used as a
correction factor on the Evanston findings. In fact, if we had not desired to
measure the yearly decrement in the rate of dental caries under fluoridation
and evaluate other factors only two examinations in Evanston would have been
necessary, the first in 1946 (before fluoridation), the second and final in
1961(19).
Much has been made of the variations
reported in the number of children examined. The baseline examination of 4,375
Evanston children and of 2,493 Oak Park children are correct. However, the data
from those children who had not used Lake Michigan water all of their lives had
to be excluded. It was also observed that some children below the age of 67
months and above the maximum of 174 months had been examined. Therefore these
out of range children were not considered in the final determination of the
caries rates. This explains the discrepancy between the Evanston 4,375 and
3,692 and the Oak Park 2,493 - 2,051 figures(20). It should be noted that when
dental caries experience rates were compared, the same number of examinations,
that is 1,991 for the six to eight-year-old children and 1,701 for the 12 to
14-year-old children, were used throughout the reports for the combined
Evanston school groups (Public White, Parochial and Public Negro)(21). Dr
Sutton on p. 167 of his report (fig. 5, Statement C) calls attention to
Evanston Dental Caries Study Report Number XVII. Here he points out that only
1,754 six to eight-year-old children and 1,556 12 to 14-yearold children were
listed. This particular report was primarily concerned with differences in sex
and race, as they influenced caries, rather than the effect of fluorides.
Therefore a comparison was made of the dental caries experience rates of white
girls to white boys; Negro girls to Negro boys; white girls to Negro girls;
white boys to Negro boys; and white children as a group to Negro children as a
group. The children were classified into male white, female white, male Negro
and female Negro for comparison(22). As Dr Sutton calls attention to the
difference in the number of examinations made in this report when compared with
other reports, we wish to point out, in explanation, that it was necessary to
not only insure that children of correct age be included but also it was
necessary that every examination be clearly classified according to race and
sex There were 236 six to eight-year-old children and 245 12 to 14-year-old
children excluded from this report because they did not fulfil the requirements
for this comparison. These children were included in other reports as no
distinction except age was made(23). It is also pointed out that this report,
Evanston No. XVII as noted above, was primarily concerned with comparison of
the caries rates of the coloured and white children and not with the effect of
fluoridation on dental caries rates. Therefore in this light, this report, No.
XVII, should not have been listed under the general heading of fluoridation as
Dr Sutton has it listed in his critique(24).
It is true that a discrepancy in
figures published in our paper XVI, Table I and in paper XVIII, Table I are at
variance. This is due to the operator of the tabulating machine providing the
wrong figures for the number of seven and eight-year-old children examined.
This error was discovered after manuscript XVI was in press and therefore the
corrections could only be made manually in the reprints supplied to readers who
requested them(25).
Commentary
on the Review by Dr J. R. Blaynex and Dr L N. Hill
(13) Reference to Part Two will show
that no suggestion was made in it that there was a "lack of
comparability" between Evanston and Oak Park; it was merely pointed out
that the manner in which Oak Park resembled Evanston was not stated.
Therefore this detailed exposition of
the similarity of the two cities is welcome, although it is unfortunate that,
when speaking of rental and dwelling values, the "mean" values were
not given as well as the "median" ones. It will be realized that the
housing picture may be very different in two towns and yet the
"median" (that is, the middle) values of the rentals and dwellings
can be the same. In view of the data shown here, regarding housing and the
training of the dentists, it is surprising that the members of the United
Kingdom Mission (1953) should have singled out Evanston for comment, remarking
on its high economic level and its "outstandingly good" dental care
(p. 149).
These data also show how reasonable was
the assumption of Dr Blayney before assessing the caries rates in Oak Park-that
that city was "the ideal control community" for Evanston (Blayney and
Tucker, 1948;). They also emphasize how strange it is that such gross differences
should be found between the initial caries rates of the children aged six to
eight years in the two cities, and reported-after a delay of ten years-by these
workers (p. 153; Fig. 3, p. 154). The fact that such gross differences can be
found in the caries rates prevalent in two cities which were so similar that
one was termed "the ideal control community" for the other (Blayney
and Tucker, 1948), confirms the necessity for pre-fluoridation examinations in
both test and control cities. Unfortunately, as pointed out in Part Two this
was not done in any of these studies.
(14) For the baseline examination in
Evanston we were committed to examine all school children within the selected
age range, regardless of the length of time they had resided in Evanston.
Although an effort was made to include them the Oak Park Parochial Schools did
not ford it convenient for us to examine their pupils.
Comment.
The latter remark is welcome for it explains the absence of data from the
parochial schools in Oak Park.
(15) When we compared the caries
prevalence rates of the two towns we found a difference. It is indeed fortunate
that our records showed the school that each child attended. In this manner we
could first eliminate the Parochial School group and then the Negro group from the
total Evanston data. It is well known that coloured people have less dental
caries than whites living in the same population centres.
Comment.
Despite their statement that "It is well knowp that coloured people have
less dental caries than whites living in the same population centres",
when conducting the initial examination in Evanston the authors combined the
data of the Negro children with those of the white children. It is now clear,
as deduced on page eighteen, that the racial and school groups were taken into
account only after it was found that there was "a lower caries rate for
school children of the control area" (Hill et al., 1951). Thereafter, when comparing the test and control
cities, the data of both the Negro and the parochial school children were
excluded from the Evanston data. No reason has been given for this exclusion of
the data of parochial school children in Evanston-who had a high caries rate
(Hill et al.) - from the data of the
main body of white children in that city. This could not be attributed to the
fording of a similarly high caries rate in the parochial school children in the
control city of Oak Park for they were not examined and, therefore, their
caries rates were unknown (see 14).
Neither has an explanation been offered
for the extraordinary reversal of this policy (the exclusion of the data of
Negro and parochial school children in Evanston) when compiling the XVIII
Report (Hill et al., 1958). This
report published, for the first time, the initial caries rates for the
permanent teeth of the children aged six to eight years which were obtained,
ten years earlier, in the control city of Oak Park.
The rates for the deciduous teeth,
which were obtained at that time, still have not been published. This report
provided the first opportunity to compare the initial caries rates of the
younger children in the test city and its control (p. 153).
(16) The caries rates for the Evanston
and Oak Park Public School white children compared favourably.
Comment.
This statement is interesting-for the caries rates for children aged six to
eight years in each of the three school groups have not been published (p.151).
It will be recalled that the mean caries rates for the six, seven, and
eight-year-old children in Evanston in 1946 were very much higher than the mean
rates for children of those ages obtained during the initial examination in Oak
Park. For the children aged six years the rate in Evanston was 46.85, but it
was only 26.89 in Oak Park (Hill et al.,
1958). Only 0.1 per cent of the Oak Park children were Negro (Hill et al., 1951), but exclusion of the data
of Negro children (who have a relatively low caries rate, see 15) from the
Evanston data would increase the rate of the remaining (white public and
parochial school) children so that in the six-year-old children, it would be
higher than 46.85. Therefore the difference between this rate and the Oak Park
rate of 26.89, for children of that age, would be increased.
Hill et al. did not say how many of the younger age group of children
attended each type of school, but only twenty-two per cent of the twelve to
fourteen-year-old children, shown in their 1957a report as examined in Evanston
in 1946, attended parochial schools. Therefore, the proportion of children aged
six, seven, and eight years who were attending parochial schools in Evanston in
1946, and their caries rate, must have been very high to permit Doctors Blayney
and Hill to state that "The caries rates for the Evanston and Oak Park Public
School white children compared favourably." Of course speculation is no
substitute for data-and this still has not been published
(17) All of this clearly indicates how
hidden variables may exist in areas which otherwise appear to be comparable,
and how important it is to be certain that comparisons are made between like
groups.
Comment.
The latter phrase is a reiteration of remarks made in the 1957a report from
this study, that it is necessary "to make comparisons of like
groups." Why then, having realized this necessity, did Hill et al. ignore it in their 1958 report
(p. 152)? In this report the data shown, for the year 1946, combined not only
that of the white children attending both public and parochial schools, but the
data of the Negro children as well. The resultant rate was then compared with
that of children in Oak Park comprising, almost entirely, white children
attending public schools. By ignoring the opinion they expressed in the
previous year (1957a)-which they now reiterate-and comparing "unlike"
groups of children, a more favourable degree of comparability was obtained
between the initial caries rates of children in the test and the control
cities.
(18) Dr Sutton expressed astonishment
that in 1955 the six and seven-year-old Evanston children had a lower caries
prevalence rate than the Aurora children of like ages. We, likewise, did not
anticipate this. However, the same critical evaluation both clinical and
roentgenological, was made of every case. This difference was due to something
other than fluoride. Possibly the presence of the dental team in the school,
year after year, has stimulated the classroom teachers and the school nurses to
place more emphasis on the teaching of oral health. Some unknown hidden
variation not related to fluorides must account for the difference.
Comment.
It is pleasing that Doctors Blayney and Hill should support the view expressed
on page 187 (para. 4) that regular dental examinations may stimulate interest
in the teeth and thus lead to improved oral health. In advancing the suggestion
that "something other than fluoride" can affect the caries rates,
they recognize the great importance which factors other than the fluoride
concentration of the water supply may have on the caries rates. This extremely
important matter was practically ignored by the authors of all these studies
when preparing their reports.
(19) Dr Sutton was concerned that the
control group was not examined annually. Neither we nor our advisers could see
a reason to require an examination of the control group other than at the
beginning and near the close of the study. This provides the baseline from
which to measure the trend of the dental caries rate during the time interval
(1947 1956). Should the rate in the last examination (1956) deviate materially
from that of the initial baseline period (1947) that figure could be used as a
correction factor in the Evanston findings. In fact, if we had not desired to
measure the yearly decrement in the rate of dental caries under fluoridation
and evaluate other factors, only two examinations in Evanston would have been
necessary, the first in 1946 (before fluoridation), the second and final in
1961.
Comment.
This statement makes two things clear. The first is that, at the commencement
of the study, neither the workers nor their advisers could have considered the
possibility, which they now acknowledge(18), that "the presence of the
dental team in the school, year after year" might have had a stimulating
effect "on the teaching of oral health." It is obvious that, if this
effect is possible, not only the test town but also its control should have
been examined "year after year". The other point which is indicated
by this statement(19) of Doctors Blayney and Hill is that despite their remark
in 1950, the importance of random variation was not and, seemingly still is not
recognized.
(20) Much has been made of the
variations reported in the number of children examined. The baseline
examination of 4,375 Evanston children and of 2,493 Oak Park children are
correct. However, the data from those children who had not used Lake Michigan
water all of their lives had to be excluded. It was also observed that some
children below the age of 67 months and above the maximum of 174 months had been
examined. Therefore these out of range children were not considered in the
final determination of the caries rates. This explains the discrepancy between
the Evanston 4,375 and 3,692 and the Oak Park 2,493 - 2,051 figures.
Comment.
This explanation of the difference between these sample sizes in Evanston and
Oak Park is welcome. It might have been deduced if the decision to exclude
"the data from those children who had not used Lake Michigan water all of
their lives" had been announced in one of the five reports giving caries
rates, issued prior to 1955 (p. 163).
In the XIX Report (Hill et al., 1959), the sample sizes shown
for the two age groups in Oak Park in 1947 (1,022 and 1,032) are almost the
same as those shown (1,020 and 1,031) in statement "E" of Figure 5.
Comparison with the statements for Evanston cannot be made for this (XIX)
report considered only "public school white children".
(21) It should be noted that when
dental caries experience rates were compared, the same number of examinations,
that is 1,991 for the six to eight-year-old children and 1,701 for the 12 to
14-year-old children were used throughout the reports for the combined Evanston
school groups (Public White Parochial and Public Negro).
Comment.
It is surprising that the suggestion was made that this statement should be
noted-for it is not correct. The number of twelve, thirteen and
fourteen-year-old children examined in Evanston in 1946 was given in Tables
III, V, VI, VII, VIII, IX and X of the XV Report (Hill et al., 1957a) as 418, 688 and 595, a total of 1,701. However, in
Tables XI and XII of the same paper different sample sizes for these ages were
shown: 414, 692 and 617, a total of 1,723. The same discrepancies were noted
between different tables in the XI Report (Hill et al., 1955) Therefore the figure 1,701 was not "used
throughout the reports for the combined Evanston school groups".
(22) Dr Sutton on p. 167 of his report
(fig. 5, Statement C) calls attention to Evanston Dental Caries Study Report
Number XVII. Here he points out that only 1,754 six to eight-year old children
and 1,556 12 to 14-yearold children were listed. This particular report was
primarily concerned with differences in sex and race, as they influenced
caries, rather than the effect of fluorides. Therefore a comparison was made of
the dental caries experience rates of white girls to white boys; Negro girls to
Negro boys; white girls to Negro girls; white boys to Negro boys; and white
children as a group to Negro children as a group. The children were classified
into male white, female white, male Negro and female Negro for comparison.
Comment.
A curious feature of this XVII Report is that although care was taken in regard
to the age, race, and sex of the subjects, no attempt was made to "limit
the examinations to continuous resident children" (Hill et al., 1957b). Thus it is reasonable to
assume that some children were examined who were not "continuous"
residents. Therefore the comparisons mentioned by Doctors Blayney and Hill were
made on mixed samples of children some of whom had not ingested fluoridated
water, those examined in 1946, and others who had done so for varying periods
of up to about eight years. This disregard of the possible effect of the
ingestion of fluorides on the caries rates, of some of the children examined,
is inexplicable.
(23) As Dr Sutton calls attention to
the difference in the number of examinations made in this report when compared
with other reports, we wish to point out, in explanation, that it was necessary
to not only insure that children of correct age be included but also it was
necessary that every examination be clearly classified according to race and
sex. There were 236 six to eight-year-old children and 245 12 to 14-year old
children excluded from this report because they did not fulfil the requirements
for this comparison. These children were included in other reports as no
distinction except age was made.
Comment.
This explanation why the sample sizes from Evanston shown in the XVII Report
(1,754 six to eight-year-old children and 1,556 twelve to fourteen-year-old
children) do not agree with those shown in other reports at first appears to be
a reasonable one. However, before it is accepted, consideration should be given
to two observations. Firstly, if the figures depicted in statements
"B" and "C" of Figure 5 which were originally given by Hill
et al. and are now confirmed by
Doctors Blayney and Hill (20, 22), are accepted as correct, the numbers of
children excluded in the two age groups (1,991-1,754 and 1,701-1,556) were 237
and 145, not 236 and 245 as stated by Doctors Blayney and Hill. It is possible
that these errors could have arisen in typing the manuscript, but this could
not be the case in regard to the second observation.
This second observation is as follows: In
the XVII Report (Hill et al., 1957b)
it was stated that "in this report no attempt has been made to limit the
examinations to continuous resident children." Thus it is almost certain
that data from both "continuous" and "non-continuous"
resident children are included in the total of 3,310 subjects mentioned in that
Report as examined in Evanston in 1946. Therefore, to determine the number of
children who were excluded from the XVII Report, comparison must be made, not
with the number of "continuous" residents of correct age that were
examined in 1946 (3,692, see 20) and "were included in other
reports", but with the total number of children ("continuous"
and "non-continuous" residents and "out of range") that
were examined in 1946, that is, 4,375.
If this is done, it can be seen that
1,065 of these children (4,375-3,310) were excluded from the XVII Report. This
figure includes some "out of range" children, for it was stated that
"some children below the age of 67 months and above the maximum of 174
months had been examined" (20) in the baseline examination of 4,375
Evanston children. Nevertheless, unless there were as many as 584 "out of
range" children (1,065 = 584 + 481), the actual number of children
excluded, because they were not of correct age or could not "be clearly
classified according to race and sex", must have been larger than the
figure of 481 given here (236 + 245) by Doctors Blayney and Hill.
(24) It is also pointed out that this
report, Evanston No. XVII as noted above, was primarily concerned with
comparison of the caries rates of the coloured and white children and not with
the effect of fluoridation on dental caries rates. Therefore in this light,
this report, No. XVII, should not have been listed under the general heading of
fluoridation as Dr Sutton has it listed in his critique.
Comment.
The XVII Report from Evanston was not listed as a fluoridation study; it
appears in the list of references (as do other papers not specifically
concerned with fluoridation) because it was mentioned in the text. It was
consulted in an attempt to investigate the confusing matter of the differences
in the sample sizes for 1946 and 1947 shown in the various reports.
(25) It is true that a discrepancy in
figures published in our paper XVI, Table I and in paper XVIII, Table I are at
variance This is due to the operator of the tabulating machine providing the
wrong figures for the number of seven and eight year old children examined.
This error was discovered after manuscript XVI was in press and therefore, the corrections
could only be made manually in the reprints supplied to the readers who
requested them.
Comment.
If the errors contained in the XVI Report had been pointed out by providing an
additional footnote to Table I in the XVIII Report (which was the next report
in which this type of data was published) the reason for the difference between
the two sets of figures would have been obvious. It should be noted that,
although the source of the errors in the 1955 rates in Table 1, XVI Report was
given, no mention has been made of the fact that, in the same table, there are
errors in computing the rates for the six to eightyear-old age group in the
years 1946 and 1948.Both of these errors were of long standing as they were
shown, four years earlier, in the X Report (Hill et al., 1952). These errors were still contained in the XV Report
(Hill et al., 1957a).
It can be seen that Doctors Blayney and
Hill devoted a considerable part of their review to two matters. The first was
the "comparability of the study and the control areas"-which was not
questioned (see comment 13). The second was a lengthy description of the
comparisons which they made between different groups of children in obtaining
the data for the XVII Report. This information was given in almost the same
words in that report, which, they stated (24), was not primarily concerned with
the effect of fluoridation on dental caries rates.
However, they have not mentioned most
of the matters which do directly concern fluoridation and caries prevalence and
which were questioned. In fact their comments have touched on matters mentioned
in only about a third of the sub-headings used in considering their study.
Their meagre explanations have accounted for the presence of some of the errors
in one table .... and have supplied a reason for the differences between the
sample sizes for the year 1947 in Oak Park, and for the disparity between two
of the three sample sizes for 1946 in Evanston (p 165) However, most of the
matters mentioned in considering the Evanston study were ignored, even those
illustrated by Figures 3 and 4.
It should be noted, therefore, that
Doctors Blayney and Hill have not commented on the majority of the errors,
omissions and mis-statements mentioned in considering the Evanston study, and
almost all of them remain unexplained.
...
Continued next page
No comments:
Post a Comment