Errors
and Omissions in Experimental Trials - 1c
THE BRANTFORD STUDIES
In the city of Brantford; Canada,
two independent trials were conducted.
1. The City Health Department
Study
In this study no control procedure
was attempted. However, it will be considered
briefly because two pre-fluoridation surveys were
made by the school dental officer and his assistant
(Hutton et al., 1954). This is the only one
of these five trials in which more than one
pre-fluoridation survey was made in the test city;
and, with the exception of Muskegon, none of the
control cities provided data obtained in successive
years from individual yearly age groups.
Were results combined or
averaged? Hutton et al. (1951) stated
that "The results of these two [pre-fluoridation]
surveys have been combined and are shown in Table
I." In Tables I and II of the Ontario Department of
Health Report (1956) the rates for those two surveys
were shown separately. With the exception of those
of the nine-year-old children, for both the
deciduous and the permanent teeth, the mean of the
two rates for each age is identical (to one decimal
place)
with the mean rate computed from the figures
of the combined survey which were supplied by the
authors (Hutton et al., 1951, Table I). This
result could have arisen only if (with the exception
of the children who were nine years old) the number
of children of the same age examined on both
occasions was equal, or almost exactly so - a most
unlikely event; or if the results were not combined,
as stated by the authors, but the rates obtained in
1944 and 1945 were averaged. The United Kingdom
Mission (1953) stated that "the average figures of
these two years" were used. If the rates for the two
years were averaged, there were errors in computing
the rates of the nine-year-old children, or errata
in one or more of those three tables. The figures
shown in Tables I and II of the Ontario Department
of Health Report (1956) should be treated with
caution, because in both of these the year of
fluoridation is stated incorrectly, and in the
former table the "% Reduction Since 1944-45" for age
seven years should be 66, not 51; whereas in the
latter one, the "% Reduction Since 1948" for age
nine to eleven years in Stratford and Sarnia should
not have been indicated by dashes, but by five and
sixteen respectively.
2. The National Health and
Welfare Study
The other study in Brantford was
conducted by the Canadian Department of National
Health and Welfare, and was described by the New
Zealand Commission of Inquiry (1957) as "the most
complete of the 10-year North American studies".
Late commencement.
Unfortunately, this trial was not begun until
January 1948, over two and a half years after the
commencement of fluoridation of the Brantford water
supply (Brown, 1951). Such delay must affect the
value of this study, unless it is assumed that the
structural theory of reduction of dental caries as a
result of the ingestion of fluorides is correct, and
that this is the only way in which fluorides may
affect the incidence of caries. This theory was
advanced by Cox and Levin in 1942, and was widely
accepted at the time these trials were initiated
(Dean et al., 1950; Ast et al., 1950).
If this theory is correct (as was noted in Part
One), little change can be expected in the DMF rates
until about six years after the commencement of the
study. It is evident that this theory must still be
held to be correct in some quarters, for the recent
report of a dental caries survey conducted by
McCauley and Frazier (1957) stated: "Although
fluoridation of the Baltimore City water supply was
begun Nov. 26, 1952, (27 months before the survey),
there was no reason to anticipate substantial change
in the caries experience of these children in this
relatively short period of time." However, even
before the commencement of the Department of
National Health and Welfare study in Brantford, the
City Health Department examiner's figures for 1947
showed great reductions in the DMF rates since the
introduction of fluoridation. This result was not
published by Hutton et al. until 1951, but
must have been available to the investigators who
"came to the scientific rescue of the project early
in 1948" (Hutton et al., 1956). Indeed, in
his first report, Brown (195 1) acknowledged the
help and advice of two of the three authors of the
City Health Department Report (Hutton et al.,
1951).
The control cities.
The city of Sarnia was chosen as the "fluoride-free"
control, and Stratford as the control city with a
water supply which "contains 1.3 ppm. of fluorine
from a natural source" (Brown, 1951). The reasons
for the selection of these cities were not given,
except that it was said: "sarnia and Stratford, two
cities in Western Ontario known to be comparable to
Brantford, except for the fluoride content of their
water supplies, agreed to serve as controls" (Brown
et at.\,1954b). Also, Brown, Josie and Stewart
(1953) said that Sarnia was "a city" which has
fluoride-free water and is sufficiently similar in
size, location, and other attributes for purposes of
the comparison". The United Kingdom Mission (1953)
stated: "Before this study was undertaken the
socio-economic status of the three communities was
examined and found to be reasonably comparable."
Superior dental care in
Brantford. The United Kingdom Mission (1953)
said: "Brantford, however, over a period of 15
years, has provided more free dental services for
children than most Canadian cities, and this has
resulted in the ratio of corrected to total defects
being higher than in either Sarnia or Stratford." It
considered that in Brantford "dental care was
outstandingly good." Also, Brown, in 1952, said:
"the recordings so far obtained
indicate both a higher treatment and an apparently
better oral hygiene status of the Brantford children
when compared with the controls, and it is therefore
suggested that caution should be exercised in the
interpretation of the rates shown. The lack of a
pre-fluoridation survey on a comparable basis is a
further limiting factor in interpreting the
results."
No pre-fluoridation survey.
The authors of this Brantford study (Brown et
al., 1953) said:
"As the study does not include a
pre-fluoridation survey, the full amount of benefit
which the Brantford teeth have received since
fluoridation cannot be illustrated directly from the
data for Brantford. Some idea of the extent of the
benefit can be obtained by comparison with the data
for Sarnia.... By 1948 the Brantford data were not
greatly different from those for Sarnia."
This remark suggests that the data
for the two cities prior to fluoridation in
Brantford were similar, and that this process had
had little effect on the caries rates up to the time
of the 1948 examination in Brantford.
Doubtful comparability of rates.
Owing to the delay in setting up this study, it
cannot be established how closely the dental caries
attack rates in Brantford resembled those in Sarnia,
at the time fluoridation was instituted in the
former city. There is evidence that the dental
condition of the children in those two cities was
not closely comparable, for Brown et al.
(1953) stated that "even by the time of the first
survey, mean tooth mortality in Brantford was much
lower than in Sarnia, for all age groups."
This comment implies that, even by
the time of the first survey, as a result of
fluoridation the tooth mortality in Brantford had
decreased considerably. This concept is not
consistent with the one mentioned in the last
paragraph. At the time of the first examinations,
the tooth mortality in the six to eight years age
group was more than four times as great in Sarnia as
it was in Brantford, and in each of the other two
age groups it was almost twice as great (Brown et
al., 1953, Table 3.)
 |
|
Figure 6.
The gross
differences observed in the tooth mortality
(teeth which are missing or which must be
extracted) in Brantford and its control city of
Sarnia, during the initial examinations.
Canadian Department of Health and Welfare study.
|
The influence of treatment.
The fact that such large differences were reported
in tooth mortality rates in the two cities even in
the older age groups suggests that dental treatment
in them was different, and the authors stated that
"Both preventive and treatment measures may have a
decided effect on tooth mortality rates" (Brown
et al., 1953). It may be recalled that the
United Kingdom Mission (1953) noted that Brantford
was unusually well provided with free dental
services "and this has resulted in the ratio of
corrected to total defects being higher than in
either Sarnia or Stratford."
Tooth mortality. Brown et
al. (1953) said that "there has been a decrease
in tooth mortality in Brantford between successive
surveys"; but, in fact, the 1953 rate (Table 3) in
the children aged six to eight years was the highest
up to that time. This statement was corrected in the
next report (Brown et al., 1954b) by
prefixing "in almost all cases" to the previous
statement. In this connection, the authors remarked
in 1953 that, "as well as the fluoridation of the
Brantford-water supply, other factors such as
differences in preventive or treatment measures are
probably affecting the Brantford position." There
appears to be no reason why those of the control
cities should not have been similarly affected.
Differences in oral hygiene.
Additional evidence which suggests that a difference
existed between the dental condition of the children
in Brantford and that of children in the control
cities is provided by the data with regard to oral
hygiene. Brown et al. (1954b) stated:
"Classification and recording of
oral hygiene was undertaken because it was
considered that marked differences in oral hygiene
as between the test and control groups might
conceivably affect the findings--or at least might
be taken into consideration as a modifying factor,
although not a strictly measurable one. However, the
figures here suggest that, since 1948, differences
in oral hygiene status could not have been a major
factor in either the caries level changes within
Brantford or the caries level differences between
the control cities."
As no comparisons were made between
the control cities, the last phrase of the quotation
is thought to refer to the caries level differences
between Brantford and each of the control cities.
The authors' Table 11 indicates that, in the first
examinations, in the test city the percentage of
subjects who had a good oral hygiene status was
almost twice as great as that present in children in
both the control ones; these were, Brantford 34.3
per cent, Sarnia 19.7 per cent and Stratford 17.8
per cent. Considerable differences between the oral
hygiene status of the children in the test and the
control cities were also recorded during the later
examinations. These were clearly "marked
differences" though the authors did not consider
them important.
The concentration of fluorides.
Brown et al. (1954b) stated:
"The Brantford Fluoridation Caries
Study was undertaken with a view to finding out
whether or not the raising of the fluoride content
of a previously fluoride-free water supply to 1 part
per million, by the mechanical addition of sodium
fluoride, would reduce the incidence of dental
caries to that which obtains where water supplies
derive about 1 part per million of fluoride from
deposits in the earth."
A fundamental requirement of a test
of this nature is that the water supply of the
control city should contain the same concentration
of fluorides as that of the test one, but Brown,
McLaren, Josie and Stewart (1956) reported: "The
Stratford water supply contains a concentration of
fluoride which is 60% higher than that used in
Brantford."
Differences of opinion.
Several different statements were made regarding the
concentrations of fluorides which were present in
the water supplies of Brantford and Stratford.
1. Brantford. The New Zealand
Commission of Inquiry (1957) said that the water
supply of Brantford was "raised to 1.2 ppm. in
1945". The authors of the City Health Department
study (Hutton et al., 1951) stated that in
February 1949, "the dosage was raised to produce
1.20 ppm."; but in 1954 they stated that "The
fluoride content of the finished water is maintained
at 1 ppm." In reporting the National Health and
Welfare Study, Brown (1952) stated that "a fluorine
concentration of between 1.0 and 1.2 ppm. has been
maintained in the water supply continuously" since
June 1945; and in 1956 Brown et al. said:
"Brantford has had more than 10 years of experience
with 1 part per million fluoride in its water
supply." These statements that the fluoride content
was "maintained" at "1 ppm" and "between 1.0 and 1.2
ppm" should be considered in conjunction with that
of the United Kingdom Mission (1953): "For example,
in 1951 the average for the year was 1.2 ppm with a
variation between 0.75 and 1.45 ppm., however, the
figures below 1.1 ppm and above 1. 3 ppm were few in
number."
2. Stratford. The supply of
Stratford was stated to have 1.2 ppm of natural
fluoride in its domestic water (Ontario Department
of Health, 1956; New Zealand Commission of Inquiry,
1957). However, in reporting the National Health and
Welfare study in 1951 and 1952, Brown stated that it
"contains 1.3 ppm. of fluorine"; and, with his
co-workers, the following year said that "in
Stratford where the water supply, obtained from deep
wells, has contained 1.3 ppm." (fluoride) "from
natural deposits continuously since 1917" (Brown
et al., 1953). In their next report dealing with
dental caries (Brown et al., 1954b), the
figure stated was "1.3 to 1.6 parts per million of
fluoride"; but in the following report these authors
(Brown et al., 1956) said that this water
supply "contained 1.6 parts per million of fluoride
since 1917", and also that 1.6 ppm fluoride content
"has been in continuous use for thirt yeight years."
These different statements, although
strange, may be considered to be unimportant from
the practical point of view, but very small changes
in the fluoride content of the water may have
considerable effects, as reports by Deatherage
(1942) and Galagan and Lamson (1953) indicated. The
latter authors found that "In water supplies of the
Arizona communities studied, concentrations of
fluoride above 0.8 ppm resulted in objectionable
dental fluorosis; concentrations of 0.6 to 0.8 ppm
resulted in an occasional diagnosis of fluorosis;
concentrations below 0.6 ppm did not cause
objectionable fluorosis." The mean temperatures of
these communities were between 67 and 72 degrees
Fahrenheit.
Three misleading statements.
In the summary of the 1954b report of the National
Health and Welfare study, Brown et al. stated
that during the period 1948-54 "dental caries
experience of children in the two control cities, on
the other hand, either has remained at about the
1948 levels, or has increased slightly, at all ages
studied." A similar statement was made by them in
the 1955 Report (Brown et al., 1956): "During
that time [more than ten years] a very important,
statistically significant reduction in tooth decay
has occurred in all the age groups studied, while in
the two control cities of Sarnia and Stratford it
has either remained at about the same level or
increased somewhat." The last sentence contains
three misleading statements about the control
cities:
(a) As this study did not commence
until 1948 with "examinations in Sarnia beginning in
March of that year, and in Stratford in October"
(Brown, 1952), and as fluoridation in Brantford
commenced in June 1945 (Hutton et al., 195
1), no information is available with regard to the
prevalence of "tooth decay" in Sarnia during the
first two and three-quarter years, or in Stratford
for the first three and a quarter years of the
ten-year period of fluoridation which these authors
were discussing in their 1955 Report. Therefore, it
cannot be known whether this condition "remained at
about the same level" in the control cities during
the early years of fluoridation in Brantford.
(b) No remarks were made in the
context of this statement in this (or the previous)
paper which suggested that reference was being made
to the permanent teeth only, but these statements
are not correct for the deciduous teeth. (However,
in the former paper, a similar statement to that
made in the summary was also made under the heading
"Mean DMF Permanent Teeth".) Decreases in the df
(decayed, filled deciduous teeth) rate were seen in
the nine to eleven years group in both Sarnia and
Stratford, that in the former city being shown by
these authors to be statistically significant (Brown
et al., 1954b, Table 10).
(c) In both cities the mean rates of
DMF and df teeth per child showed changes which were
said to be statistically significant (Brown et al.,
1954b).
Omission of decreases. The
decreases in the caries attack rates of the
deciduous teeth, which have been mentioned above,
were also omitted in Table II of the Report to the
Minister of Health, Province of Ontario, Canada, by
the Division of Medical Statistics, Ontario
Department of Health, which was made in 1955
(Ontario Department of Health, 1956). Under the
heading "% Reduction Since 1948", these decreases
were not shown, but instead, in the appropriate
positions dashes were printed, despite the fact that
in Sarnia the percentage reduction (determined by
the method commonly used in these studies) was 16
per cent, almost as great as that of 18 per cent
shown for the same age group in the test city;
furthermore, this reduction in Sarnia was stated by
the authors (Brown et al., 1954b) to be
statistically significant.
Different rates reported. It
should be noted that the deficiency in the data of
the National Health and Welfare study, owing to its
late commencement, could not be decreased by
comparing the rates obtained by its examiners with
those reported by the City Health Department
examiner, because of the considerably lower rates
recorded by the last-mentioned examiner when
impecting similar groups of children. For instance,
in their examination in 1948, for children aged six
to eight, nine to eleven and twelve to fourteen
years, Brown et al. (1953) obtained rates of
1.41, 4.07 and 7.68 respectively for the permanent
teeth, compared with rates of 0.84, 3.37 and 6.11,
for the same age groups of children, in the same
city and in the same year, obtained by the City
Health Department examiner (calculated from Table
IV, Hutton et al., 1951).
Significant fluctuations in
controls. In the two control cities "where it is
presumed that there has been no appreciable change
in either preventive or treatment services" (United
Kingdom Mission Report; 1953), it can be seen in
Tables 4, 6, 8 and 10 published by Brown et al.
(1954b) that some considerable fluctuations in the
caries attack rates were recorded; more than half of
the inter-year differences in each of the control
cities being shown to be statistically significant.
However, in the text it was stated that the "dental
caries experience of children in the two control
cities . . . either has remained at about the 1948
levels, or has increased slightly, at all ages
studied."
(1) Sarnia. In this city the changes
between examinations of the rates for the deciduous
teeth were not very marked, but there was a
significant one between 1948 and 1954 in the nine to
eleven years age group. However, in the DMF
permanent teeth, there were four definitely
significant (three standard error level) and one
significant change in the nine comparisons made. In
regard to the first permanent molars, there were six
significant (including three definitely significant)
alterations in the rates, in the nine comparisons
made (Brown et al., 1954b).
(2) Stratford. In this city, the
rate of df teeth per child showed a significant
difference in one case out of the four comparisons
made between successive examinations (Brown et al.,
1954b). In the DMF permanent teeth per child, the
results of the four examinations were: 0.41, 0.75,
0.47 and 0.67 for the six to eight years group;
1.13, 1.76, 1.46 and 1.89 for the nine to eleven
years age group; 2.55, 3.12, 3.02 and 3.77 for the
twelve to fourteen years age group (Brown et al.,
1954b, 1956). These variations between examinations
were so large that five out of the six comparisons
made (in the 1954b report) between successive
examinations were said to be statistically
significant, four of them being at the three
standard error level. In the last report published
(1956), Brown et al. abandoned the method
which they had used in the two previous ones, that
of showing the standard error of the mean values of
the DMF rates, and of making "Inter-City" and
"Inter-Year" comparisons (Brown et al., 1953,
1954b). Therefore it was not stated whether the
differences between the 1954 and the 1955 DMF rates
in Stratford were significant, but it can be seen
that they were marked; the difference of 0.75 in the
twelve to fourteen years group being considerably
larger than any of those stated in the 1954b report
to be significant differences between various
examinations in that city. When the DMF rates for
the first permanent molars are considered, similar
marked changes are seen, and in five of the six
comparisons the differences were significant (four
definitely so).
Larger "percentage" changes in a
control. If one resorts to the method commonly
used in these trials - that of expressing the
alteration in the DMF rate as a percentage of the
original rate - these unexplained increases in the
control city of Stratford between 1948 and 1955,
although they were described as "no change" (Ontario
Department of Health, 1956), and as "a slightly
higher prevalence of dental caries in 1955, over the
1948 levels" (Brown et al., 1956), are found
to be 63 per cent, 67 per cent and 48 per cent, for
the six to eight, nine to eleven and twelve to
fourteen years age groups respectively. In each case
these percentage changes are considerably larger
than those of 51 per cent, 44 per cent and 37 per
cent which can be computed from the data reported
for Brantford. The last-mentioned changes were
attributed to fluoridation, and each was stated to
indicate "a very important, statistically
significant reduction in tooth decay" (Brown et
al., 1956).
This is just one instance of the
strange results which are obtained when this method
of calculation is used. It should be realized that
it was the one most commonly employed in
fluoridation trials, and was used in formulating the
often-expressed claim that (as stated by Arnold
et al., 1956): "In children born since
fluoridation was put into effect, the caries rate
for the permanent teeth was reduced on the average
by about 60 per cent." The recent World Health
Organization Press Release (WHO/45, 4 September
1957) stated - with no mention of age - "The
prevalence of dental caries in the permanent teeth
of children decreased some 60 percent".
A smaller "percentage decrease"
after long fluoridation. The "percentage
decreases" which have just been mentioned (51 per
cent, 44 per cent and 37 per cent, calculated by the
method described in the last paragraph) were not
stated in the 1956 report of Brown et al.,
but the figure of 51 per cent for the six to eight
years age group is considerably less impressive than
the figure of "approximately 69%" published in the
1954b report from this study. Although the final
report (1956) gave the rates for 1948 and 1955 only,
and therefore did not show the fluctuations between
examinations, from the 1954b and 1956 reports of
Brown et al. it is seen that the marked
change in the "percentage" decrease which has just
been mentioned was due to the DMF rate in Brantford
in 1955, for this age group, being the highest seen
since 1951. Ignoring the fact that in "children born
subsequent to fluoridation" the "decrease" in the
DMF rate had dropped to only 51 per cent, the
authors stated in the final sentence of their final
report (1956): "For every three decayed teeth they
would have had, they have only one."
More misleading comments.
Turning from the reports made by the authors of this
study about the control cities to some of the
comments made by others, it is seen that these are
even more misleading. Only two will be mentioned.
Martin (1956) stated that during "the 1948-54
period" the "DMF figures for the two control areas
have remained at 1948 levels." The authors of the
Ontario Department of Health Report (1956) went so
far as to state to their Minister of Health that "it
had been established that there has been no change
in the already low dental caries attack rates in
Stratford ... or in the relatively high rates for
Sarnia".
These two statements are contrary to
the results published by the authors of the study
(Brown et al., 1954b), which showed that in
both the control cities there were statistically
significant differences between the caries attack
rates at successive examinations. Out of the fifteen
comparisons made, only five differences in the rates
were not significant, two changes were significant
and eight changes were definitely significant.
Unexplained significant changes
in controls. All the changes in the caries
attack rates in the control cities which were
reported to be significant are unlikely to be chance
variations; therefore, to what factor or factors
must they be attributed. It is possible that they
were due, in whole or in part, to alterations in the
"weighting", such as were found in the Evanston
study as a result of combining the caries attack
rates of children of different ages (Hill et al.,
1952). However, as the age composition of the groups
was not stated in this study, it cannot be
determined to what degree the data was distorted by
"weighting", a condition which is almost inevitably
present when data drawn from several different
yearly age groups are combined.
Apart from deficiencies which are
found in other studies also, in this trial there is
an absence of any information regarding the caries
attack rates in Brantford and Sarnia, prior to the
fluoridation of the water supply of the former city.
There is also the fact that no explanation was given
by the authors for the significant variations in the
caries rates in the control areas. Therefore, a
marked decrease in dental caries in the test city as
a result of fluoridation cannot be said to have been
established.
THE NEWBURGH STUDY
The fluoridation trial conducted in
Newburgh differs from the other studies in two
important ways:
1. In almost all the comparisons
made, the data obtained were compared with those
from Kingston, the "fluoride-free" control city,
instead of the method used in the other trials, by
which most comparisons were made between the initial
and the latest observations in the test city.
2. The caries attack rates were
stated per 100 erupted teeth, instead of per 100
children or per child. The Evanston study was the
only other one in which the caries rate per 100
erupted teeth was published; Hill et al. in
1955 and 1957a showed this rate, but only for
children aged twelve to fourteen years.
The control city.
Kingston was used as the control area. "Both cities
are situated on the Hudson River about 30 miles
apart. Each has a population of approximately
30,000. The climate of both cities is also similar,
and their water supplies at the outset of this study
were comparable and have remained so, except for the
addition of sodium fluoride to Newburgh's supply" (Ast
et al., 1950). Ast and Chase (1953) added the
information that the two cities had a "comparable
age, sex, and color distribution"; and Schlesinger,
Overton and Chase (1950) mentioned that they "bore a
close resemblance to each other in respect to size
and socio-economic conditions".
Late examination of control city.
In Kingston, as in the other "fluoridefree" control
cities that have been considered, the basic
examinations were not made until after the
fluoridation of the water supply of the test city.
Fluoridation was started in Newburgh on 2 May 1945 (Ast
et al., 1950), but the examinations in
Kingston were not conducted until "Sept., 1945 -
Feb., 1946" (Ast et al, 1950).
Considerably different
composition of waters. In 1950 Ast et al.,
stated that the water supplies of Newburgh and
Kingston "at the outset of this study were
comparable and have remained so, except for the
addition of sodium fluoride to Newburgh's supply."
However, both the source and the composition of the
water supplies of these two cities are different.
The United Kingdom Mission (1953) stated that the
source of Newburgh's water is from "surface water.
Algae growths in spring and summer checked by copper
sulphate blown on the surface of the water as a
powder." The source of Kingston's supply was
described as "Mountain spring impounded. Auxiliary
supply, small spring reservoir" (Lohr and Love,
1954).
In regard to the composition and
other characteristics of these waters, according to
analyses of the finished waters made in February
1952 by the U.S. Geological Survey (Lohr and Love,
1954), in each of the ten items - magnesium, sodium,
potassium, bicarbonate, sulphate, chloride,
dissolved solids, specific conductance, hardness and
alkalinity - the values for the Newburgh water were
at least four times as great as those obtained from
analysis of the Kingston supply. In the very
important matter of the calcium content, the
Newburgh value of 35 ppm (Ca) was more than five
times as large as that of the Kingston one of 6.6
ppm (Ca). Changes in the supplies during the period
of the trial, owing to natural or to
treatment-chemical variations, are unlikely to have
affected these gross differences more than slightly.
 |
|
Figure 7.
The considerably
different calcium and magnesium content and
hardness of the water supplies of Newburgh and
its control city of Kingston, February 1952.
Eight other characteristics of the Newburgh
water were at least four times as large as they
were in Kingston. The authors of this study
stated that these waters "at the outset of this
study were comparable and have remained so" (Ast
et al., 1950).
|
An unsatisfactory control. In
proposing this study, Ast (1943) said: "Much care
must be exercised in the selection of study areas
which should be comparable in as many essential
factors as possible." The first of these factors
which he mentioned was the "chemical composition of
past and present water supply". Therefore it is
surprising that Kingston was selected as the control
city for Newburgh, for it is clear that in this very
important matter the two cities showed considerably
different values. The importance of the close
comparability of the water supplies was emphasized
by the statement of the American Water Works
Association (1949) that the experimental
verification of the fluoride-dental caries
hypothesis "obviously necessitates the use of a
nearby "control" city with a water supply comparable
in all respects to that to which fluoride is being
added."
Variations in methods used.
An outstanding characteristic of this study is the
variation in the methods used, both in gathering the
data and in the presentation of the results. There
were changes in the examiners; on some occasions
clinical examinations only were made and on others
X-rays were also used. The statisticians changed, as
did their presentation of the data in age groups.
The sampling method varied in regard to residence
qualifications, and changes occurred in the age
range of the children who were examined. In one
report data was obtained from selected schools only.
In some examinations the sampling method was
different in the control city from that used in the
test one, All these matters will now be considered
more fully.
The dental findings. These
were published in five papers. Ast, Smith, Wachs and
Cantwell, in 1956, said: "Progress reports were
published after three, four, six and eight years of
fluoride experience in Newburgh" (Ast et al.,
1950, 1951; Ast and Chase, 1953; Ast et al.,
1955). The last-mentioned report (Ast et al.,
1955) "after eight years of fluoride experience"
gave the results obtained during the examinations of
1953-4. The final report, giving the results for
1954-5, apparently one year later than those in the
fourth dental report, was said to show the "dental
findings after ten years of fluoride experience" (Ast
et al., 1956). However, as fluoridation in
Newburgh commenced on "May 2, 1945", and as the
examinations given in the final report were made
"between October 1954 and June 1955" (Ast et al.,
1956), it would appear that, at the most, only a
small part of the data of the final examination was
obtained "after ten years of fluoride experience."
Different examiners used. The
initial examinations in both cities were made by
Finn. "The subsequent examinations in Kingston using
the same technic were made by two dental hygienists"
(Ast et al., 1950). The examinations in
1951-2 were conducted by two examiners, but "Due to
loss of one of the examiners during the examination
year, it was deemed advisable to use only those
examinations made by the remaining examiner in both
cities" (Ast and Chase, 1953). The clinical
examinations in 1953-4, and the final ones, were
made by Wachs (Ast et al., 1955, 1956). These
changes were made despite the fact that in 1943 Ast
said that "the examinations throughout the study
should be made by the same dentist because of the
marked variation in diagnosis of small carious
lesions, pits, and fissures by different dentists."
The clinical examinations were
supplemented by the use of X-rays in the years
1949-50, 1953-4 and 1954-5 (Ast et al.,
1956). In the first of these, which was confined to
children aged seven, nine and eleven years, the
X-rays were taken by a staff dentist and were read
by Ast and Finn (Ast et al., 1951). The next
series was taken by Wachs and was read by Bushel (Ast
et al., 1955); the final X-rays were taken by
Wachs and a staff hygienist, and they were read by
Wachs and Smith (Ast et al., 1956).
Non-comparability of data. In
the last two reports (Ast et al., 1955,
1956), the carious cavities that were detected by
the X-ray were added to those found in the clinical
examinations. Ast et al. in 1955 said that
"the data in this report cannot be compared directly
to those earlier data based on clinical examinations
alone." However, in Table 3 of the 1956 report, the
results of the clinical examination are shown
separately, but a satisfactory comparison with those
obtained in the earlier years is prevented by the
fact that in this report the data were not published
for yearly age groups, but for the age ranges six to
nine and ten to twelve years. Data for the other two
age groups which were shown in the final report,
thirteen to fourteen and sixteen years, were not
published in the previous ones.
The rates for the deciduous teeth
were given in only one report (Ast et al.,
1951).
Examiner variability. The
between and within-examiner variability was not
investigated, although, early in the study, the
importance of this matter was recognized by Ast
et al. (1950) when they stated: "We cannot
entirely rule out the possibility of variation in
the interpretations of the examiners. The fact that
more than one examiner was used might alter the
differences between Newburgh and Kingston to some
extent." In the following year (Ast et al.,
1951) it was stated: "In the present report an
attempt is made to demonstrate that through an
objective roentgenographic examination of the teeth
of selected age groups, the question of examiner
bias in this study is not likely to account for the
differences noted." However, the only data published
were those of the first permanent molars; and the
finding that "the DMF roentgenographic findings of
the first permanent molars only" in selected age
groups shows "consistent differences at each age in
favor of Newburgh" does not provide an estimate of
examiner variability such as could have been
obtained readily by normal statistical methods.
In addition to the changes in the
examiners and in the examination methods, there were
changes in the statisticians. The report after three
years of fluoridation was made in collaboration with
one statistician; those after four, six and eight
years with a different one; and a third statistician
was employed in the preparation of the final report.
Different adjustment procedures.
In most of the tables in this study a "Crude
rate" and an "Adjusted rate" are shown. The
incongruity of making these small adjustments to
rates that were obtained by combining data from
children of considerably different ages does not
appear to have been realized. In some cases even
data from children aged between six and twelve years
were added (Ast et al., 1950, 1951; Ast and
Chase, 1953), the great increase in the caries
attack rate between those ages being ignored. The
adjustments were made (depending on the type of
data) to the tooth population, the first permanent
molar population, or the distribution of children.
In the first three reports of dental findings (Ast
et al., 1950, 1951; Ast and Chase, 1953),
they were all made to the appropriate situation in
Kingston during the 1955-6 examinations, but the
adjustment system was then changed, the crude rates
after eight years of fluoridation being adjusted to
the situation in Kingston in 1953-4 (Ast et al.,
1955), and those shown in the final report to that
present in the control city in 1954-5 (Ast et al.,
1956).
Variations in age groups. In
discussing the Evanston study, it has already been
pointed out that the method of combining the results
of different age groups may result in "weighting"
the data, so that comparisons between the test and
the control cities may be affected. In the examples
given >from other fluoridation trials in which this
method was used, the age groups were consistent from
examination to examination; but in the
Newburgh-Kingston study the groups varied between
examinations, between comparisons made from data
obtained during the same examinations, and even the
age range of the subjects inspected varied from time
to time. In regard to the DMF rate per 100 erupted
permanent teeth, the groups were as follows: 6-7,
8-9,10-12 (Ast et al. 1950); 6, 7, 8, 9, 10,
11 and 12 (Ast et al., 195 1; Ast and Chase,
1953); 6, 7, 8, 9 and 10 (Ast et al., 1955);
and 6-9, 10-12, 13-14 and 16 years (Ast et al.,
1956).
Changes in the age groups were also
made in reporting the other data presented in this
study, but in many cases the groups were different
from those which have just been mentioned.
Grouping of data hinders
comparisons. In the final report, Ast et al.
(1956) said: "The data are combined for six to nine
year old children because these children in Newburgh
had used fluoridated water throughout their lives";
and the age groups ten to twelve years and thirteen
to fourteen years were associated with the tooth
calcification pattern. No explanation has been found
for the grouping used by Ast et al. in 1950,
but this matter will be considered later.
Whatever may have been the reason
for adding the data of children of different ages,
it has the unfortunate result of making it very
difficult to compare the rates which were present in
the test (and in the control) city at different
stages of the trial, especially as, in the 1955
report of Ast et al., the rates obtained from
the clinical examinations were not shown separately
from those computed from the combined clinical and
X-ray results.
"Weighting". Even if the
explanation advanced by the authors of this study is
considered to be a reasonable one, there remains the
danger of "weighting" the data by combining into one
category such divergent material as is provided by
children of different yearly ages. One of the tables
in which obvious "weighting" is seen is Table I of
the first report (Ast et al., 1950),
"weighting" being present in several different
forms. In the control city, the total DMF rate per
100 teeth (ages six to twelve years) is "weighted";
for the total number of teeth examined is made up
(in 1945-6) of only 11 per cent from the six to
seven years age group, with its comparatively low
DMF rate, and of 67 per cent from the ten to twelve
years group with its comparatively high rate (22 per
cent was from age eight to nine years). In the
latest examination shown in that table (1947-8), the
two percentages were 17 and 59 respectively, so that
the comparison between the results of the two
examinations is also "weighted". Similar instances
of "weighting" are also seen in the data >from the
test city; but as these are of a different degree,
the comparison between Newburgh and Kingston is
another instance of "weighting" (Table 1, Ast et
al., 1951). It can be seen that some "weighting"
occurred within the age groups used in the baseline
examinations, principally in the eight to nine years
group in both cities.
Fewer erupted teeth than
expected. In the final report (Ast et al.,
1956), from Table I it can be calculated that the
number of erupted permanent teeth in the six to nine
years group in Newburgh was less than the number
expected, on the assumption that the mean age of
eruption of each type of tooth was the same as in
the children in Kingston. Also, in the ten to twelve
years group (by assuming that in these children at
least the eight incisors and the four first molars
would have erupted) the number of erupted permanent
canines, bicuspids and second molars was fewer in
Newburgh than would be expected. Statistically
speaking, both these differences are highly
significant.
... Continued on next page.
No comments:
Post a Comment