|
Errors and
Omissions in Experimental Trials - 1c
THE BRANTFORD
STUDIES
In
the city of Brantford; Canada, two independent trials were conducted.
1. The City
Health Department Study
In
this study no control procedure was attempted. However, it will be considered
briefly because two pre-fluoridation surveys were made by the school
dental officer and his assistant (Hutton et al., 1954). This is the only one of these five trials in
which more than one pre-fluoridation survey was made in the test city;
and, with the exception of Muskegon, none of the control cities provided
data obtained in successive years from individual yearly age groups.
Were results
combined or averaged? Hutton et al. (1951) stated that
"The results of these two [pre-fluoridation] surveys have been
combined and are shown in Table I." In Tables I and II of the
Ontario Department of Health Report (1956) the rates for those two
surveys were shown separately. With the exception of those of the
nine-year-old children, for both the deciduous and the permanent teeth,
the mean of the two rates for each age is identical (to one decimal
place) with the mean rate computed from the figures of the combined
survey which were supplied by the authors (Hutton et al., 1951, Table I). This result could have arisen only if
(with the exception of the children who were nine years old) the number
of children of the same age examined on both occasions was equal, or
almost exactly so - a most unlikely event; or if the results were not
combined, as stated by the authors, but the rates obtained in 1944 and
1945 were averaged. The United Kingdom Mission (1953) stated that
"the average figures of these two years" were used. If the
rates for the two years were averaged, there were errors in computing the
rates of the nine-year-old children, or errata in one or more of those
three tables. The figures shown in Tables I and II of the Ontario
Department of Health Report (1956) should be treated with caution,
because in both of these the year of fluoridation is stated incorrectly,
and in the former table the "% Reduction Since 1944-45" for age
seven years should be 66, not 51; whereas in the latter one, the "%
Reduction Since 1948" for age nine to eleven years in Stratford and
Sarnia should not have been indicated by dashes, but by five and sixteen
respectively.
2. The
National Health and Welfare Study
The
other study in Brantford was conducted by the Canadian Department of
National Health and Welfare, and was described by the New Zealand
Commission of Inquiry (1957) as "the most complete of the 10-year
North American studies".
Late
commencement. Unfortunately, this trial was
not begun until January 1948, over two and a half years after the
commencement of fluoridation of the Brantford water supply (Brown, 1951).
Such delay must affect the value of this study, unless it is assumed that
the structural theory of reduction of dental caries as a result of the
ingestion of fluorides is correct, and that this is the only way in which
fluorides may affect the incidence of caries. This theory was advanced by
Cox and Levin in 1942, and was widely accepted at the time these trials
were initiated (Dean et al.,
1950; Ast et al., 1950). If
this theory is correct (as was noted in Part One), little change can be
expected in the DMF rates until about six years after the commencement of
the study. It is evident that this theory must still be held to be
correct in some quarters, for the recent report of a dental caries survey
conducted by McCauley and Frazier (1957) stated: "Although fluoridation
of the Baltimore City water supply was begun Nov. 26, 1952, (27 months
before the survey), there was no reason to anticipate substantial change
in the caries experience of these children in this relatively short
period of time." However, even before the commencement of the
Department of National Health and Welfare study in Brantford, the City
Health Department examiner's figures for 1947 showed great reductions in
the DMF rates since the introduction of fluoridation. This result was not
published by Hutton et al.
until 1951, but must have been available to the investigators who
"came to the scientific rescue of the project early in 1948"
(Hutton et al., 1956). Indeed,
in his first report, Brown (195 1) acknowledged the help and advice of
two of the three authors of the City Health Department Report (Hutton et al., 1951).
The control
cities.
The city of Sarnia was chosen as the
"fluoride-free" control, and Stratford as the control city with
a water supply which "contains 1.3 ppm. of fluorine from a natural
source" (Brown, 1951). The reasons for the selection of these cities
were not given, except that it was said: "sarnia and Stratford, two
cities in Western Ontario known to be comparable to Brantford, except for
the fluoride content of their water supplies, agreed to serve as
controls" (Brown et at.\,1954b). Also, Brown, Josie and Stewart
(1953) said that Sarnia was "a city" which has fluoride-free
water and is sufficiently similar in size, location, and other attributes
for purposes of the comparison". The United Kingdom Mission (1953)
stated: "Before this study was undertaken the socio-economic status
of the three communities was examined and found to be reasonably
comparable."
Superior
dental care in Brantford. The United
Kingdom Mission (1953) said: "Brantford, however, over a period of
15 years, has provided more free dental services for children than most
Canadian cities, and this has resulted in the ratio of corrected to total
defects being higher than in either Sarnia or Stratford." It
considered that in Brantford "dental care was outstandingly
good." Also, Brown, in 1952, said:
"the
recordings so far obtained indicate both a higher treatment and an
apparently better oral hygiene status of the Brantford children when
compared with the controls, and it is therefore suggested that caution
should be exercised in the interpretation of the rates shown. The lack of
a pre-fluoridation survey on a comparable basis is a further limiting
factor in interpreting the results."
No
pre-fluoridation survey.
The authors of this Brantford study (Brown et al., 1953) said:
"As
the study does not include a pre-fluoridation survey, the full amount of
benefit which the Brantford teeth have received since fluoridation cannot
be illustrated directly from the data for Brantford. Some idea of the
extent of the benefit can be obtained by comparison with the data for
Sarnia.... By 1948 the Brantford data were not greatly different from
those for Sarnia."
This
remark suggests that the data for the two cities prior to fluoridation in
Brantford were similar, and that this process had had little effect on
the caries rates up to the time of the 1948 examination in Brantford.
Doubtful
comparability of rates. Owing to the
delay in setting up this study, it cannot be established how closely the
dental caries attack rates in Brantford resembled those in Sarnia, at the
time fluoridation was instituted in the former city. There is evidence
that the dental condition of the children in those two cities was not
closely comparable, for Brown et al.
(1953) stated that "even by the time of the first survey, mean tooth
mortality in Brantford was much lower than in Sarnia, for all age
groups."
This
comment implies that, even by the time of the first survey, as a result
of fluoridation the tooth mortality in Brantford had decreased
considerably. This concept is not consistent with the one mentioned in
the last paragraph. At the time of the first examinations, the tooth
mortality in the six to eight years age group was more than four times as
great in Sarnia as it was in Brantford, and in each of the other two age
groups it was almost twice as great (Brown et al., 1953, Table 3.)
|
|
|
Figure
6. The gross differences observed
in the tooth mortality (teeth which are missing or which must be
extracted) in Brantford and its control city of Sarnia, during the
initial examinations. Canadian Department of Health and Welfare study.
|
The influence
of treatment. The fact that such large
differences were reported in tooth mortality rates in the two cities even
in the older age groups suggests that dental treatment in them was
different, and the authors stated that "Both preventive and
treatment measures may have a decided effect on tooth mortality
rates" (Brown et al.,
1953). It may be recalled that the United Kingdom Mission (1953) noted
that Brantford was unusually well provided with free dental services
"and this has resulted in the ratio of corrected to total defects
being higher than in either Sarnia or Stratford."
Tooth
mortality. Brown et al. (1953) said that "there has been a decrease in
tooth mortality in Brantford between successive surveys"; but, in
fact, the 1953 rate (Table 3) in the children aged six to eight years was
the highest up to that time. This statement was corrected in the next report
(Brown et al., 1954b) by
prefixing "in almost all cases" to the previous statement. In
this connection, the authors remarked in 1953 that, "as well as the
fluoridation of the Brantford-water supply, other factors such as
differences in preventive or treatment measures are probably affecting
the Brantford position." There appears to be no reason why those of
the control cities should not have been similarly affected.
Differences
in oral hygiene. Additional evidence which
suggests that a difference existed between the dental condition of the
children in Brantford and that of children in the control cities is
provided by the data with regard to oral hygiene. Brown et al. (1954b) stated:
"Classification
and recording of oral hygiene was undertaken because it was considered
that marked differences in oral hygiene as between the test and control
groups might conceivably affect the findings--or at least might be taken
into consideration as a modifying factor, although not a strictly
measurable one. However, the figures here suggest that, since 1948,
differences in oral hygiene status could not have been a major factor in
either the caries level changes within Brantford or the caries level
differences between the control cities."
As
no comparisons were made between the control cities, the last phrase of
the quotation is thought to refer to the caries level differences between
Brantford and each of the control cities. The authors' Table 11 indicates
that, in the first examinations, in the test city the percentage of
subjects who had a good oral hygiene status was almost twice as great as
that present in children in both the control ones; these were, Brantford
34.3 per cent, Sarnia 19.7 per cent and Stratford 17.8 per cent.
Considerable differences between the oral hygiene status of the children
in the test and the control cities were also recorded during the later
examinations. These were clearly "marked differences" though
the authors did not consider them important.
The
concentration of fluorides. Brown et al. (1954b) stated:
"The
Brantford Fluoridation Caries Study was undertaken with a view to finding
out whether or not the raising of the fluoride content of a previously
fluoride-free water supply to 1 part per million, by the mechanical
addition of sodium fluoride, would reduce the incidence of dental caries
to that which obtains where water supplies derive about 1 part per
million of fluoride from deposits in the earth."
A
fundamental requirement of a test of this nature is that the water supply
of the control city should contain the same concentration of fluorides as
that of the test one, but Brown, McLaren, Josie and Stewart (1956)
reported: "The Stratford water supply contains a concentration of
fluoride which is 60% higher than that used in Brantford."
Differences of
opinion. Several different statements
were made regarding the concentrations of fluorides which were present in
the water supplies of Brantford and Stratford.
1. Brantford.
The New Zealand Commission of Inquiry (1957) said that the water supply
of Brantford was "raised to 1.2 ppm. in 1945". The authors of
the City Health Department study (Hutton et al., 1951) stated that in February 1949, "the dosage
was raised to produce 1.20 ppm."; but in 1954 they stated that
"The fluoride content of the finished water is maintained at 1
ppm." In reporting the National Health and Welfare Study, Brown
(1952) stated that "a fluorine concentration of between 1.0 and 1.2
ppm. has been maintained in the water supply continuously" since
June 1945; and in 1956 Brown et al.
said: "Brantford has had more than 10 years of experience with 1
part per million fluoride in its water supply." These statements
that the fluoride content was "maintained" at "1 ppm"
and "between 1.0 and 1.2 ppm" should be considered in
conjunction with that of the United Kingdom Mission (1953): "For
example, in 1951 the average for the year was 1.2 ppm with a variation
between 0.75 and 1.45 ppm., however, the figures below 1.1 ppm and above
1. 3 ppm were few in number."
2. Stratford.
The supply of Stratford was stated to have 1.2 ppm of natural fluoride in
its domestic water (Ontario Department of Health, 1956; New Zealand
Commission of Inquiry, 1957). However, in reporting the National Health
and Welfare study in 1951 and 1952, Brown stated that it "contains
1.3 ppm. of fluorine"; and, with his co-workers, the following year
said that "in Stratford where the water supply, obtained from deep
wells, has contained 1.3 ppm." (fluoride) "from natural
deposits continuously since 1917" (Brown et al., 1953). In their next report dealing with dental
caries (Brown et al., 1954b),
the figure stated was "1.3 to 1.6 parts per million of
fluoride"; but in the following report these authors (Brown et al., 1956) said that this water
supply "contained 1.6 parts per million of fluoride since
1917", and also that 1.6 ppm fluoride content "has been in
continuous use for thirt yeight years."
These
different statements, although strange, may be considered to be
unimportant from the practical point of view, but very small changes in
the fluoride content of the water may have considerable effects, as
reports by Deatherage (1942) and Galagan and Lamson (1953) indicated. The
latter authors found that "In water supplies of the Arizona
communities studied, concentrations of fluoride above 0.8 ppm resulted in
objectionable dental fluorosis; concentrations of 0.6 to 0.8 ppm resulted
in an occasional diagnosis of fluorosis; concentrations below 0.6 ppm did
not cause objectionable fluorosis." The mean temperatures of these
communities were between 67 and 72 degrees Fahrenheit.
Three
misleading statements. In the
summary of the 1954b report of the National Health and Welfare study,
Brown et al. stated that during
the period 1948-54 "dental caries experience of children in the two
control cities, on the other hand, either has remained at about the 1948
levels, or has increased slightly, at all ages studied." A similar
statement was made by them in the 1955 Report (Brown et al., 1956): "During that time [more than ten years] a
very important, statistically significant reduction in tooth decay has
occurred in all the age groups studied, while in the two control cities
of Sarnia and Stratford it has either remained at about the same level or
increased somewhat." The last sentence contains three misleading
statements about the control cities:
(a)
As this study did not commence until 1948 with "examinations in
Sarnia beginning in March of that year, and in Stratford in October"
(Brown, 1952), and as fluoridation in Brantford commenced in June 1945 (Hutton
et al., 195 1), no information
is available with regard to the prevalence of "tooth decay" in
Sarnia during the first two and three-quarter years, or in Stratford for
the first three and a quarter years of the ten-year period of
fluoridation which these authors were discussing in their 1955 Report.
Therefore, it cannot be known whether this condition "remained at
about the same level" in the control cities during the early years
of fluoridation in Brantford.
(b)
No remarks were made in the context of this statement in this (or the
previous) paper which suggested that reference was being made to the
permanent teeth only, but these statements are not correct for the
deciduous teeth. (However, in the former paper, a similar statement to
that made in the summary was also made under the heading "Mean DMF
Permanent Teeth".) Decreases in the df (decayed, filled deciduous
teeth) rate were seen in the nine to eleven years group in both Sarnia
and Stratford, that in the former city being shown by these authors to be
statistically significant (Brown et
al., 1954b, Table 10).
(c)
In both cities the mean rates of DMF and df teeth per child showed
changes which were said to be statistically significant (Brown et al., 1954b).
Omission of
decreases. The decreases in the caries
attack rates of the deciduous teeth, which have been mentioned above,
were also omitted in Table II of the Report to the Minister of Health,
Province of Ontario, Canada, by the Division of Medical Statistics,
Ontario Department of Health, which was made in 1955 (Ontario Department
of Health, 1956). Under the heading "% Reduction Since 1948",
these decreases were not shown, but instead, in the appropriate positions
dashes were printed, despite the fact that in Sarnia the percentage reduction
(determined by the method commonly used in these studies) was 16 per
cent, almost as great as that of 18 per cent shown for the same age group
in the test city; furthermore, this reduction in Sarnia was stated by the
authors (Brown et al., 1954b) to
be statistically significant.
Different
rates reported. It should be noted that
the deficiency in the data of the National Health and Welfare study,
owing to its late commencement, could not be decreased by comparing the
rates obtained by its examiners with those reported by the City Health
Department examiner, because of the considerably lower rates recorded by
the last-mentioned examiner when impecting similar groups of children.
For instance, in their examination in 1948, for children aged six to eight,
nine to eleven and twelve to fourteen years, Brown et al. (1953) obtained rates of 1.41, 4.07 and 7.68
respectively for the permanent teeth, compared with rates of 0.84, 3.37
and 6.11, for the same age groups of children, in the same city and in
the same year, obtained by the City Health Department examiner
(calculated from Table IV, Hutton et
al., 1951).
Significant
fluctuations in controls. In the two
control cities "where it is presumed that there has been no
appreciable change in either preventive or treatment services"
(United Kingdom Mission Report; 1953), it can be seen in Tables 4, 6, 8
and 10 published by Brown et al.
(1954b) that some considerable fluctuations in the caries attack rates
were recorded; more than half of the inter-year differences in each of
the control cities being shown to be statistically significant. However,
in the text it was stated that the "dental caries experience of
children in the two control cities . . . either has remained at about the
1948 levels, or has increased slightly, at all ages studied."
(1)
Sarnia. In this city the changes between examinations of the rates for
the deciduous teeth were not very marked, but there was a significant one
between 1948 and 1954 in the nine to eleven years age group. However, in
the DMF permanent teeth, there were four definitely significant (three
standard error level) and one significant change in the nine comparisons
made. In regard to the first permanent molars, there were six significant
(including three definitely significant) alterations in the rates, in the
nine comparisons made (Brown et al.,
1954b).
(2)
Stratford. In this city, the rate of df teeth per child showed a
significant difference in one case out of the four comparisons made between
successive examinations (Brown et
al., 1954b). In the DMF permanent teeth per child, the results of the
four examinations were: 0.41, 0.75, 0.47 and 0.67 for the six to eight
years group; 1.13, 1.76, 1.46 and 1.89 for the nine to eleven years age
group; 2.55, 3.12, 3.02 and 3.77 for the twelve to fourteen years age
group (Brown et al., 1954b,
1956). These variations between examinations were so large that five out
of the six comparisons made (in the 1954b report) between successive
examinations were said to be statistically significant, four of them
being at the three standard error level. In the last report published
(1956), Brown et al. abandoned
the method which they had used in the two previous ones, that of showing
the standard error of the mean values of the DMF rates, and of making
"Inter-City" and "Inter-Year" comparisons (Brown et al., 1953, 1954b). Therefore it
was not stated whether the differences between the 1954 and the 1955 DMF
rates in Stratford were significant, but it can be seen that they were
marked; the difference of 0.75 in the twelve to fourteen years group
being considerably larger than any of those stated in the 1954b report to
be significant differences between various examinations in that city.
When the DMF rates for the first permanent molars are considered, similar
marked changes are seen, and in five of the six comparisons the
differences were significant (four definitely so).
Larger
"percentage" changes in a control.
If one resorts to the method commonly used in these trials - that of
expressing the alteration in the DMF rate as a percentage of the original
rate - these unexplained increases in the control city of Stratford
between 1948 and 1955, although they were described as "no
change" (Ontario Department of Health, 1956), and as "a
slightly higher prevalence of dental caries in 1955, over the 1948
levels" (Brown et al.,
1956), are found to be 63 per cent, 67 per cent and 48 per cent, for the
six to eight, nine to eleven and twelve to fourteen years age groups
respectively. In each case these percentage changes are considerably
larger than those of 51 per cent, 44 per cent and 37 per cent which can
be computed from the data reported for Brantford. The last-mentioned
changes were attributed to fluoridation, and each was stated to indicate
"a very important, statistically significant reduction in tooth
decay" (Brown et al.,
1956).
This
is just one instance of the strange results which are obtained when this
method of calculation is used. It should be realized that it was the one
most commonly employed in fluoridation trials, and was used in
formulating the often-expressed claim that (as stated by Arnold et al., 1956): "In children
born since fluoridation was put into effect, the caries rate for the
permanent teeth was reduced on the average by about 60 per cent."
The recent World Health Organization Press Release (WHO/45, 4 September
1957) stated - with no mention of age - "The prevalence of dental
caries in the permanent teeth of children decreased some 60 percent".
A smaller
"percentage decrease" after long fluoridation.
The "percentage decreases" which have just been mentioned (51
per cent, 44 per cent and 37 per cent, calculated by the method described
in the last paragraph) were not stated in the 1956 report of Brown et al., but the figure of 51 per
cent for the six to eight years age group is considerably less impressive
than the figure of "approximately 69%" published in the 1954b
report from this study. Although the final report (1956) gave the rates
for 1948 and 1955 only, and therefore did not show the fluctuations
between examinations, from the 1954b and 1956 reports of Brown et al. it is seen that the marked
change in the "percentage" decrease which has just been
mentioned was due to the DMF rate in Brantford in 1955, for this age
group, being the highest seen since 1951. Ignoring the fact that in
"children born subsequent to fluoridation" the
"decrease" in the DMF rate had dropped to only 51 per cent, the
authors stated in the final sentence of their final report (1956):
"For every three decayed teeth they would have had, they have only
one."
More
misleading comments. Turning from the reports
made by the authors of this study about the control cities to some of the
comments made by others, it is seen that these are even more misleading.
Only two will be mentioned. Martin (1956) stated that during "the
1948-54 period" the "DMF figures for the two control areas have
remained at 1948 levels." The authors of the Ontario Department of
Health Report (1956) went so far as to state to their Minister of Health
that "it had been established that there has been no change in the
already low dental caries attack rates in Stratford ... or in the
relatively high rates for Sarnia".
These
two statements are contrary to the results published by the authors of
the study (Brown et al.,
1954b), which showed that in both the control cities there were
statistically significant differences between the caries attack rates at
successive examinations. Out of the fifteen comparisons made, only five
differences in the rates were not significant, two changes were
significant and eight changes were definitely significant.
Unexplained
significant changes in controls. All the
changes in the caries attack rates in the control cities which were
reported to be significant are unlikely to be chance variations;
therefore, to what factor or factors must they be attributed. It is
possible that they were due, in whole or in part, to alterations in the
"weighting", such as were found in the Evanston study as a result
of combining the caries attack rates of children of different ages (Hill et al., 1952). However, as the age
composition of the groups was not stated in this study, it cannot be
determined to what degree the data was distorted by
"weighting", a condition which is almost inevitably present
when data drawn from several different yearly age groups are combined.
Apart
from deficiencies which are found in other studies also, in this trial
there is an absence of any information regarding the caries attack rates
in Brantford and Sarnia, prior to the fluoridation of the water supply of
the former city. There is also the fact that no explanation was given by
the authors for the significant variations in the caries rates in the
control areas. Therefore, a marked decrease in dental caries in the test
city as a result of fluoridation cannot be said to have been established.
THE NEWBURGH
STUDY
The
fluoridation trial conducted in Newburgh differs from the other studies
in two important ways:
1.
In almost all the comparisons made, the data obtained were compared with
those from Kingston, the "fluoride-free" control city, instead
of the method used in the other trials, by which most comparisons were
made between the initial and the latest observations in the test city.
2.
The caries attack rates were stated per 100 erupted teeth, instead of per
100 children or per child. The Evanston study was the only other one in
which the caries rate per 100 erupted teeth was published; Hill et al. in 1955 and 1957a showed
this rate, but only for children aged twelve to fourteen years.
The control
city.
Kingston was used as the control area. "Both
cities are situated on the Hudson River about 30 miles apart. Each has a
population of approximately 30,000. The climate of both cities is also
similar, and their water supplies at the outset of this study were
comparable and have remained so, except for the addition of sodium
fluoride to Newburgh's supply" (Ast et al., 1950). Ast and Chase (1953) added the information
that the two cities had a "comparable age, sex, and color
distribution"; and Schlesinger, Overton and Chase (1950) mentioned
that they "bore a close resemblance to each other in respect to size
and socio-economic conditions".
Late
examination of control city. In Kingston,
as in the other "fluoridefree" control cities that have been
considered, the basic examinations were not made until after the
fluoridation of the water supply of the test city. Fluoridation was
started in Newburgh on 2 May 1945 (Ast et al., 1950), but the examinations in Kingston were not
conducted until "Sept., 1945 - Feb., 1946" (Ast et al, 1950).
Considerably
different composition of waters. In 1950 Ast et al., stated that the water
supplies of Newburgh and Kingston "at the outset of this study were comparable
and have remained so, except for the addition of sodium fluoride to
Newburgh's supply." However, both the source and the composition of
the water supplies of these two cities are different. The United Kingdom
Mission (1953) stated that the source of Newburgh's water is from
"surface water. Algae growths in spring and summer checked by copper
sulphate blown on the surface of the water as a powder." The source
of Kingston's supply was described as "Mountain spring impounded.
Auxiliary supply, small spring reservoir" (Lohr and Love, 1954).
In
regard to the composition and other characteristics of these waters,
according to analyses of the finished waters made in February 1952 by the
U.S. Geological Survey (Lohr and Love, 1954), in each of the ten items -
magnesium, sodium, potassium, bicarbonate, sulphate, chloride, dissolved
solids, specific conductance, hardness and alkalinity - the values for
the Newburgh water were at least four times as great as those obtained
from analysis of the Kingston supply. In the very important matter of the
calcium content, the Newburgh value of 35 ppm (Ca) was more than five
times as large as that of the Kingston one of 6.6 ppm (Ca). Changes in
the supplies during the period of the trial, owing to natural or to
treatment-chemical variations, are unlikely to have affected these gross
differences more than slightly.
|
|
|
Figure
7. The considerably different
calcium and magnesium content and hardness of the water supplies of
Newburgh and its control city of Kingston, February 1952. Eight other
characteristics of the Newburgh water were at least four times as large
as they were in Kingston. The authors of this study stated that these
waters "at the outset of this study were comparable and have
remained so" (Ast et al.,
1950).
|
An
unsatisfactory control. In proposing
this study, Ast (1943) said: "Much care must be exercised in the
selection of study areas which should be comparable in as many essential
factors as possible." The first of these factors which he mentioned
was the "chemical composition of past and present water supply".
Therefore it is surprising that Kingston was selected as the control city
for Newburgh, for it is clear that in this very important matter the two
cities showed considerably different values. The importance of the close
comparability of the water supplies was emphasized by the statement of
the American Water Works Association (1949) that the experimental
verification of the fluoride-dental caries hypothesis "obviously
necessitates the use of a nearby "control" city with a water
supply comparable in all respects to that to which fluoride is being
added."
Variations in
methods used.
An outstanding characteristic of this study is the variation in the
methods used, both in gathering the data and in the presentation of the
results. There were changes in the examiners; on some occasions clinical
examinations only were made and on others X-rays were also used. The
statisticians changed, as did their presentation of the data in age
groups. The sampling method varied in regard to residence qualifications,
and changes occurred in the age range of the children who were examined.
In one report data was obtained from selected schools only. In some
examinations the sampling method was different in the control city from
that used in the test one, All these matters will now be considered more
fully.
The dental
findings. These were published in five
papers. Ast, Smith, Wachs and Cantwell, in 1956, said: "Progress
reports were published after three, four, six and eight years of fluoride
experience in Newburgh" (Ast et
al., 1950, 1951; Ast and Chase, 1953; Ast et al., 1955). The last-mentioned report (Ast et al., 1955) "after eight
years of fluoride experience" gave the results obtained during the
examinations of 1953-4. The final report, giving the results for 1954-5,
apparently one year later than those in the fourth dental report, was
said to show the "dental findings after ten years of fluoride
experience" (Ast et al.,
1956). However, as fluoridation in Newburgh commenced on "May 2,
1945", and as the examinations given in the final report were made
"between October 1954 and June 1955" (Ast et al., 1956), it would appear that, at the most, only a
small part of the data of the final examination was obtained "after
ten years of fluoride experience."
Different
examiners used. The initial examinations
in both cities were made by Finn. "The subsequent examinations in
Kingston using the same technic were made by two dental hygienists"
(Ast et al., 1950). The
examinations in 1951-2 were conducted by two examiners, but "Due to
loss of one of the examiners during the examination year, it was deemed
advisable to use only those examinations made by the remaining examiner
in both cities" (Ast and Chase, 1953). The clinical examinations in
1953-4, and the final ones, were made by Wachs (Ast et al., 1955, 1956). These changes were made despite the fact
that in 1943 Ast said that "the examinations throughout the study
should be made by the same dentist because of the marked variation in
diagnosis of small carious lesions, pits, and fissures by different
dentists."
The
clinical examinations were supplemented by the use of X-rays in the years
1949-50, 1953-4 and 1954-5 (Ast et
al., 1956). In the first of these, which was confined to children
aged seven, nine and eleven years, the X-rays were taken by a staff
dentist and were read by Ast and Finn (Ast et al., 1951). The next series was taken by Wachs and was
read by Bushel (Ast et al.,
1955); the final X-rays were taken by Wachs and a staff hygienist, and
they were read by Wachs and Smith (Ast et al., 1956).
Non-comparability
of data. In the last two reports (Ast et al., 1955, 1956), the carious
cavities that were detected by the X-ray were added to those found in the
clinical examinations. Ast et al.
in 1955 said that "the data in this report cannot be compared
directly to those earlier data based on clinical examinations
alone." However, in Table 3 of the 1956 report, the results of the
clinical examination are shown separately, but a satisfactory comparison
with those obtained in the earlier years is prevented by the fact that in
this report the data were not published for yearly age groups, but for
the age ranges six to nine and ten to twelve years. Data for the other
two age groups which were shown in the final report, thirteen to fourteen
and sixteen years, were not published in the previous ones.
The
rates for the deciduous teeth were given in only one report (Ast et al., 1951).
Examiner
variability. The between and within-examiner
variability was not investigated, although, early in the study, the
importance of this matter was recognized by Ast et al. (1950) when they stated: "We cannot entirely rule
out the possibility of variation in the interpretations of the examiners.
The fact that more than one examiner was used might alter the differences
between Newburgh and Kingston to some extent." In the following year
(Ast et al., 1951) it was
stated: "In the present report an attempt is made to demonstrate
that through an objective roentgenographic examination of the teeth of
selected age groups, the question of examiner bias in this study is not
likely to account for the differences noted." However, the only data
published were those of the first permanent molars; and the finding that
"the DMF roentgenographic findings of the first permanent molars
only" in selected age groups shows "consistent differences at
each age in favor of Newburgh" does not provide an estimate of
examiner variability such as could have been obtained readily by normal
statistical methods.
In
addition to the changes in the examiners and in the examination methods,
there were changes in the statisticians. The report after three years of
fluoridation was made in collaboration with one statistician; those after
four, six and eight years with a different one; and a third statistician
was employed in the preparation of the final report.
Different
adjustment procedures. In most of
the tables in this study a "Crude rate" and an "Adjusted
rate" are shown. The incongruity of making these small adjustments
to rates that were obtained by combining data from children of
considerably different ages does not appear to have been realized. In
some cases even data from children aged between six and twelve years were
added (Ast et al., 1950, 1951; Ast
and Chase, 1953), the great increase in the caries attack rate between
those ages being ignored. The adjustments were made (depending on the
type of data) to the tooth population, the first permanent molar
population, or the distribution of children. In the first three reports
of dental findings (Ast et al.,
1950, 1951; Ast and Chase, 1953), they were all made to the appropriate
situation in Kingston during the 1955-6 examinations, but the adjustment
system was then changed, the crude rates after eight years of
fluoridation being adjusted to the situation in Kingston in 1953-4 (Ast et al., 1955), and those shown in
the final report to that present in the control city in 1954-5 (Ast et al., 1956).
Variations in
age groups. In discussing the Evanston study,
it has already been pointed out that the method of combining the results
of different age groups may result in "weighting" the data, so
that comparisons between the test and the control cities may be affected.
In the examples given >from other fluoridation trials in which this
method was used, the age groups were consistent from examination to
examination; but in the Newburgh-Kingston study the groups varied between
examinations, between comparisons made from data obtained during the same
examinations, and even the age range of the subjects inspected varied
from time to time. In regard to the DMF rate per 100 erupted permanent
teeth, the groups were as follows: 6-7, 8-9,10-12 (Ast et al. 1950); 6, 7, 8, 9, 10, 11
and 12 (Ast et al., 195 1; Ast
and Chase, 1953); 6, 7, 8, 9 and 10 (Ast et al., 1955); and 6-9, 10-12, 13-14 and 16 years (Ast et al., 1956).
Changes
in the age groups were also made in reporting the other data presented in
this study, but in many cases the groups were different from those which
have just been mentioned.
Grouping of
data hinders comparisons. In the final
report, Ast et al. (1956) said:
"The data are combined for six to nine year old children because
these children in Newburgh had used fluoridated water throughout their
lives"; and the age groups ten to twelve years and thirteen to
fourteen years were associated with the tooth calcification pattern. No
explanation has been found for the grouping used by Ast et al. in 1950, but this matter
will be considered later.
Whatever
may have been the reason for adding the data of children of different
ages, it has the unfortunate result of making it very difficult to
compare the rates which were present in the test (and in the control)
city at different stages of the trial, especially as, in the 1955 report
of Ast et al., the rates
obtained from the clinical examinations were not shown separately from
those computed from the combined clinical and X-ray results.
"Weighting".
Even if the explanation advanced by the authors of
this study is considered to be a reasonable one, there remains the danger
of "weighting" the data by combining into one category such
divergent material as is provided by children of different yearly ages.
One of the tables in which obvious "weighting" is seen is Table
I of the first report (Ast et al.,
1950), "weighting" being present in several different forms. In
the control city, the total DMF rate per 100 teeth (ages six to twelve
years) is "weighted"; for the total number of teeth examined is
made up (in 1945-6) of only 11 per cent from the six to seven years age
group, with its comparatively low DMF rate, and of 67 per cent from the
ten to twelve years group with its comparatively high rate (22 per cent
was from age eight to nine years). In the latest examination shown in that
table (1947-8), the two percentages were 17 and 59 respectively, so that
the comparison between the results of the two examinations is also
"weighted". Similar instances of "weighting" are also
seen in the data >from the test city; but as these are of a different
degree, the comparison between Newburgh and Kingston is another instance
of "weighting" (Table 1, Ast et al., 1951). It can be seen that some "weighting"
occurred within the age groups used in the baseline examinations,
principally in the eight to nine years group in both cities.
Fewer erupted
teeth than expected. In the final report (Ast
et al., 1956), from Table I it
can be calculated that the number of erupted permanent teeth in the six
to nine years group in Newburgh was less than the number expected, on the
assumption that the mean age of eruption of each type of tooth was the
same as in the children in Kingston. Also, in the ten to twelve years
group (by assuming that in these children at least the eight incisors and
the four first molars would have erupted) the number of erupted permanent
canines, bicuspids and second molars was fewer in Newburgh than would be
expected. Statistically speaking, both these differences are highly
significant.
... Continued
on next page.
|
No comments:
Post a Comment