Errors
and Omissions in Experimental Trials - 1c
THE
BRANTFORD STUDIES
In the city of Brantford; Canada, two independent
trials were conducted.
1. The
City Health Department Study
In this study no control procedure was attempted.
However, it will be considered briefly because two pre-fluoridation surveys
were made by the school dental officer and his assistant (Hutton et al., 1954). This is the only one of
these five trials in which more than one pre-fluoridation survey was made in
the test city; and, with the exception of Muskegon, none of the control cities
provided data obtained in successive years from individual yearly age groups.
Were
results combined or averaged?
Hutton et al. (1951) stated that
"The results of these two [pre-fluoridation] surveys have been combined
and are shown in Table I." In Tables I and II of the Ontario Department of
Health Report (1956) the rates for those two surveys were shown separately.
With the exception of those of the nine-year-old children, for both the
deciduous and the permanent teeth, the mean of the two rates for each age is
identical (to one decimal place) with the mean rate computed from the figures
of the combined survey which were supplied by the authors (Hutton et al., 1951, Table I). This result
could have arisen only if (with the exception of the children who were nine
years old) the number of children of the same age examined on both occasions
was equal, or almost exactly so - a most unlikely event; or if the results were
not combined, as stated by the authors, but the rates obtained in 1944 and 1945
were averaged. The United Kingdom Mission (1953) stated that "the average
figures of these two years" were used. If the rates for the two years were
averaged, there were errors in computing the rates of the nine-year-old
children, or errata in one or more of those three tables. The figures shown in
Tables I and II of the Ontario Department of Health Report (1956) should be
treated with caution, because in both of these the year of fluoridation is
stated incorrectly, and in the former table the "% Reduction Since
1944-45" for age seven years should be 66, not 51; whereas in the latter
one, the "% Reduction Since 1948" for age nine to eleven years in
Stratford and Sarnia should not have been indicated by dashes, but by five and
sixteen respectively.
2.
The National Health and Welfare Study
The other study in Brantford was
conducted by the Canadian Department of National Health and Welfare, and was
described by the New Zealand Commission of Inquiry (1957) as "the most
complete of the 10-year North American studies".
Late
commencement. Unfortunately, this trial was not
begun until January 1948, over two and a half years after the commencement of
fluoridation of the Brantford water supply (Brown, 1951). Such delay must
affect the value of this study, unless it is assumed that the structural theory
of reduction of dental caries as a result of the ingestion of fluorides is
correct, and that this is the only way in which fluorides may affect the
incidence of caries. This theory was advanced by Cox and Levin in 1942, and was
widely accepted at the time these trials were initiated (Dean et al., 1950; Ast et al., 1950). If this theory is correct (as was noted in Part
One), little change can be expected in the DMF rates until about six years
after the commencement of the study. It is evident that this theory must still
be held to be correct in some quarters, for the recent report of a dental
caries survey conducted by McCauley and Frazier (1957) stated: "Although
fluoridation of the Baltimore City water supply was begun Nov. 26, 1952, (27
months before the survey), there was no reason to anticipate substantial change
in the caries experience of these children in this relatively short period of
time." However, even before the commencement of the Department of National
Health and Welfare study in Brantford, the City Health Department examiner's
figures for 1947 showed great reductions in the DMF rates since the introduction
of fluoridation. This result was not published by Hutton et al. until 1951, but must have been available to the
investigators who "came to the scientific rescue of the project early in
1948" (Hutton et al., 1956).
Indeed, in his first report, Brown (195 1) acknowledged the help and advice of
two of the three authors of the City Health Department Report (Hutton et al., 1951).
The
control cities.
The city of Sarnia was chosen as the
"fluoride-free" control, and Stratford as the control city with a
water supply which "contains 1.3 ppm. of fluorine from a natural
source" (Brown, 1951). The reasons for the selection of these cities were
not given, except that it was said: "sarnia and Stratford, two cities in
Western Ontario known to be comparable to Brantford, except for the fluoride
content of their water supplies, agreed to serve as controls" (Brown et
at.\,1954b). Also, Brown, Josie and Stewart (1953) said that Sarnia was "a
city" which has fluoride-free water and is sufficiently similar in size,
location, and other attributes for purposes of the comparison". The United
Kingdom Mission (1953) stated: "Before this study was undertaken the
socio-economic status of the three communities was examined and found to be
reasonably comparable."
Superior
dental care in Brantford. The United Kingdom Mission
(1953) said: "Brantford, however, over a period of 15 years, has provided
more free dental services for children than most Canadian cities, and this has
resulted in the ratio of corrected to total defects being higher than in either
Sarnia or Stratford." It considered that in Brantford "dental care
was outstandingly good." Also, Brown, in 1952, said:
"the recordings so far obtained
indicate both a higher treatment and an apparently better oral hygiene status
of the Brantford children when compared with the controls, and it is therefore
suggested that caution should be exercised in the interpretation of the rates
shown. The lack of a pre-fluoridation survey on a comparable basis is a further
limiting factor in interpreting the results."
No
pre-fluoridation survey.
The authors of this Brantford study (Brown et al., 1953) said:
"As the study does not include a
pre-fluoridation survey, the full amount of benefit which the Brantford teeth
have received since fluoridation cannot be illustrated directly from the data
for Brantford. Some idea of the extent of the benefit can be obtained by
comparison with the data for Sarnia.... By 1948 the Brantford data were not
greatly different from those for Sarnia."
This remark suggests that the data for
the two cities prior to fluoridation in Brantford were similar, and that this
process had had little effect on the caries rates up to the time of the 1948
examination in Brantford.
Doubtful
comparability of rates. Owing to the delay in setting
up this study, it cannot be established how closely the dental caries attack
rates in Brantford resembled those in Sarnia, at the time fluoridation was
instituted in the former city. There is evidence that the dental condition of
the children in those two cities was not closely comparable, for Brown et al. (1953) stated that "even by
the time of the first survey, mean tooth mortality in Brantford was much lower
than in Sarnia, for all age groups."
This comment implies that, even by the
time of the first survey, as a result of fluoridation the tooth mortality in
Brantford had decreased considerably. This concept is not consistent with the
one mentioned in the last paragraph. At the time of the first examinations, the
tooth mortality in the six to eight years age group was more than four times as
great in Sarnia as it was in Brantford, and in each of the other two age groups
it was almost twice as great (Brown et al.,
1953, Table 3.)
 |
Figure
6. The gross differences observed in the tooth mortality
(teeth which are missing or which must be extracted) in Brantford and its
control city of Sarnia, during the initial examinations. Canadian Department
of Health and Welfare study.
|
The
influence of treatment. The fact that such large
differences were reported in tooth mortality rates in the two cities even in
the older age groups suggests that dental treatment in them was different, and
the authors stated that "Both preventive and treatment measures may have a
decided effect on tooth mortality rates" (Brown et al., 1953). It may be recalled that the United Kingdom Mission
(1953) noted that Brantford was unusually well provided with free dental
services "and this has resulted in the ratio of corrected to total defects
being higher than in either Sarnia or Stratford."
Tooth
mortality. Brown et al. (1953) said that "there has been a decrease in tooth
mortality in Brantford between successive surveys"; but, in fact, the 1953
rate (Table 3) in the children aged six to eight years was the highest up to
that time. This statement was corrected in the next report (Brown et al., 1954b) by prefixing "in
almost all cases" to the previous statement. In this connection, the
authors remarked in 1953 that, "as well as the fluoridation of the
Brantford-water supply, other factors such as differences in preventive or
treatment measures are probably affecting the Brantford position." There
appears to be no reason why those of the control cities should not have been
similarly affected.
Differences
in oral hygiene. Additional evidence which suggests that
a difference existed between the dental condition of the children in Brantford
and that of children in the control cities is provided by the data with regard
to oral hygiene. Brown et al. (1954b)
stated:
"Classification and recording of
oral hygiene was undertaken because it was considered that marked differences
in oral hygiene as between the test and control groups might conceivably affect
the findings--or at least might be taken into consideration as a modifying
factor, although not a strictly measurable one. However, the figures here
suggest that, since 1948, differences in oral hygiene status could not have
been a major factor in either the caries level changes within Brantford or the
caries level differences between the control cities."
As no comparisons were made between the
control cities, the last phrase of the quotation is thought to refer to the
caries level differences between Brantford and each of the control cities. The
authors' Table 11 indicates that, in the first examinations, in the test city
the percentage of subjects who had a good oral hygiene status was almost twice
as great as that present in children in both the control ones; these were,
Brantford 34.3 per cent, Sarnia 19.7 per cent and Stratford 17.8 per cent. Considerable
differences between the oral hygiene status of the children in the test and the
control cities were also recorded during the later examinations. These were
clearly "marked differences" though the authors did not consider them
important.
The
concentration of fluorides. Brown et al. (1954b) stated:
"The Brantford Fluoridation Caries
Study was undertaken with a view to finding out whether or not the raising of
the fluoride content of a previously fluoride-free water supply to 1 part per
million, by the mechanical addition of sodium fluoride, would reduce the
incidence of dental caries to that which obtains where water supplies derive
about 1 part per million of fluoride from deposits in the earth."
A fundamental requirement of a test of
this nature is that the water supply of the control city should contain the
same concentration of fluorides as that of the test one, but Brown, McLaren,
Josie and Stewart (1956) reported: "The Stratford water supply contains a
concentration of fluoride which is 60% higher than that used in
Brantford."
Differences
of opinion. Several different statements were made
regarding the concentrations of fluorides which were present in the water
supplies of Brantford and Stratford.
1.
Brantford. The New Zealand Commission of Inquiry
(1957) said that the water supply of Brantford was "raised to 1.2 ppm. in
1945". The authors of the City Health Department study (Hutton et al., 1951) stated that in February
1949, "the dosage was raised to produce 1.20 ppm."; but in 1954 they
stated that "The fluoride content of the finished water is maintained at 1
ppm." In reporting the National Health and Welfare Study, Brown (1952)
stated that "a fluorine concentration of between 1.0 and 1.2 ppm. has been
maintained in the water supply continuously" since June 1945; and in 1956
Brown et al. said: "Brantford
has had more than 10 years of experience with 1 part per million fluoride in
its water supply." These statements that the fluoride content was
"maintained" at "1 ppm" and "between 1.0 and 1.2
ppm" should be considered in conjunction with that of the United Kingdom
Mission (1953): "For example, in 1951 the average for the year was 1.2 ppm
with a variation between 0.75 and 1.45 ppm., however, the figures below 1.1 ppm
and above 1. 3 ppm were few in number."
2.
Stratford. The supply of Stratford was stated to
have 1.2 ppm of natural fluoride in its domestic water (Ontario Department of
Health, 1956; New Zealand Commission of Inquiry, 1957). However, in reporting
the National Health and Welfare study in 1951 and 1952, Brown stated that it
"contains 1.3 ppm. of fluorine"; and, with his co-workers, the
following year said that "in Stratford where the water supply, obtained
from deep wells, has contained 1.3 ppm." (fluoride) "from natural
deposits continuously since 1917" (Brown et al., 1953). In their next report dealing with dental caries
(Brown et al., 1954b), the figure
stated was "1.3 to 1.6 parts per million of fluoride"; but in the
following report these authors (Brown et
al., 1956) said that this water supply "contained 1.6 parts per
million of fluoride since 1917", and also that 1.6 ppm fluoride content
"has been in continuous use for thirt yeight years."
These different statements, although
strange, may be considered to be unimportant from the practical point of view,
but very small changes in the fluoride content of the water may have
considerable effects, as reports by Deatherage (1942) and Galagan and Lamson
(1953) indicated. The latter authors found that "In water supplies of the
Arizona communities studied, concentrations of fluoride above 0.8 ppm resulted
in objectionable dental fluorosis; concentrations of 0.6 to 0.8 ppm resulted in
an occasional diagnosis of fluorosis; concentrations below 0.6 ppm did not
cause objectionable fluorosis." The mean temperatures of these communities
were between 67 and 72 degrees Fahrenheit.
Three
misleading statements. In the summary of the 1954b
report of the National Health and Welfare study, Brown et al. stated that during the period 1948-54 "dental caries
experience of children in the two control cities, on the other hand, either has
remained at about the 1948 levels, or has increased slightly, at all ages
studied." A similar statement was made by them in the 1955 Report (Brown et al., 1956): "During that time
[more than ten years] a very important, statistically significant reduction in
tooth decay has occurred in all the age groups studied, while in the two
control cities of Sarnia and Stratford it has either remained at about the same
level or increased somewhat." The last sentence contains three misleading
statements about the control cities:
(a) As this study did not commence
until 1948 with "examinations in Sarnia beginning in March of that year,
and in Stratford in October" (Brown, 1952), and as fluoridation in
Brantford commenced in June 1945 (Hutton et
al., 195 1), no information is available with regard to the prevalence of
"tooth decay" in Sarnia during the first two and three-quarter years,
or in Stratford for the first three and a quarter years of the ten-year period
of fluoridation which these authors were discussing in their 1955 Report.
Therefore, it cannot be known whether this condition "remained at about
the same level" in the control cities during the early years of
fluoridation in Brantford.
(b) No remarks were made in the context
of this statement in this (or the previous) paper which suggested that
reference was being made to the permanent teeth only, but these statements are
not correct for the deciduous teeth. (However, in the former paper, a similar
statement to that made in the summary was also made under the heading
"Mean DMF Permanent Teeth".) Decreases in the df (decayed, filled
deciduous teeth) rate were seen in the nine to eleven years group in both
Sarnia and Stratford, that in the former city being shown by these authors to
be statistically significant (Brown et al.,
1954b, Table 10).
(c) In both cities the mean rates of
DMF and df teeth per child showed changes which were said to be statistically
significant (Brown et al., 1954b).
Omission
of decreases. The decreases in the caries attack
rates of the deciduous teeth, which have been mentioned above, were also
omitted in Table II of the Report to the Minister of Health, Province of Ontario,
Canada, by the Division of Medical Statistics, Ontario Department of Health,
which was made in 1955 (Ontario Department of Health, 1956). Under the heading
"% Reduction Since 1948", these decreases were not shown, but
instead, in the appropriate positions dashes were printed, despite the fact
that in Sarnia the percentage reduction (determined by the method commonly used
in these studies) was 16 per cent, almost as great as that of 18 per cent shown
for the same age group in the test city; furthermore, this reduction in Sarnia
was stated by the authors (Brown et al.,
1954b) to be statistically significant.
Different
rates reported. It should be noted that the deficiency
in the data of the National Health and Welfare study, owing to its late commencement,
could not be decreased by comparing the rates obtained by its examiners with
those reported by the City Health Department examiner, because of the
considerably lower rates recorded by the last-mentioned examiner when impecting
similar groups of children. For instance, in their examination in 1948, for
children aged six to eight, nine to eleven and twelve to fourteen years, Brown et al. (1953) obtained rates of 1.41,
4.07 and 7.68 respectively for the permanent teeth, compared with rates of
0.84, 3.37 and 6.11, for the same age groups of children, in the same city and
in the same year, obtained by the City Health Department examiner (calculated
from Table IV, Hutton et al., 1951).
Significant
fluctuations in controls. In the two control cities
"where it is presumed that there has been no appreciable change in either
preventive or treatment services" (United Kingdom Mission Report; 1953),
it can be seen in Tables 4, 6, 8 and 10 published by Brown et al. (1954b) that some considerable fluctuations in the caries
attack rates were recorded; more than half of the inter-year differences in
each of the control cities being shown to be statistically significant.
However, in the text it was stated that the "dental caries experience of
children in the two control cities . . . either has remained at about the 1948
levels, or has increased slightly, at all ages studied."
(1) Sarnia. In this city the changes
between examinations of the rates for the deciduous teeth were not very marked,
but there was a significant one between 1948 and 1954 in the nine to eleven
years age group. However, in the DMF permanent teeth, there were four
definitely significant (three standard error level) and one significant change
in the nine comparisons made. In regard to the first permanent molars, there
were six significant (including three definitely significant) alterations in
the rates, in the nine comparisons made (Brown et al., 1954b).
(2) Stratford. In this city, the rate
of df teeth per child showed a significant difference in one case out of the
four comparisons made between successive examinations (Brown et al., 1954b). In the DMF permanent
teeth per child, the results of the four examinations were: 0.41, 0.75, 0.47
and 0.67 for the six to eight years group; 1.13, 1.76, 1.46 and 1.89 for the
nine to eleven years age group; 2.55, 3.12, 3.02 and 3.77 for the twelve to
fourteen years age group (Brown et al.,
1954b, 1956). These variations between examinations were so large that five out
of the six comparisons made (in the 1954b report) between successive
examinations were said to be statistically significant, four of them being at
the three standard error level. In the last report published (1956), Brown et al. abandoned the method which they
had used in the two previous ones, that of showing the standard error of the
mean values of the DMF rates, and of making "Inter-City" and
"Inter-Year" comparisons (Brown et
al., 1953, 1954b). Therefore it was not stated whether the differences
between the 1954 and the 1955 DMF rates in Stratford were significant, but it
can be seen that they were marked; the difference of 0.75 in the twelve to
fourteen years group being considerably larger than any of those stated in the
1954b report to be significant differences between various examinations in that
city. When the DMF rates for the first permanent molars are considered, similar
marked changes are seen, and in five of the six comparisons the differences
were significant (four definitely so).
Larger
"percentage" changes in a control. If one
resorts to the method commonly used in these trials - that of expressing the
alteration in the DMF rate as a percentage of the original rate - these
unexplained increases in the control city of Stratford between 1948 and 1955,
although they were described as "no change" (Ontario Department of
Health, 1956), and as "a slightly higher prevalence of dental caries in
1955, over the 1948 levels" (Brown et
al., 1956), are found to be 63 per cent, 67 per cent and 48 per cent, for
the six to eight, nine to eleven and twelve to fourteen years age groups
respectively. In each case these percentage changes are considerably larger
than those of 51 per cent, 44 per cent and 37 per cent which can be computed
from the data reported for Brantford. The last-mentioned changes were
attributed to fluoridation, and each was stated to indicate "a very
important, statistically significant reduction in tooth decay" (Brown et al., 1956).
This is just one instance of the
strange results which are obtained when this method of calculation is used. It
should be realized that it was the one most commonly employed in fluoridation
trials, and was used in formulating the often-expressed claim that (as stated
by Arnold et al., 1956): "In
children born since fluoridation was put into effect, the caries rate for the
permanent teeth was reduced on the average by about 60 per cent." The
recent World Health Organization Press Release (WHO/45, 4 September 1957)
stated - with no mention of age - "The prevalence of dental caries in the
permanent teeth of children decreased some 60 percent".
A
smaller "percentage decrease" after long fluoridation.
The "percentage decreases" which have just been mentioned (51 per
cent, 44 per cent and 37 per cent, calculated by the method described in the
last paragraph) were not stated in the 1956 report of Brown et al., but the figure of 51 per cent
for the six to eight years age group is considerably less impressive than the
figure of "approximately 69%" published in the 1954b report from this
study. Although the final report (1956) gave the rates for 1948 and 1955 only,
and therefore did not show the fluctuations between examinations, from the
1954b and 1956 reports of Brown et al.
it is seen that the marked change in the "percentage" decrease which
has just been mentioned was due to the DMF rate in Brantford in 1955, for this
age group, being the highest seen since 1951. Ignoring the fact that in
"children born subsequent to fluoridation" the "decrease"
in the DMF rate had dropped to only 51 per cent, the authors stated in the
final sentence of their final report (1956): "For every three decayed
teeth they would have had, they have only one."
More
misleading comments. Turning from the reports made by the
authors of this study about the control cities to some of the comments made by
others, it is seen that these are even more misleading. Only two will be
mentioned. Martin (1956) stated that during "the 1948-54 period" the
"DMF figures for the two control areas have remained at 1948 levels."
The authors of the Ontario Department of Health Report (1956) went so far as to
state to their Minister of Health that "it had been established that there
has been no change in the already low dental caries attack rates in Stratford
... or in the relatively high rates for Sarnia".
These two statements are contrary to
the results published by the authors of the study (Brown et al., 1954b), which showed that in both the control cities there
were statistically significant differences between the caries attack rates at
successive examinations. Out of the fifteen comparisons made, only five
differences in the rates were not significant, two changes were significant and
eight changes were definitely significant.
Unexplained
significant changes in controls. All the changes in
the caries attack rates in the control cities which were reported to be
significant are unlikely to be chance variations; therefore, to what factor or
factors must they be attributed. It is possible that they were due, in whole or
in part, to alterations in the "weighting", such as were found in the
Evanston study as a result of combining the caries attack rates of children of
different ages (Hill et al., 1952).
However, as the age composition of the groups was not stated in this study, it
cannot be determined to what degree the data was distorted by
"weighting", a condition which is almost inevitably present when data
drawn from several different yearly age groups are combined.
Apart from deficiencies which are found
in other studies also, in this trial there is an absence of any information
regarding the caries attack rates in Brantford and Sarnia, prior to the
fluoridation of the water supply of the former city. There is also the fact
that no explanation was given by the authors for the significant variations in the
caries rates in the control areas. Therefore, a marked decrease in dental
caries in the test city as a result of fluoridation cannot be said to have been
established.
THE
NEWBURGH STUDY
The fluoridation trial conducted in
Newburgh differs from the other studies in two important ways:
1. In almost all the comparisons made,
the data obtained were compared with those from Kingston, the
"fluoride-free" control city, instead of the method used in the other
trials, by which most comparisons were made between the initial and the latest
observations in the test city.
2. The caries attack rates were stated
per 100 erupted teeth, instead of per 100 children or per child. The Evanston
study was the only other one in which the caries rate per 100 erupted teeth was
published; Hill et al. in 1955 and
1957a showed this rate, but only for children aged twelve to fourteen years.
The
control city.
Kingston was used as the control area. "Both cities are
situated on the Hudson River about 30 miles apart. Each has a population of
approximately 30,000. The climate of both cities is also similar, and their
water supplies at the outset of this study were comparable and have remained
so, except for the addition of sodium fluoride to Newburgh's supply" (Ast et al., 1950). Ast and Chase (1953)
added the information that the two cities had a "comparable age, sex, and
color distribution"; and Schlesinger, Overton and Chase (1950) mentioned
that they "bore a close resemblance to each other in respect to size and
socio-economic conditions".
Late
examination of control city. In Kingston, as in
the other "fluoridefree" control cities that have been considered,
the basic examinations were not made until after the fluoridation of the water
supply of the test city. Fluoridation was started in Newburgh on 2 May 1945
(Ast et al., 1950), but the
examinations in Kingston were not conducted until "Sept., 1945 - Feb.,
1946" (Ast et al, 1950).
Considerably
different composition of waters. In 1950 Ast et al., stated that the water supplies
of Newburgh and Kingston "at the outset of this study were comparable and
have remained so, except for the addition of sodium fluoride to Newburgh's
supply." However, both the source and the composition of the water
supplies of these two cities are different. The United Kingdom Mission (1953)
stated that the source of Newburgh's water is from "surface water. Algae
growths in spring and summer checked by copper sulphate blown on the surface of
the water as a powder." The source of Kingston's supply was described as
"Mountain spring impounded. Auxiliary supply, small spring reservoir"
(Lohr and Love, 1954).
In regard to the composition and other
characteristics of these waters, according to analyses of the finished waters
made in February 1952 by the U.S. Geological Survey (Lohr and Love, 1954), in
each of the ten items - magnesium, sodium, potassium, bicarbonate, sulphate,
chloride, dissolved solids, specific conductance, hardness and alkalinity - the
values for the Newburgh water were at least four times as great as those
obtained from analysis of the Kingston supply. In the very important matter of
the calcium content, the Newburgh value of 35 ppm (Ca) was more than five times
as large as that of the Kingston one of 6.6 ppm (Ca). Changes in the supplies
during the period of the trial, owing to natural or to treatment-chemical
variations, are unlikely to have affected these gross differences more than
slightly.
 |
Figure
7. The considerably different calcium and magnesium content
and hardness of the water supplies of Newburgh and its control city of
Kingston, February 1952. Eight other characteristics of the Newburgh water
were at least four times as large as they were in Kingston. The authors of
this study stated that these waters "at the outset of this study were
comparable and have remained so" (Ast et al., 1950).
|
An
unsatisfactory control. In proposing this study, Ast
(1943) said: "Much care must be exercised in the selection of study areas
which should be comparable in as many essential factors as possible." The
first of these factors which he mentioned was the "chemical composition of
past and present water supply". Therefore it is surprising that Kingston
was selected as the control city for Newburgh, for it is clear that in this
very important matter the two cities showed considerably different values. The
importance of the close comparability of the water supplies was emphasized by
the statement of the American Water Works Association (1949) that the
experimental verification of the fluoride-dental caries hypothesis
"obviously necessitates the use of a nearby "control" city with
a water supply comparable in all respects to that to which fluoride is being
added."
Variations
in methods used.
An outstanding characteristic of this study is the variation in the methods
used, both in gathering the data and in the presentation of the results. There
were changes in the examiners; on some occasions clinical examinations only
were made and on others X-rays were also used. The statisticians changed, as
did their presentation of the data in age groups. The sampling method varied in
regard to residence qualifications, and changes occurred in the age range of
the children who were examined. In one report data was obtained from selected
schools only. In some examinations the sampling method was different in the
control city from that used in the test one, All these matters will now be
considered more fully.
The
dental findings. These were published in five papers.
Ast, Smith, Wachs and Cantwell, in 1956, said: "Progress reports were
published after three, four, six and eight years of fluoride experience in
Newburgh" (Ast et al., 1950,
1951; Ast and Chase, 1953; Ast et al.,
1955). The last-mentioned report (Ast et
al., 1955) "after eight years of fluoride experience" gave the
results obtained during the examinations of 1953-4. The final report, giving
the results for 1954-5, apparently one year later than those in the fourth
dental report, was said to show the "dental findings after ten years of
fluoride experience" (Ast et al.,
1956). However, as fluoridation in Newburgh commenced on "May 2,
1945", and as the examinations given in the final report were made "between
October 1954 and June 1955" (Ast et
al., 1956), it would appear that, at the most, only a small part of the
data of the final examination was obtained "after ten years of fluoride
experience."
Different
examiners used. The initial examinations in both cities
were made by Finn. "The subsequent examinations in Kingston using the same
technic were made by two dental hygienists" (Ast et al., 1950). The examinations in 1951-2 were conducted by two
examiners, but "Due to loss of one of the examiners during the examination
year, it was deemed advisable to use only those examinations made by the
remaining examiner in both cities" (Ast and Chase, 1953). The clinical
examinations in 1953-4, and the final ones, were made by Wachs (Ast et al., 1955, 1956). These changes were
made despite the fact that in 1943 Ast said that "the examinations
throughout the study should be made by the same dentist because of the marked
variation in diagnosis of small carious lesions, pits, and fissures by
different dentists."
The clinical examinations were
supplemented by the use of X-rays in the years 1949-50, 1953-4 and 1954-5 (Ast et al., 1956). In the first of these,
which was confined to children aged seven, nine and eleven years, the X-rays
were taken by a staff dentist and were read by Ast and Finn (Ast et al., 1951). The next series was taken
by Wachs and was read by Bushel (Ast et
al., 1955); the final X-rays were taken by Wachs and a staff hygienist, and
they were read by Wachs and Smith (Ast et
al., 1956).
Non-comparability
of data. In the last two reports (Ast et al., 1955, 1956), the carious
cavities that were detected by the X-ray were added to those found in the
clinical examinations. Ast et al. in
1955 said that "the data in this report cannot be compared directly to those
earlier data based on clinical examinations alone." However, in Table 3 of
the 1956 report, the results of the clinical examination are shown separately,
but a satisfactory comparison with those obtained in the earlier years is
prevented by the fact that in this report the data were not published for
yearly age groups, but for the age ranges six to nine and ten to twelve years.
Data for the other two age groups which were shown in the final report,
thirteen to fourteen and sixteen years, were not published in the previous
ones.
The rates for the deciduous teeth were
given in only one report (Ast et al.,
1951).
Examiner
variability. The between and within-examiner
variability was not investigated, although, early in the study, the importance
of this matter was recognized by Ast et
al. (1950) when they stated: "We cannot entirely rule out the
possibility of variation in the interpretations of the examiners. The fact that
more than one examiner was used might alter the differences between Newburgh
and Kingston to some extent." In the following year (Ast et al., 1951) it was stated: "In
the present report an attempt is made to demonstrate that through an objective
roentgenographic examination of the teeth of selected age groups, the question
of examiner bias in this study is not likely to account for the differences
noted." However, the only data published were those of the first permanent
molars; and the finding that "the DMF roentgenographic findings of the
first permanent molars only" in selected age groups shows "consistent
differences at each age in favor of Newburgh" does not provide an estimate
of examiner variability such as could have been obtained readily by normal
statistical methods.
In addition to the changes in the
examiners and in the examination methods, there were changes in the
statisticians. The report after three years of fluoridation was made in
collaboration with one statistician; those after four, six and eight years with
a different one; and a third statistician was employed in the preparation of
the final report.
Different
adjustment procedures. In most of the tables in this
study a "Crude rate" and an "Adjusted rate" are shown. The
incongruity of making these small adjustments to rates that were obtained by
combining data from children of considerably different ages does not appear to
have been realized. In some cases even data from children aged between six and
twelve years were added (Ast et al.,
1950, 1951; Ast and Chase, 1953), the great increase in the caries attack rate
between those ages being ignored. The adjustments were made (depending on the
type of data) to the tooth population, the first permanent molar population, or
the distribution of children. In the first three reports of dental findings
(Ast et al., 1950, 1951; Ast and Chase,
1953), they were all made to the appropriate situation in Kingston during the
1955-6 examinations, but the adjustment system was then changed, the crude
rates after eight years of fluoridation being adjusted to the situation in
Kingston in 1953-4 (Ast et al.,
1955), and those shown in the final report to that present in the control city
in 1954-5 (Ast et al., 1956).
Variations
in age groups. In discussing the Evanston study, it
has already been pointed out that the method of combining the results of different
age groups may result in "weighting" the data, so that comparisons
between the test and the control cities may be affected. In the examples given
>from other fluoridation trials in which this method was used, the age
groups were consistent from examination to examination; but in the
Newburgh-Kingston study the groups varied between examinations, between
comparisons made from data obtained during the same examinations, and even the
age range of the subjects inspected varied from time to time. In regard to the
DMF rate per 100 erupted permanent teeth, the groups were as follows: 6-7,
8-9,10-12 (Ast et al. 1950); 6, 7, 8,
9, 10, 11 and 12 (Ast et al., 195 1;
Ast and Chase, 1953); 6, 7, 8, 9 and 10 (Ast et al., 1955); and 6-9, 10-12, 13-14 and 16 years (Ast et al., 1956).
Changes in the age groups were also
made in reporting the other data presented in this study, but in many cases the
groups were different from those which have just been mentioned.
Grouping
of data hinders comparisons. In the final
report, Ast et al. (1956) said:
"The data are combined for six to nine year old children because these
children in Newburgh had used fluoridated water throughout their lives";
and the age groups ten to twelve years and thirteen to fourteen years were
associated with the tooth calcification pattern. No explanation has been found
for the grouping used by Ast et al.
in 1950, but this matter will be considered later.
Whatever may have been the reason for
adding the data of children of different ages, it has the unfortunate result of
making it very difficult to compare the rates which were present in the test
(and in the control) city at different stages of the trial, especially as, in
the 1955 report of Ast et al., the
rates obtained from the clinical examinations were not shown separately from
those computed from the combined clinical and X-ray results.
"Weighting".
Even if the explanation advanced by the authors of this
study is considered to be a reasonable one, there remains the danger of
"weighting" the data by combining into one category such divergent
material as is provided by children of different yearly ages. One of the tables
in which obvious "weighting" is seen is Table I of the first report
(Ast et al., 1950), "weighting"
being present in several different forms. In the control city, the total DMF
rate per 100 teeth (ages six to twelve years) is "weighted"; for the
total number of teeth examined is made up (in 1945-6) of only 11 per cent from
the six to seven years age group, with its comparatively low DMF rate, and of
67 per cent from the ten to twelve years group with its comparatively high rate
(22 per cent was from age eight to nine years). In the latest examination shown
in that table (1947-8), the two percentages were 17 and 59 respectively, so
that the comparison between the results of the two examinations is also
"weighted". Similar instances of "weighting" are also seen
in the data >from the test city; but as these are of a different degree, the
comparison between Newburgh and Kingston is another instance of
"weighting" (Table 1, Ast et al.,
1951). It can be seen that some "weighting" occurred within the age
groups used in the baseline examinations, principally in the eight to nine
years group in both cities.
Fewer
erupted teeth than expected. In
the final report (Ast et al., 1956),
from Table I it can be calculated that the number of erupted permanent teeth in
the six to nine years group in Newburgh was less than the number expected, on
the assumption that the mean age of eruption of each type of tooth was the same
as in the children in Kingston. Also, in the ten to twelve years group (by
assuming that in these children at least the eight incisors and the four first
molars would have erupted) the number of erupted permanent canines, bicuspids
and second molars was fewer in Newburgh than would be expected. Statistically
speaking, both these differences are highly significant.
No comments:
Post a Comment