Errors
and Omissions in Experimental Trials - 1d
THE BRANTFORD STUDIES [cont.]
Delayed eruption or "weighting"?
These results could have arisen by there being a
delay in the eruption of these teeth in the Newburgh
children, for it is unlikely that the eruption rate
altered in the Kingston subjects. However, Ast et
al., (195 1) said that "there does not seem to
be any change in the eruption pattern among the
children in Newburgh, the study city, as compared
with those in Kingston, the control city." No
definition of an "erupted tooth" was given, but it
is presumed that the authors of this study did not
adopt the odd method used in Evanston, where "Only
teeth which were 50 per cent or more erupted were
considered. A carious or filled tooth was, of
course, considered regardless of its stage of
eruption" (Hill et al., 1955).
The conclusion of Ast et al.
that has just been mentioned was reached only four
years after the commencement of fluoridation, and as
the teeth considered were partially formed prior to
the commencement of that process, they would not
show effects which the ingestion of fluoridated
water may produce on the early stages of tooth
development.
If delay in eruption did not occur
in Newburgh, the lower number of permanent teeth
present at those ages in that city compared with
that present in Kingston was due to a "weighting"
effect; there having been, in proportion, more young
children in each of these age groups in Newburgh
than there were in Kingston. If this is the case, as
it is reasonable to assume that the mean DMF rates
of the younger children were lower than those of the
older ones, it would appear that in these age groups
the contrast between the DMF rates in Newburgh and
those in the control city was exaggerated in the
final report.
"Smoothing" of initial rates.
In 1951 Ast et al. reported that the
"initial clinical examinations made in Newburgh and
Kingston in 1944-1946 were made by one examiner, at
which time the DMF rates were the same." However,
reference to Table 2 in that paper shows that the
DMF rates per 100 erupted permanent teeth were, in
Newburgh and Kingston respectively, at age six
years, 8.5, 7.2; age seven years 11.7, 12.0; age
eight years 17.1, 17.3; age nine years 21.2, 18.9;
age ten years 21.9, 21.3; age eleven years 21.8,
21.8, and age twelve years 25.3, 25.4. Also, Table
5. which shows the DF rates per 100 deciduous teeth
present, gives the rates in Newburgh and Kingston
respectively as 27.2, 21.5 at age five years; 34.2,
32.1 at age six years; 42.3, 43.3 at age seven
years, and 48.0, 47.2 for the eight-year-old
children. Data for the DF rates of the deciduous
teeth of older children were not provided.
In the first report of this study (Ast
et al., 1950) no results were given for the
deciduous teeth, and the results for the permanent
ones were presented in three age groups, six to
seven, eight to nine and ten to twelve years. It can
be seen that by adding the data from children aged
six years, in whom the DMF rate in Newburgh was
higher than that in Kingston, to those of the
seven-year-old children, in whom the reverse
situation was present, the divergence between the
rates prevalent in the two cities was reduced. In
Table I (Ast et al., 1950) the combined rate
was shown as 10.7 in Newburgh and 10.8 in Kingston.
In a similar manner, the addition of the data for
eight and nine-year old children and those of
children who were ten, eleven and twelve years of
age produced a levelling effect between the rates of
the two cities in these two combined age groups.
This process of combining data >from children of
different ages, when reporting the DMF rate per 100
erupted permanent teeth, although it was employed in
only the first and the last dental reports, may have
been used in order to simplify the presentation of
the data; but it had the unfortunate effect of
disguising differences between the DMF rates in the
two cities at the time of the basic examinations. In
the next report the situation was stated more
accurately, Ast and Chase (1953) saying that "the
DMF rates in both cities were approximately the same
at the start of the study".
Fluctuations in the control city.
In this, as in other studies, it is found that the
comments made in the text tend to underrate the
changes that took place in the dental caries attack
rates in the control city. In the summary of the
paper by Ast et al. (195 1) it was stated
that "the DMF rates in the control city of Kingston
show no changes." In that paper, Table 2 shows the
DMF rates per 100 erupted permanent teeth; in
Kingston the "per cent change" in the rates of the
four age categories six, seven, eight and nine years
were 30.5, 7.5, 0.6 and 9.5 respectively. Small
changes were shown for ages ten, eleven and twelve
years. It should be noted that the six, seven and
eight-year-old children all showed decreased rates
between 1945-6 and 1949. No attempt was made to
explain these decreases, and the water of Kingston
"remained fluoride deficient throughout the study
period" (Ast et al., 1956).
Fluctuations disguised. The
method used by Ast et al. in 1951 was to
compute the mean DMF rate per 100 teeth in all the
children aged six to twelve years; the Kingston rate
for this combined age group declining slightly from
20.2 to 19.9 between 1945-6 and 1949. However, by
adjusting to the "permanent tooth population in
Kingston 1945-6 examinations", the authors showed
that the rate of 19.9 became 20.2. On this basis it
could be claimed that the "rate" in Kingston had not
changed, but the incorrect statement was made that
the DMF "rates" in the control city of Kingston
showed no "changes". These rates of 19.9 and 20.2
were produced by combining the data of young
children - that had few erupted permanent teeth and
relatively low DMF rates per 100 teeth with data of
older children that had most of their teeth erupted,
and considerably higher DMF rates per 100 teeth. The
rate obtained in 1949 was then adjusted. This
procedure, no doubt unintentionally, disguised the
fluctuations in the rates in the control city.
In Table 2 (Ast et al., 195
1) the "per cent change" in the Kingston children
aged nine years was shown as 9.5, but if the figures
18.9 and 19.1 are the correct ones for the years
1945-46 and 1949, the "per cent change" should have
been stated as 1.1, not 9.5.
Variability of caries rates.
Unfortunately, the variability of even the mean
caries rates cannot be studied, for the rates of
yearly age groups were not published in the first
and the last reports (Ast et al., 1950,
1956), and the only results shown in the 1955 report
of Ast et al. were based on a combined
clinical and X-ray examination.
The meagre data supplied for
deciduous teeth. Data regarding the caries rates
of the permanent teeth were shown in each report of
this study; however, only very meagre data were
published for the deciduous ones. None were made
available in the first report (Ast et al.,
1950). In the following year (Ast et al., 195
1) the DF rates per 100 deciduous teeth were given,
but only for children aged five, six, seven and
eight years, and in each age group the rates had
decreased both in the test and in the control
cities.
Unexplained marked decreases in
the control. No explanation was given by Ast
et al. (1951) for the decreases in the DF rates
in Kingston, where the greatest relative decrease,
from 32.1 per cent DF to 24.8 per cent DF, was seen
in the teeth of the six year-old children. It would
have been of great interest to see whether this
trend was maintained in later years, but DF rates
were not stated in the tables contained in any of
the later reports. However, in the following one (Ast
and Chase, 1953) the situation in regard to the
deciduous teeth of children five, six, seven and
eight years old (now termed "def teeth per 100
deciduous teeth present") was depicted
diagrammatically by means of a histogram, these
unexplained decreases in the def rates in the
control city being clearly seen, a small one at age
five years, and considerable ones at the ages of
six, seven and eight years.
The increase in caries-free teeth
in the control. The only other
information published regarding the deciduous teeth
was expressed in terms of "Children with caries free
deciduous cuspids, first and second molars". This
type of table appeared first in the 1951 report of
Ast et al., and the results were given for
only those children who were five or six years of
age. In both age groups in Kingston the figures
suggest an increase in these caries free teeth, the
six-year-old children changing, between 1945-6 and
1949, from 17.2 per cent to 25.5 per cent free from
caries. These changes were mentioned, but no attempt
was made to explain them. In the next report (Ast
and Chase, 1953) data for children aged seven years
were also included. This report showed that, between
1945-6 and 1951-2, the percentage of children in
Kingston who had these deciduous teeth free from
caries showed a slight decrease at age five years
(28.2 per cent to 26.4 per cent); but in the
six-year old children the percentage increased from
17.2 to 26.3; and in those who were seven years of
age, it practically doubled (8.3 to 16.5). On this
occasion, these changes in the control city were not
even mentioned.
"Analysis" of findings. In
the 1955 report of Ast et al. it was stated
that "As an indication of the benefits of water
fluoridation to deciduous teeth, a previous report
[in 1953] analyzed the findings among the 6 to 7
year old children in each city after six to seven
years following the initiation of water
fluoridation." Actually the report showed findings
for the children aged five, six and seven years who
had caries-free deciduous cuspids, first and second
deciduous molars. At the ages of six and seven
years, in both the test and the control cities,
there were increases in the percentages of these
teeth that were free from caries; therefore,
although these increases were greater in Newburgh,
they should not have been attributed solely to water
fluoridation. In any case, the publication of one
table showing, in this selected group of deciduous
teeth, the percentage changes that have just been
mentioned, a histogram depicting the def rates, and
twenty lines of comment in the text on the results
displayed, can hardly be said to indicate the
benefits of fluoridation to deciduous teeth, or even
to constitute an adequate analysis of the findings
in regard to the deciduous teeth present in children
aged six and seven years.
Changes in caries-free teeth in
the control. In the 1955 report of Ast
et al. the age range was changed by not
publishing the results for the five-year old
children, but showing, for the first time, the
results for caries-free deciduous cuspids first and
second deciduous molars, for eight and nine-year old
children. However, these results cannot be compared
with those of the previous years, as they were based
on a combined clinical and X-ray examination.
Nevertheless, a comparison can be made with the
rates shown in the final report (Ast et al.,
1956). In the year between the 1953-4 and 1954-5
examinations, the rates in Kingston for the ages
six, seven, eight and nine years changed from 10.6,
7.0, 7.9 and 0.0 to 11.1, 4.7, 1.8 and 1.6 for the
respective ages. Such changes are not unexpected,
for marked variations were seen in Evanston, where,
also, the examinations were made by a clinical plus
X-ray procedure. For instance, the percentages of
children aged seven years who were drinking
fluoridated water and who had caries free deciduous
teeth were, in successive examinations, 11.33
(pre-fluoridation), 8.71, 3.87, 10,66, 13.01 and
17.86 (Hill et al., 1956). It would seem that
assessments made on the basis of caries-free groups
of deciduous teeth are not very reliable.
Changes in the sampling method.
Consideration of these five dental reports shows
that the sampling method changed from time to time,
and that the method used in the control city was
sometimes the same and sometimes different from that
used in the test one. In the first report (Ast et
al., 1950) it was stated:
"we are considering only those
children age 6-12 who were in the original base
study and who have had each successive examination
until they reach age 12. Also included are new
school children who entered the study at age 6
subsequent to the first examination and were present
at each of the successive examinations. Thus, this
study group will have only those children who we are
assuming have had continuous residence in their
respective cities."
Continuous residence only
assumed. It can be seen that the "continuous
residence" of each subject was based on assumption
only, and not on statements made in a questionnaire,
such as was used in Evanston (Blayney and Tucker,
1948). Therefore, it is possible that children could
have been absent from the city for considerable
periods between the times of successive
examinations. Also, there is no assurance that the
six-year-old children entering the study in any of
the post-fluoridation examinations had not come to
live in the area since the commencement of the
study. Therefore, it is doubtful whether the
objective of having "reasonable assurance that the
children studied had had continuous residence in
their respective cities" (Ast et al., 195 1)
can be said to have been attained.
Population changes in Newburgh.
"Early in 1950 questionnaires were given to more
than 3,200 children in the Newburgh schools for
completion by their parents" (Ast et al., 195
1). The questions asked were not stated, nor was the
number of replies received, but it was said that:
An analysis of the answers to those
questionnaires shows that the Newburgh population is
a relatively stable one and that the inclusion of
the small migrant groups does not alter the caries
picture to any significant degree. Consequently, in
this report there are included all 5 to 12 year old
children present in the schools in Newburgh and
Kingston on the days the examinations were made" (Ast
et al., 195 1).
Since information in regard to the
caries attack rates in these migrant groups could
not have been obtained directly from the
questionnaires, it is presumed that the dental
record cards of those children were grouped and that
the cards of the children who were judged from the
answers not to be migrants were also grouped, and
the data contained in the two groups in regard to
the caries attack rates were compared. If that
process was carried out, it was not mentioned, nor
were data published which would enable the reader to
assess the situation. If no differences were found
between the two groups, it must be considered to be
strange because by that time it was said that "The
DMF rates among permanent teeth of 6 to 12 year old
children in Newburgh show a consistent downward
trend" (Ast et al., 195 1). The United
Kingdom Mission (1953) reported that the authors of
this study had "found that the proportion of
immigrants in Newburgh and Kingston was too small to
affect the comparison." However, although the
Newburgh population was said to be "relatively
stable", in the 1954-5 examinations in that city 24
per cent of the children were excluded because they
failed to fulfil the residence qualifications (Ast
et al., 1956).
The workers who conducted the
paediatric study in these cities, Schlesinger et
al., in 1950 said that in each city "An effort
was made to select... children from families which
might reasonably be expected to remain for the
duration of the study." In spite of that precaution,
they found that 29.9 per cent of their subjects in
Newburgh moved from the city during the period of
the study (Schlesinger, Overton, Chase and Cantwell,
1956).
Population movement in Kingston.
No mention was made of the issue of a
questionnaire to children in the control city;
apparently it was assumed that migrants to that city
would have come >from areas with "fluoride-free"
water supplies. Schlesinger et al. (1956)
found that 22.2 per cent of the children included in
the paediatric examinations moved from Kingston
during the period of the study; presumably a similar
number of new residents settled in the city.
It may be considered that in moving
from one locality to another, interruptions could
occur to regular conservative and prophylactic
treatment of the children, so that their dental
health may not have been as good as that of children
who lived for many years in the same city. It is
possible also that regular dental examinations, by
stimulating interest in the teeth, may improve
eating habits and oral hygiene measures.
Considerable alterations in
populations. In Table I of Ast et al.
(1950) the number of permanent teeth erupted is
shown. The numbers given for Newburgh in the
examination of 1944-5 for the three age groups six
to seven, eight to nine and ten to twelve years are
respectively 3,579, 7,937 and 24,586. However, by
adding in Table I of Ast and Chase (1953), the
number of erupted teeth - for the same age groups,
and in the same examination - are 5,379, 10,033 and
27,186. It was stated in the former report that "we
are considering only those children age 6-12 who
were in the original base study and who have had
each successive examination until they reach age
12." It therefore appears that to meet those
requirements, it was necessary to exclude, for the
three age groups, 33 per cent, 21 per cent and 10
per cent of the number of erupted teeth, and,
presumably, similar percentages of children. A like
situation was seen in regard to the Kingston data,
the percentages of teeth excluded being 24, 26 and
12. After only four years, it was apparently
necessary to omit these large proportions of the
data in order to consider only those children who
were "continuous residents", no other explanation
being evident for the different numbers of erupted
teeth that were stated in the two papers. Although
the population of Newburgh may have been "relatively
stable" when compared with some unnamed population,
it is obvious that the number of migrants was so
great that they should have been excluded from the
study.
Data of migrants excluded only in
Newburgh. The necessity for excluding the data
of migrants was later realized, and the method of
including in the study all the children present in
the schools on the day of the examination - although
it was continued in Kingston - was abandoned in
Newburgh. Ast et al. (1955) stated: "Based on
residence histories, the Newburgh study group was
limited to those who had used Newburgh water since
the introduction of sodium fluoride on May 2, 1945."
In the final report, also, only those children who
had lived continuously in Newburgh were included,
but "All the Kingston children examined are included
in this report" (Ast et al., 1956).
Alterations in sample size.
The sample size and the age distribution of the
children were altered during the course of this
study. The data included in the first three dental
reports were obtained from the "entire elementary
school populations" (Ast and Chase, 1953), except
that in some years some of the children were
excluded in Newburgh on residential grounds, and
that in 1951-2, owing to the loss of an examiner,
only half of the children in each city were
included. However, in the 1953-4 series the age
range was restricted to six to ten years, and the
number of children examined was only a small
fraction of those inspected in the same age groups
during other examinations. Ast et al. (1956)
said that the preceding report "dealt with rather
small groups of children (about 375 children ages
six to ten in each city), and there was considerable
difference in age distribution."
Sampling by selection.
The method of sampling used in the 1953-4
examination must be considered to be unorthodox, and
was described by Ast et al. (1955) in these
words:
"The current series includes a
limited number of schools which were chosen because
of the availability of X-ray facilities. From
previous data on DMF rates by school, it was
determined that the selected Kingston school had a
caries rate which was among the lowest in the city,
while the rates for the three Newburgh schools were
distributed through the range of rates for that
city. This has the effect of minimizing the
difference in the DMF rates between the two cities."
A decrease in the "per cent
difference". In the final report (Ast et al.,
1956, Table 1) the "per cent difference" between the
DMF rate per 100 erupted teeth of children aged six
to nine years in Newburgh and Kingston was given as
56.7. This is a smaller difference than any of those
shown for the ages six, seven, eight and nine years
(74.7, 68.3, 58.1 and 66.0 respectively), in the
previous (1955) report, despite the fact that it was
stated in that report, that the sampling method used
had minimized the difference between the DMF rates
in the two cities. A trial period of ten to twelve
years was suggested by Ast (1943), and was mentioned
in the authors' first report (Ast et al.,
1950). In view of the decrease in the "per cent
difference" between the test and the control cities,
which was revealed in the final report, it is
unfortunate that the trial was stopped as soon as
the minimum period proposed by the authors had
elapsed.
DISCUSSION
The Expert Committee on Water
Fluoridation of the World Health Organization (1958)
stated that "Hundreds of controlled fluoridation
programmes are now in operation in many countries.
Some have been in progress for the past 12 years, so
that conclusions are based on experience." This
statement suggests that there is a large amount of
experimental evidence in regard to the process of
artificial fluoridation. It is very doubtful whether
this is the case. If hundreds of fluoridation
programmes have been conducted with experimental
controls, it is strange, and very unfortunate. that
such a large body of data has not been published;
for, except in the cases of the trials which have
been considered here, published data concerning
fluoridation trials are very meagre. It would seem,
therefore, that the Expert Committee did not use the
term "controlled" in its experimental sense, but in
that of regulated measurement of the fluoride salt,
such as in its statement that "The precision of
fluoride application should be carefully
controlled."
The United Kingdom Mission (1953)
which visited North America in 1952, in its report
referred to "the Fluoridation Studies", and
enumerated only six study centres; and Jenkins
(1955) mentioned "the six study centres on the
American continent". In addition to the four test
cities which have been considered, the Mission
referred to Sheboygan, Wisconsin, and to Marshall,
Texas. No control city for Sheboygan was mentioned;
and the Mission stated that in the latter study "The
neighbouring town of Jacksonville with a
fluoride-free water supply was selected as control,
but although caries experience in the two areas was
compared after 2 1/2 years of fluoridation, the most
valuable basis for comparison is the baseline data
of Marshall itself." The Mission quoted two
unpublished reports as the source of its information
in regard to the latter study. An indication of the
minor importance of these two trials is the fact
that in the 240-page report of the New Zealand
Commission of Inquiry (1957) no data from them were
presented, the former city being mentioned twice and
the latter only once. Furthermore, the directors of
these studies, Doctors F A. Bull and E. Taylor, were
not named in the report, and the extensive
bibliography did not include any papers published by
them.
The crucial importance, even at the
present time, of the trials conducted in Newburgh,
Grand Rapids, Brantford and Evanston was
demonstraled by the report made in 1957 by the New
Zealand Commission, the hearings of which did not
conclude until April of that year, and by the report
of the Expert Committee of the W.H.O. (1958), which
met during August 1957. The only evidence mentioned
by the Commission with regard to the dental results
of the addition of fluorides to water supplies was
that obtained in those four cities. The Expert
Committee referred to only the first three of those
cities in the few lines of its report which
mentioned dental results of fluoridation.
In discussing the general design
used in fluoridation studies, the United Kingdom
Mission (1953) said:
"In a fluoridation study, two nearby
towns, comparable in all respects, are chosen, both
having an almost fluoride-free domestic water
supply, preferably from the same source. The water
of one town is fluoridated while that of the other
remains untreated, this town serving as the control.
Before fluoridation is started the teeth of the
children in both towns are examined in detail to
ascertain if caries experience is similar and to
determine its prevalence in the various age groups.
Further examinations are carried out at yearly
intervals and the dental condition of the children
in the fluoridated town is compared with that of
similar groups in the control town. The
prefluoridation data also serve as a basis for
comparison. The caries incidence may also be
compared with that in a town where a similar
concentration of fluoride occurs in the water
naturally. In practice it is often difficult to
obtain all these conditions and in some studies
there is no independent control."
The term "comparable in all
respects" describes a theoretical ideal for a test
and a control town rather than a practical
possibility. In regard to the other matters
mentioned in the design these studies exhibited
numerous deficiencies. No control was employed in
the City Council study in Brantford, and the Grand
Rapids study lost its control in 1951 as a result of
the fluoridation of the Muskegon water supply. In
the extremely important matter of the water
supplies, both the source and the composition of the
Newburgh water is considerably different from that
of Kingston. Further examinations were "carried out
at yearly intervals" only in Grand Rapids-Muskegon,
and in Newburgh-Kingston until 1952; if yearly
examinations were made in the latter study after
that year, the results for all years were not
published. In the Evanston trial, only two
examinations were made in the control city, and few
data from it have been published; in the test city
only one age group was examined each year. In
Brantford and in Evanston, and in the first and the
last dental reports from the Newburgh-Kingston
study, data from children of different yearly ages
were added, thus introducing the possibility of
"weighting". In some instances, at least, the degree
of "weighting" found indicated that the comparisons
were not being made between similar groups in the
test and the control cities. No pre-fluoridation
data were gathered in Brantford by the Canadian
Department of National Health and Welfare, for that
study was not commenced until over two and a half
years after the fluoridation of the city water
supply.
In all of the studies that have been
considered, it has been seen that fluoridation of
the water supply of the test city was initiated
before the initial caries rates in the control city
were known. This late examination of the control
cities, on first thought, may not seem to be of much
consequence. However, it means that, in all of these
studies, a matter of fundamental importance was
disregarded-it could not have been established that
the children of similar ages in the test and the
control cities, prior to the commencement of the
experiment, had reasonably comparable caries attack
rates. Therefore, the statement of the United
Kingdom Mission (1953) that "Before fluoridation is
started the teeth of the children in both towns are
examined in detail to ascertain if caries experience
is similar and to determine its prevalence in the
various age groups" appears to have been based on
assumptions only.
Caries attack rates may be expressed
as decayed, missing and filled teeth per 100 erupted
teeth, or expressed as per 100 children or per
child. The former method was preferred by the
authors of the Newburgh trial "because individual
teeth may be subjected independently to the hazard
of caries" (Ast et al., 1956). In the
Evanston study, the rate per 100 erupted teeth was
given, but, curiously, only for children aged twelve
to fourteen years. All other cases (the younger
children in Evanston, and the other studies) in
which the caries attack rates per 100 children or
per child were given are based on the assumption,
unsupported by published evidence, that in each age
group the mean numbers of each category of erupted
teeth per 100 children is very similar in the test
and the control cities, and that little variation
occurs from year to year. If this is not the case,
comparisons between the rates prevalent in the test
and the control cities, and those seen in different
years, are not valid. Feltman (1956) gave fluorides
in tablet form td-pregnant women and young children,
and reported that "Many children in the study group
showed a marked delay in the eruption of the
deciduous teeth. This delay is in some instances a
cause for alarm by the parents. The second incisor,
second molars, and cuspids are the most frequently
delayed, in many cases by as much as a year from the
accepted average eruption dates." Of course, if
fluoridation results in the eruption rate of teeth
being retarded, a decrease in caries experience
would be expected due to the shorter time of
exposure of the teeth to the risk of caries. It will
be recalled that data were published in the Evanston
study which were compatible with a continuous and
marked decline in the rate of eruption of first
permanent molars during the first four to five years
of fluoridation, but that further comparisons could
not be made because this type of data was not
published for younger children in later reports.
In order to decrease the chance of
misinterpretation, extensive use has been made of
direct quotation from the original reports, and to
avoid unnecessary repetition, consideration of the
comments made on the results reported from these
control cities, apart from those made by the authors
of these studies, has been restricted to the
statements of only a few writers.
It is felt that it is not necessary
to discuss further the matters which have been noted
above, for they are self-explanatory. It has been
shown that the reports of the controls used in these
fluoridation trials contain arithmetical and
statistical errors, and that results and relevant
data were omitted. Also, misleading statements were
made which denied, ignored, or underrated the
unexplained changes in caries attack rates which
took place in the control cities, and which
suggested that the pre-fluoridation data from the
test cities, and those obtained during the basic
examinations in control ones, were more closely
comparable than was the case. Jean R. Forrest, the
Senior Dental Officer, Ministry of Health, who was a
member of the United Kingdom Mission and of the
Expert Committee on Water Fluoridation of the World
Health Organization, in 1957 contrasted "the
emotional type of opposition" to fluoridation, to
"the precise correct statements of scientists,"
However, the situation which has been encountered is
more aptly described by the words of Wade Hampton
Frost "an outstanding American epidemiologist" (Bews,
1951). More than thirty years ago Frost (1925) said:
"It is frequently easy to exhibit
some figures which, though not really to the point,
will nevertheless serve to impress an uncritical
public, and the temptation may be great to give
them, at least by implication, an unduly favourable
interpretation. It is more difficult and more
tedious to present the full argument, based on all
the facts, and it is perhaps a little humiliating to
admit that the statistical evidence is deficient
because we have failed to collect it; but to do this
is not only more scientific, it is in the end more
convincing, and after all there is no free choice,
because it is the only honest method, whether it be
convenient or not. Finally, it is the only way of
progress, for the first step towards collecting
better evidence is to recognize the deficiencies of
that which is at hand."
More than eleven years after the
initiation of the last of these five trials, the
deficiencies of their controls still remain
unrecognized. The endorsements of fluoridation by
medical and dental associations, by public health
authorities, and even the recent one by the Expert
Committee on Water Fluoridation of the World Health
Organization (1958), appear to have been based
mainly on the opinions of the authors and of others.
Indeed, in the report of that Committee, under the
heading "Results of fluoridation", instead of
results being considered, comment was confined to:
"Reports of the results after 10 years of controlled
fluoridation in three cities". Examinations of the
data obtained in these trials, which have been
published by other endorsing bodies, are also
inadequate or absent. It is an understatement to
term this failure regrettable.
In 1951, Appleton stated that in any
future fluoridation trial: "The experiment should be
genuine, and not one in name only. In designing such
an experiment, a careful and competent analysis of
those now in progress should first be made, in order
to see how they might be improved or extended."
In the early part of this paper some
basic experimental considerations were mentioned. In
cities in which it is intended to compare the caries
attack rates of the children in a fluoridation
trial, the three main factors which should be as
closely comparable as is practically convenient are
the composition of the water supply, the climate and
the dental caries attack rates. Four trials having
"fluoride-free" control cities have been considered.
The composition of Newburgh's water supply is
considerably different from that of its control
city. There were gross differences between Evanston
and its control city regarding the initial caries
attack rates in the younger children. In the
Department of National Health and Welfare study in
Brantford, as the first examination was made over
two and a half years after the commencement of
fluoridation, it cannot be known what the
pre-fluoridation rates in that city would have been,
if assessed by those examiners; thus, it cannot be
determined how closely the (1945) Brantford rates
resembled those of Sarnia. In the Grand Rapids
study, the fluoridation of the water supply of
Muskegon in 1951 severely limited its usefulness as
the control city.
In each trial both the test city and
its control were selected. For instance, "Oak Park
graciously offered to serve as the control
community" for Evanston (Blayney and Tucker, 1948).
Two cities which had agreed to participate in the
experiment, after having been found suitable for
comparison, should have been allotted at random to
be test or control. It will be recalled that in at
least two of the cities selected as test ones,
Evanston and Brantford, "dental care was
outstandingly good" (United Kingdom Mission, 1953)
It has been pointed out in Part One
that in all these trials no attempt was made to
devise a randomization procedure in the clinical
examinations which would have eliminated examiner
bias, nor were estimates made of examiner
variability.
Two statements made by authors of
these studies may be recalled. In 1950 Hill et al.
said: "It is to be expected that the rate of caries
in all teeth varies from year to year due to chance.
A significant reduction of caries prevalence can
therefore be assumed to exist only when the
statistical analysis of the data provides almost
absolute certainty that the observed differences are
not due to chance." However, as was mentioned in
Part One this very important matter of random
variation has been ignored in all these studies.
Blayney and Tucker (1948) stated that: "A study of
this nature must have an adequate control." It has
been seen that the controls used in these trials
cannot be considered to be adequate.
It would appear that these
shortcomings have not been recognized, for those who
conducted these studies, and other writers, have
expressed their satisfaction with the methods used.
For instance, Ast and Chase, the authors of the 1953
report on the Newburgh-Kingston study, referred to
"the carefully controlled studies such as the
Newburgh-Kingston, Grand Rapids-Muskegon, and the
Evanston studies"; and Mather (1957) said: "This
study at Brantford was most carefully set up and has
been under the strictest control."
Approval of the methods used in
these studies was also expressed by the New Zealand
Commission of Inquiry (1957), for it considered
that: "All these investigations" in Brantford,
Newburgh and Grand Rapids "were designed and
executed with great thoroughness." The Commission
also said: "We have examined the statistical
evidence brought forward by the advocates of
fluoridation, and the conclusions they have drawn
from that material ... We have found nothing to
invalidate the statistics or cast doubt on their
reliability." It will be realized that many of the
deficiencies of these studies can be noted only when
different reports >from the same study are compared.
It seems that the Commission was handicapped in this
regard, for although its "List of exhibits produced
at public hearings" mentions over 250 items, such as
papers, books, charts and letters, it includes only
the final report, or the one which was the most
recently published at that time, of the numerous
reports showing the dental caries attack rates which
were published in each of these studies. It would
appear that none of the earlier accounts of these
trials were shown to the Commission, nor were they
mentioned in the "bibliography" of 144 references.
None of the reports from the City Health Department
trial in Brantford were listed as exhibits. The
paper by Brown, Kohli, Macdonald and McLaren (1954a)
which is mentioned deals only with gingival results.
Although the Commission had the assistance of legal
counsel in gathering the evidence, no mention was
made of the employment of a statistician to assist
its members in evaluating the numerical data.
The Expert Committee on Water
Fluoridation of the World Health Organization (1958)
also expressed its satisfaction with the methods
used in these trials. Out of the hundreds of
controlled fluoridation programmes which it stated
have been set up, it mentioned only the Newburgh,
Grand Rapids and Brantford (City Health Department)
studies in the sixteen lines which allotted to the
mention, one cannot say consideration, of the
results of fluoridation on dental caries prevalence.
Presumably these three trials were cited because the
Committee considered that they were the most
important and reliable studies, and it said that
they were "carefully planned and controlled". As
this opinion of the Committee was made in referring
to the three studies which it cited in mentioning
results of fluoridation, it is reasonable to assume,
at least in this instance, that the term
"controlled" was used in its experimental sense.
If this is the case, the inaccuracy
of that statement of the Expert Committee is
astonishing, for it will be recalled that, of the
three studies which the Committee quoted, the
Brantford (City Health Department) study, far from
being carefully controlled, was not controlled at
all. Furthermore, the control for the Grand Rapids
study was abandoned after only six years, at the
crucial stage of the trial when the first of the
permanent teeth were erupting in the children of the
test city who had ingested fluoridated water
throughout their lives. Therefore the control , was
abandoned before any assessment of caries activity
in those teeth could be made. In regard to the
remaining study mentioned by the Expert Committee,
the Newburgh trial, after the unexplained decreases
in the DF rates for deciduous teeth, which were
shown as having occurred in Kingston, the control
city, between 19456 and 1949, no further caries
rates for deciduous teeth were published. Also, the
erratic changes which were made in the methods used
in this trial are not consistent with careful
planning, nor is the choice as the test and control
areas of two cities with water supplies which were
of considerably different composition.
It has been acknowledged for many
years that one of the fundamental procedures in
planning an experiment is the establishment of a
statistical design for the procedures before work is
commenced. The deficiencies in the basic statistical
requirements of a good experimental design are only
too obvious in all these studies. Therefore, it is
surprising that the Expert Committee did not point
out these deficiencies, but, on the contrary,
described the three studies which it mentioned as
carefully planned ones. The importance of these
matters is emphasized by the authoritative statement
of Fisher (1951) that: "If the design of an
experiment is faulty, any method of interpretation
which makes it out to be decisive must be faulty
too."
This investigation of reports of
these fluoridation trials was instituted when a
preliminary examination of the methods used revealed
disturbing facts, and solely because it was felt
that, as Appleton (195 1) expressed it:
"Professionals and specialists have the duty of
insisting upon a scientific demonstration of a high
probability that a proposed method will be useful
and safe, before it is recommended for general
adoption. The maintenance of this attitude is of
paramount importance."
The deficiencies of these trials not
having been recognized, many cities have already
fluoridated their water supplies on advice which is
based largely on the results that have been
considered. It is, therefore, an important and
urgent matter that a more accurate assessment of the
efficacy of this process should be obtained, but,
unfortunately, it appears that little long-term
experimental evidence is available. Therefore,
despite the limitations imposed by the methods used
in these studies, consideration should be given to a
careful and competent examination of the whole of
the original data obtained in them. The findings
resulting from such an examination would be of
assistance in designing future fluoridation trials,
and would provide a far more adequate assessment of
the results reported from these studies than it is
possible to obtain from an examination of the very
limited data that have been published.
At least until such a report is
available for examination, it would be wise to
maintain an open mind in regard to the efficacy of
artificial fluoridation.
SUMMARY
1. Endorsements of the process of the mechanical
fluoridation of public water supplies rely mainly on
five experimental trials.
2. The controls used in these studies are
considered.
3. The reliability of the results reported is
affected by:
(a) odd experimental and statistical methods;
(b) failure to consider random variation and
examiner variability, and to eliminate examiner
bias;
(c) omission of relevant data;
(d) arithmetical errors;
(e) misleading comments.
4. Controls were either doubtful or inadequate.
5. No control was employed in one trial.
6. The published data do not justify the statement
that caries rates remained the same in control
cities.
7. The sound
basis on which the efficacy of a public health
measure must be assessed is not provided by these
five crucial trials.
No comments:
Post a Comment