Errors and Omissions in
Experimental Trials - 1d
THE BRANTFORD STUDIES [cont.]
Delayed eruption or
"weighting"?
These results could have arisen by there being a delay in the eruption of these
teeth in the Newburgh children, for it is unlikely that the eruption rate
altered in the Kingston subjects. However, Ast et al., (195 1) said that "there does not seem to be any
change in the eruption pattern among the children in Newburgh, the study city,
as compared
with those in Kingston, the control city." No definition of an
"erupted tooth" was given, but it is presumed that the authors of
this study did not adopt the odd method used in Evanston, where "Only
teeth which were 50 per cent or more erupted were considered. A carious or
filled tooth was, of course, considered regardless of its stage of
eruption" (Hill et al., 1955).
The
conclusion of Ast et al. that has
just been mentioned was reached only four years after the commencement of
fluoridation, and as the teeth considered were partially formed prior to the
commencement of that process, they would not show effects which the ingestion
of fluoridated water may produce on the early stages of tooth development.
If delay
in eruption did not occur in Newburgh, the lower number of permanent teeth
present at those ages in that city compared with that present in Kingston was
due to a "weighting" effect; there having been, in proportion, more
young children in each of these age groups in Newburgh than there were in
Kingston. If this is the case, as it is reasonable to assume that the mean DMF
rates of the younger children were lower than those of the older ones, it would
appear that in these age groups the contrast between the DMF rates in Newburgh
and those in the control city was exaggerated in the final report.
"Smoothing"
of initial rates. In 1951 Ast et al. reported that the "initial clinical examinations made
in Newburgh and Kingston in 1944-1946 were made by one examiner, at which time
the DMF rates were the same." However, reference to Table 2 in that paper
shows that the DMF rates per 100 erupted permanent teeth were, in Newburgh and
Kingston respectively, at age six years, 8.5, 7.2; age seven years 11.7, 12.0;
age eight years 17.1, 17.3; age nine years 21.2, 18.9; age ten years 21.9,
21.3; age eleven years 21.8, 21.8, and age twelve years 25.3, 25.4. Also, Table
5. which shows the DF rates per 100 deciduous teeth present, gives the rates in
Newburgh and Kingston respectively as 27.2, 21.5 at age five years; 34.2, 32.1
at age six years; 42.3, 43.3 at age seven years, and 48.0, 47.2 for the
eight-year-old children. Data for the DF rates of the deciduous teeth of older
children were not provided.
In the
first report of this study (Ast et al.,
1950) no results were given for the deciduous teeth, and the results for the
permanent ones were presented in three age groups, six to seven, eight to nine
and ten to twelve years. It can be seen that by adding the data from children
aged six years, in whom the DMF rate in Newburgh was higher than that in Kingston,
to those of the seven-year-old children, in whom the reverse situation was
present, the divergence between the rates prevalent in the two cities was
reduced. In Table I (Ast et al.,
1950) the combined rate was shown as 10.7 in Newburgh and 10.8 in Kingston. In
a similar manner, the addition of the data for eight and nine-year old children
and those of children who were ten, eleven and twelve years of age produced a
levelling effect between the rates of the two cities in these two combined age
groups. This process of combining data >from children of different ages,
when reporting the DMF rate per 100 erupted permanent teeth, although it was
employed in only the first and the last dental reports, may have been used in
order to simplify the presentation of the data; but it had the unfortunate
effect of disguising differences between the DMF rates in the two cities at the
time of the basic examinations. In the next report the situation was stated
more accurately, Ast and Chase (1953) saying that "the DMF rates in both
cities were approximately the same at the start of the study".
Fluctuations in the control city. In this, as in other studies, it
is found that the comments made in the text tend to underrate the changes that
took place in the dental caries attack rates in the control city. In the
summary of the paper by Ast et al.
(195 1) it was stated that "the DMF rates in the control city of Kingston
show no changes." In that paper, Table 2 shows the DMF rates per 100
erupted permanent teeth; in Kingston the "per cent change" in the
rates of the four age categories six, seven, eight and nine years were 30.5,
7.5, 0.6 and 9.5 respectively. Small changes were shown for ages ten, eleven
and twelve years. It should be noted that the six, seven and eight-year-old children
all showed decreased rates between 1945-6 and 1949. No attempt was made to
explain these decreases, and the water of Kingston "remained fluoride
deficient throughout the study period" (Ast et al., 1956).
Fluctuations disguised. The method used by Ast et al. in 1951 was to compute the mean
DMF rate per 100 teeth in all the children aged six to twelve years; the
Kingston rate for this combined age group declining slightly from 20.2 to 19.9
between 1945-6 and 1949. However, by adjusting to the "permanent tooth
population in Kingston 1945-6 examinations", the authors showed that the
rate of 19.9 became 20.2. On this basis it could be claimed that the
"rate" in Kingston had not changed, but the incorrect statement was
made that the DMF "rates" in the control city of Kingston showed no
"changes". These rates of 19.9 and 20.2 were produced by combining
the data of young children - that had few erupted permanent teeth and
relatively low DMF rates per 100 teeth with data of older children that had most
of their teeth erupted, and considerably higher DMF rates per 100 teeth. The
rate obtained in 1949 was then adjusted. This procedure, no doubt
unintentionally, disguised the fluctuations in the rates in the control city.
In Table 2
(Ast et al., 195 1) the "per cent
change" in the Kingston children aged nine years was shown as 9.5, but if
the figures 18.9 and 19.1 are the correct ones for the years 1945-46 and 1949,
the "per cent change" should have been stated as 1.1, not 9.5.
Variability of caries rates. Unfortunately, the variability of
even the mean caries rates cannot be studied, for the rates of yearly age
groups were not published in the first and the last reports (Ast et al., 1950, 1956), and the only
results shown in the 1955 report of Ast et
al. were based on a combined clinical and X-ray examination.
The meagre data supplied for
deciduous teeth.
Data regarding the caries rates of the permanent teeth were shown in each
report of this study; however, only very meagre data were published for the
deciduous ones. None were made available in the first report (Ast et al., 1950). In the following year
(Ast et al., 195 1) the DF rates per
100 deciduous teeth were given, but only for children aged five, six, seven and
eight years, and in each age group the rates had decreased both in the test and
in the control cities.
Unexplained marked decreases in the
control. No
explanation was given by Ast et al.
(1951) for the decreases in the DF rates in Kingston, where the greatest
relative decrease, from 32.1 per cent DF to 24.8 per cent DF, was seen in the
teeth of the six year-old children. It would have been of great interest to see
whether this trend was maintained in later years, but DF rates were not stated
in the tables contained in any of the later reports. However, in the following
one (Ast and Chase, 1953) the situation in regard to the deciduous teeth of
children five, six, seven and eight years old (now termed "def teeth per
100 deciduous teeth present") was depicted diagrammatically by means of a
histogram, these unexplained decreases in the def rates in the control city
being clearly seen, a small one at age five years, and considerable ones at the
ages of six, seven and eight years.
The increase in caries-free teeth
in the control. The only other information published regarding the
deciduous teeth was expressed in terms of "Children with caries free
deciduous cuspids, first and second molars". This type of table appeared
first in the 1951 report of Ast et al.,
and the results were given for only those children who were five or six years
of age. In both age groups in Kingston the figures suggest an increase in these
caries free teeth, the six-year-old children changing, between 1945-6 and 1949,
from 17.2 per cent to 25.5 per cent free from caries. These changes were mentioned,
but no attempt was made to explain them. In the next report (Ast and Chase,
1953) data for children aged seven years were also included. This report showed
that, between 1945-6 and 1951-2, the percentage of children in Kingston who had
these deciduous teeth free from caries showed a slight decrease at age five
years (28.2 per cent to 26.4 per cent); but in the six-year old children the
percentage increased from 17.2 to 26.3; and in those who were seven years of
age, it practically doubled (8.3 to 16.5). On this occasion, these changes in
the control city were not even mentioned.
"Analysis" of findings. In the 1955 report of Ast et al. it was stated that "As an
indication of the benefits of water fluoridation to deciduous teeth, a previous
report [in 1953] analyzed the findings among the 6 to 7 year old children in
each city after six to seven years following the initiation of water
fluoridation." Actually the report showed findings for the children aged
five, six and seven years who had caries-free deciduous cuspids, first and
second deciduous molars. At the ages of six and seven years, in both the test
and the control cities, there were increases in the percentages of these teeth
that were free from caries; therefore, although these increases were greater in
Newburgh, they should not have been attributed solely to water fluoridation. In
any case, the publication of one table showing, in this selected group of
deciduous teeth, the percentage changes that have just been mentioned, a
histogram depicting the def rates, and twenty lines of comment in the text on
the results displayed, can hardly be said to indicate the benefits of
fluoridation to deciduous teeth, or even to constitute an adequate analysis of
the findings in regard to the deciduous teeth present in children aged six and
seven years.
Changes in caries-free teeth in the
control. In the 1955 report of Ast et al. the age range was changed by not publishing the results for
the five-year old children, but showing, for the first time, the results for
caries-free deciduous cuspids first and second deciduous molars, for eight and
nine-year old children. However, these results cannot be compared with those of
the previous years, as they were based on a combined clinical and X-ray
examination. Nevertheless, a comparison can be made with the rates shown in the
final report (Ast et al., 1956). In
the year between the 1953-4 and 1954-5 examinations, the rates in Kingston for
the ages six, seven, eight and nine years changed from 10.6, 7.0, 7.9 and 0.0
to 11.1, 4.7, 1.8 and 1.6 for the respective ages. Such changes are not
unexpected, for marked variations were seen in Evanston, where, also, the
examinations were made by a clinical plus X-ray procedure. For instance, the
percentages of children aged seven years who were drinking fluoridated water
and who had caries free deciduous teeth were, in successive examinations, 11.33
(pre-fluoridation), 8.71, 3.87, 10,66, 13.01 and 17.86 (Hill et al., 1956). It would seem that
assessments made on the basis of caries-free groups of deciduous teeth are not
very reliable.
Changes in the sampling method. Consideration of these five dental
reports shows that the sampling method changed from time to time, and that the
method used in the control city was sometimes the same and sometimes different
from that used in the test one. In the first report (Ast et al., 1950) it was stated:
"we
are considering only those children age 6-12 who were in the original base
study and who have had each successive examination until they reach age 12.
Also included are new school children who entered the study at age 6 subsequent
to the first examination and were present at each of the successive
examinations. Thus, this study group will have only those children who we are
assuming have had continuous residence in their respective cities."
Continuous residence only assumed. It can be seen that the
"continuous residence" of each subject was based on assumption only,
and not on statements made in a questionnaire, such as was used in Evanston
(Blayney and Tucker, 1948). Therefore, it is possible that children could have
been absent from the city for considerable periods between the times of
successive examinations. Also, there is no assurance that the six-year-old
children entering the study in any of the post-fluoridation examinations had
not come to live in the area since the commencement of the study. Therefore, it
is doubtful whether the objective of having "reasonable assurance that the
children studied had had continuous residence in their respective cities"
(Ast et al., 195 1) can be said to
have been attained.
Population changes in Newburgh. "Early in 1950 questionnaires
were given to more than 3,200 children in the Newburgh schools for completion
by their parents" (Ast et al.,
195 1). The questions asked were not stated, nor was the number of replies
received, but it was said that:
An
analysis of the answers to those questionnaires shows that the Newburgh
population is a relatively stable one and that the inclusion of the small
migrant groups does not alter the caries picture to any significant degree.
Consequently, in this report there are included all 5 to 12 year old children
present in the schools in Newburgh and Kingston on the days the examinations
were made" (Ast et al., 195 1).
Since information
in regard to the caries attack rates in these migrant groups could not have
been obtained directly from the questionnaires, it is presumed that the dental
record cards of those children were grouped and that the cards of the children
who were judged from the answers not to be migrants were also grouped, and the
data contained in the two groups in regard to the caries attack rates were
compared. If that process was carried out, it was not mentioned, nor were data
published which would enable the reader to assess the situation. If no
differences were found between the two groups, it must be considered to be
strange because by that time it was said that "The DMF rates among
permanent teeth of 6 to 12 year old children in Newburgh show a consistent downward
trend" (Ast et al., 195 1). The
United Kingdom Mission (1953) reported that the authors of this study had
"found that the proportion of immigrants in Newburgh and Kingston was too
small to affect the comparison." However, although the Newburgh population
was said to be "relatively stable", in the 1954-5 examinations in
that city 24 per cent of the children were excluded because they failed to
fulfil the residence qualifications (Ast et
al., 1956).
The
workers who conducted the paediatric study in these cities, Schlesinger et al., in 1950 said that in each city
"An effort was made to select... children from families which might
reasonably be expected to remain for the duration of the study." In spite
of that precaution, they found that 29.9 per cent of their subjects in Newburgh
moved from the city during the period of the study (Schlesinger, Overton, Chase
and Cantwell, 1956).
Population movement in Kingston. No mention was made of the issue of
a questionnaire to children in the control city; apparently it was assumed that
migrants to that city would have come >from areas with
"fluoride-free" water supplies. Schlesinger et al. (1956) found that 22.2 per cent of the children included in
the paediatric examinations moved from Kingston during the period of the study;
presumably a similar number of new residents settled in the city.
It may be
considered that in moving from one locality to another, interruptions could
occur to regular conservative and prophylactic treatment of the children, so
that their dental health may not have been as good as that of children who
lived for many years in the same city. It is possible also that regular dental
examinations, by stimulating interest in the teeth, may improve eating habits
and oral hygiene measures.
Considerable alterations in
populations. In Table I of Ast et al. (1950) the number of permanent teeth erupted is shown. The
numbers given for Newburgh in the examination of 1944-5 for the three age
groups six to seven, eight to nine and ten to twelve years are respectively
3,579, 7,937 and 24,586. However, by adding in Table I of Ast and Chase (1953),
the number of erupted teeth - for the same age groups, and in the same
examination - are 5,379, 10,033 and 27,186. It was stated in the former report
that "we are considering only those children age 6-12 who were in the
original base study and who have had each successive examination until they
reach age 12." It therefore appears that to meet those requirements, it
was necessary to exclude, for the three age groups, 33 per cent, 21 per cent
and 10 per cent of the number of erupted teeth, and, presumably, similar
percentages of children. A like situation was seen in regard to the Kingston
data, the percentages of teeth excluded being 24, 26 and 12. After only four
years, it was apparently necessary to omit these large proportions of the data
in order to consider only those children who were "continuous
residents", no other explanation being evident for the different numbers
of erupted teeth that were stated in the two papers. Although the population of
Newburgh may have been "relatively stable" when compared with some
unnamed population, it is obvious that the number of migrants was so great that
they should have been excluded from the study.
Data of migrants excluded only in Newburgh. The necessity for excluding the
data of migrants was later realized, and the method of including in the study
all the children present in the schools on the day of the examination -
although it was continued in Kingston - was abandoned in Newburgh. Ast et al. (1955) stated: "Based on
residence histories, the Newburgh study group was limited to those who had used
Newburgh water since the introduction of sodium fluoride on May 2, 1945."
In the final report, also, only those children who had lived continuously in
Newburgh were included, but "All the Kingston children examined are
included in this report" (Ast et al.,
1956).
Alterations in sample size. The sample size and the age
distribution of the children were altered during the course of this study. The
data included in the first three dental reports were obtained from the
"entire elementary school populations" (Ast and Chase, 1953), except
that in some years some of the children were excluded in Newburgh on
residential grounds, and that in 1951-2, owing to the loss of an examiner, only
half of the children in each city were included. However, in the 1953-4 series
the age range was restricted to six to ten years, and the number of children
examined was only a small fraction of those inspected in the same age groups
during other examinations. Ast et al.
(1956) said that the preceding report "dealt with rather small groups of
children (about 375 children ages six to ten in each city), and there was
considerable difference in age distribution."
Sampling by selection. The
method of sampling used in the 1953-4 examination must be considered to be
unorthodox, and was described by Ast et
al. (1955) in these words:
"The
current series includes a limited number of schools which were chosen because
of the availability of X-ray facilities. From previous data on DMF rates by
school, it was determined that the selected Kingston school had a caries rate
which was among the lowest in the city, while the rates for the three Newburgh
schools were distributed through the range of rates for that city. This has the
effect of minimizing the difference in the DMF rates between the two
cities."
A decrease in the "per cent
difference".
In the final report (Ast et al.,
1956, Table 1) the "per cent difference" between the DMF rate per 100
erupted teeth of children aged six to nine years in Newburgh and Kingston was
given as 56.7. This is a smaller difference than any of those shown for the
ages six, seven, eight and nine years (74.7, 68.3, 58.1 and 66.0 respectively),
in the previous (1955) report, despite the fact that it was stated in that
report, that the sampling method used had minimized the difference between the
DMF rates in the two cities. A trial period of ten to twelve years was
suggested by Ast (1943), and was mentioned in the authors' first report (Ast et al., 1950). In view of the decrease
in the "per cent difference" between the test and the control cities,
which was revealed in the final report, it is unfortunate that the trial was
stopped as soon as the minimum period proposed by the authors had elapsed.
DISCUSSION
The Expert
Committee on Water Fluoridation of the World Health Organization (1958) stated
that "Hundreds of controlled fluoridation programmes are now in operation
in many countries. Some have been in progress for the past 12 years, so that
conclusions are based on experience." This statement suggests that there
is a large amount of experimental evidence in regard to the process of
artificial fluoridation. It is very doubtful whether this is the case. If hundreds
of fluoridation programmes have been conducted with experimental controls, it
is strange, and very unfortunate. that such a large body of data has not been
published; for, except in the cases of the trials which have been considered
here, published data concerning fluoridation trials are very meagre. It would
seem, therefore, that the Expert Committee did not use the term
"controlled" in its experimental sense, but in that of regulated
measurement of the fluoride salt, such as in its statement that "The
precision of fluoride application should be carefully controlled."
The United
Kingdom Mission (1953) which visited North America in 1952, in its report
referred to "the Fluoridation Studies", and enumerated only six study
centres; and Jenkins (1955) mentioned "the six study centres on the
American continent". In addition to the four test cities which have been
considered, the Mission referred to Sheboygan, Wisconsin, and to Marshall,
Texas. No control city for Sheboygan was mentioned; and the Mission stated that
in the latter study "The neighbouring town of Jacksonville with a
fluoride-free water supply was selected as control, but although caries
experience in the two areas was compared after 2 1/2 years of fluoridation, the
most valuable basis for comparison is the baseline data of Marshall
itself." The Mission quoted two unpublished reports as the source of its
information in regard to the latter study. An indication of the minor
importance of these two trials is the fact that in the 240-page report of the
New Zealand Commission of Inquiry (1957) no data from them were presented, the
former city being mentioned twice and the latter only once. Furthermore, the
directors of these studies, Doctors F A. Bull and E. Taylor, were not named in
the report, and the extensive bibliography did not include any papers published
by them.
The
crucial importance, even at the present time, of the trials conducted in
Newburgh, Grand Rapids, Brantford and Evanston was demonstraled by the report
made in 1957 by the New Zealand Commission, the hearings of which did not
conclude until April of that year, and by the report of the Expert Committee of
the W.H.O. (1958), which met during August 1957. The only evidence mentioned by
the Commission with regard to the dental results of the addition of fluorides
to water supplies was that obtained in those four cities. The Expert Committee
referred to only the first three of those cities in the few lines of its report
which mentioned dental results of fluoridation.
In
discussing the general design used in fluoridation studies, the United Kingdom
Mission (1953) said:
"In a
fluoridation study, two nearby towns, comparable in all respects, are chosen,
both having an almost fluoride-free domestic water supply, preferably from the
same source. The water of one town is fluoridated while that of the other
remains untreated, this town serving as the control. Before fluoridation is
started the teeth of the children in both towns are examined in detail to
ascertain if caries experience is similar and to determine its prevalence in
the various age groups. Further examinations are carried out at yearly
intervals and the dental condition of the children in the fluoridated town is
compared with that of similar groups in the control town. The prefluoridation
data also serve as a basis for comparison. The caries incidence may also be
compared with that in a town where a similar concentration of fluoride occurs
in the water naturally. In practice it is often difficult to obtain all these
conditions and in some studies there is no independent control."
The term
"comparable in all respects" describes a theoretical ideal for a test
and a control town rather than a practical possibility. In regard to the other
matters mentioned in the design these studies exhibited numerous deficiencies.
No control was employed in the City Council study in Brantford, and the Grand
Rapids study lost its control in 1951 as a result of the fluoridation of the
Muskegon water supply. In the extremely important matter of the water supplies,
both the source and the composition of the Newburgh water is considerably
different from that of Kingston. Further examinations were "carried out at
yearly intervals" only in Grand Rapids-Muskegon, and in Newburgh-Kingston
until 1952; if yearly examinations were made in the latter study after that
year, the results for all years were not published. In the Evanston trial, only
two examinations were made in the control city, and few data from it have been
published; in the test city only one age group was examined each year. In
Brantford and in Evanston, and in the first and the last dental reports from
the Newburgh-Kingston study, data from children of different yearly ages were
added, thus introducing the possibility of "weighting". In some instances,
at least, the degree of "weighting" found indicated that the
comparisons were not being made between similar groups in the test and the
control cities. No pre-fluoridation data were gathered in Brantford by the
Canadian Department of National Health and Welfare, for that study was not
commenced until over two and a half years after the fluoridation of the city
water supply.
In all of
the studies that have been considered, it has been seen that fluoridation of
the water supply of the test city was initiated before the initial caries rates
in the control city were known. This late examination of the control cities, on
first thought, may not seem to be of much consequence. However, it means that,
in all of these studies, a matter of fundamental importance was disregarded-it
could not have been established that the children of similar ages in the test
and the control cities, prior to the commencement of the experiment, had
reasonably comparable caries attack rates. Therefore, the statement of the
United Kingdom Mission (1953) that "Before fluoridation is started the
teeth of the children in both towns are examined in detail to ascertain if
caries experience is similar and to determine its prevalence in the various age
groups" appears to have been based on assumptions only.
Caries
attack rates may be expressed as decayed, missing and filled teeth per 100
erupted teeth, or expressed as per 100 children or per child. The former method
was preferred by the authors of the Newburgh trial "because individual
teeth may be subjected independently to the hazard of caries" (Ast et al., 1956). In the Evanston study,
the rate per 100 erupted teeth was given, but, curiously, only for children
aged twelve to fourteen years. All other cases (the younger children in
Evanston, and the other studies) in which the caries attack rates per 100
children or per child were given are based on the assumption, unsupported by
published evidence, that in each age group the mean numbers of each category of
erupted teeth per 100 children is very similar in the test and the control
cities, and that little variation occurs from year to year. If this is not the
case, comparisons between the rates prevalent in the test and the control
cities, and those seen in different years, are not valid. Feltman (1956) gave
fluorides in tablet form td-pregnant women and young children, and reported
that "Many children in the study group showed a marked delay in the
eruption of the deciduous teeth. This delay is in some instances a cause for
alarm by the parents. The second incisor, second molars, and cuspids are the
most frequently delayed, in many cases by as much as a year from the accepted
average eruption dates." Of course, if fluoridation results in the
eruption rate of teeth being retarded, a decrease in caries experience would be
expected due to the shorter time of exposure of the teeth to the risk of
caries. It will be recalled that data were published in the Evanston study
which were compatible with a continuous and marked decline in the rate of
eruption of first permanent molars during the first four to five years of
fluoridation, but that further comparisons could not be made because this type
of data was not published for younger children in later reports.
In order
to decrease the chance of misinterpretation, extensive use has been made of
direct quotation from the original reports, and to avoid unnecessary
repetition, consideration of the comments made on the results reported from
these control cities, apart from those made by the authors of these studies, has
been restricted to the statements of only a few writers.
It is felt
that it is not necessary to discuss further the matters which have been noted
above, for they are self-explanatory. It has been shown that the reports of the
controls used in these fluoridation trials contain arithmetical and statistical
errors, and that results and relevant data were omitted. Also, misleading
statements were made which denied, ignored, or underrated the unexplained
changes in caries attack rates which took place in the control cities, and
which suggested that the pre-fluoridation data from the test cities, and those
obtained during the basic examinations in control ones, were more closely
comparable than was the case. Jean R. Forrest, the Senior Dental Officer,
Ministry of Health, who was a member of the United Kingdom Mission and of the
Expert Committee on Water Fluoridation of the World Health Organization, in
1957 contrasted "the emotional type of opposition" to fluoridation,
to "the precise correct statements of scientists," However, the
situation which has been encountered is more aptly described by the words of
Wade Hampton Frost "an outstanding American epidemiologist" (Bews,
1951). More than thirty years ago Frost (1925) said:
"It
is frequently easy to exhibit some figures which, though not really to the
point, will nevertheless serve to impress an uncritical public, and the
temptation may be great to give them, at least by implication, an unduly
favourable interpretation. It is more difficult and more tedious to present the
full argument, based on all the facts, and it is perhaps a little humiliating
to admit that the statistical evidence is deficient because we have failed to
collect it; but to do this is not only more scientific, it is in the end more
convincing, and after all there is no free choice, because it is the only
honest method, whether it be convenient or not. Finally, it is the only way of
progress, for the first step towards collecting better evidence is to recognize
the deficiencies of that which is at hand."
More than
eleven years after the initiation of the last of these five trials, the
deficiencies of their controls still remain unrecognized. The endorsements of
fluoridation by medical and dental associations, by public health authorities,
and even the recent one by the Expert Committee on Water Fluoridation of the
World Health Organization (1958), appear to have been based mainly on the
opinions of the authors and of others. Indeed, in the report of that Committee,
under the heading "Results of fluoridation", instead of results being
considered, comment was confined to: "Reports of the results after 10
years of controlled fluoridation in three cities". Examinations of the
data obtained in these trials, which have been published by other endorsing bodies,
are also inadequate or absent. It is an understatement to term this failure
regrettable.
In 1951,
Appleton stated that in any future fluoridation trial: "The experiment
should be genuine, and not one in name only. In designing such an experiment, a
careful and competent analysis of those now in progress should first be made,
in order to see how they might be improved or extended."
In the
early part of this paper some basic experimental considerations were mentioned.
In cities in which it is intended to compare the caries attack rates of the
children in a fluoridation trial, the three main factors which should be as
closely comparable as is practically convenient are the composition of the
water supply, the climate and the dental caries attack rates. Four trials
having "fluoride-free" control cities have been considered. The
composition of Newburgh's water supply is considerably different from that of
its control city. There were gross differences between Evanston and its control
city regarding the initial caries attack rates in the younger children. In the
Department of National Health and Welfare study in Brantford, as the first
examination was made over two and a half years after the commencement of
fluoridation, it cannot be known what the pre-fluoridation rates in that city
would have been, if assessed by those examiners; thus, it cannot be determined
how closely the (1945) Brantford rates resembled those of Sarnia. In the Grand
Rapids study, the fluoridation of the water supply of Muskegon in 1951 severely
limited its usefulness as the control city.
In each
trial both the test city and its control were selected. For instance, "Oak
Park graciously offered to serve as the control community" for Evanston
(Blayney and Tucker, 1948). Two cities which had agreed to participate in the
experiment, after having been found suitable for comparison, should have been
allotted at random to be test or control. It will be recalled that in at least
two of the cities selected as test ones, Evanston and Brantford, "dental
care was outstandingly good" (United Kingdom Mission, 1953)
It has
been pointed out in Part One that in all these trials no attempt was made to
devise a randomization procedure in the clinical examinations which would have
eliminated examiner bias, nor were estimates made of examiner variability.
Two
statements made by authors of these studies may be recalled. In 1950 Hill et al. said: "It is to be expected
that the rate of caries in all teeth varies from year to year due to chance. A
significant reduction of caries prevalence can therefore be assumed to exist
only when the statistical analysis of the data provides almost absolute
certainty that the observed differences are not due to chance." However,
as was mentioned in Part One this very important matter of random variation has
been ignored in all these studies. Blayney and Tucker (1948) stated that:
"A study of this nature must have an adequate control." It has been
seen that the controls used in these trials cannot be considered to be adequate.
It would
appear that these shortcomings have not been recognized, for those who
conducted these studies, and other writers, have expressed their satisfaction
with the methods used. For instance, Ast and Chase, the authors of the 1953
report on the Newburgh-Kingston study, referred to "the carefully
controlled studies such as the Newburgh-Kingston, Grand Rapids-Muskegon, and
the Evanston studies"; and Mather (1957) said: "This study at
Brantford was most carefully set up and has been under the strictest
control."
Approval
of the methods used in these studies was also expressed by the New Zealand
Commission of Inquiry (1957), for it considered that: "All these
investigations" in Brantford, Newburgh and Grand Rapids "were
designed and executed with great thoroughness." The Commission also said:
"We have examined the statistical evidence brought forward by the
advocates of fluoridation, and the conclusions they have drawn from that
material ... We have found nothing to invalidate the statistics or cast doubt
on their reliability." It will be realized that many of the deficiencies
of these studies can be noted only when different reports >from the same
study are compared. It seems that the Commission was handicapped in this
regard, for although its "List of exhibits produced at public
hearings" mentions over 250 items, such as papers, books, charts and
letters, it includes only the final report, or the one which was the most
recently published at that time, of the numerous reports showing the dental
caries attack rates which were published in each of these studies. It would
appear that none of the earlier accounts of these trials were shown to the
Commission, nor were they mentioned in the "bibliography" of 144
references. None of the reports from the City Health Department trial in
Brantford were listed as exhibits. The paper by Brown, Kohli, Macdonald and
McLaren (1954a) which is mentioned deals only with gingival results. Although
the Commission had the assistance of legal counsel in gathering the evidence,
no mention was made of the employment of a statistician to assist its members
in evaluating the numerical data.
The Expert
Committee on Water Fluoridation of the World Health Organization (1958) also
expressed its satisfaction with the methods used in these trials. Out of the
hundreds of controlled fluoridation programmes which it stated have been set
up, it mentioned only the Newburgh, Grand Rapids and Brantford (City Health
Department) studies in the sixteen lines which allotted to the mention, one
cannot say consideration, of the results of fluoridation on dental caries
prevalence. Presumably these three trials were cited because the Committee
considered that they were the most important and reliable studies, and it said
that they were "carefully planned and controlled". As this opinion of
the Committee was made in referring to the three studies which it cited in
mentioning results of fluoridation, it is reasonable to assume, at least in
this instance, that the term "controlled" was used in its
experimental sense.
If this is
the case, the inaccuracy of that statement of the Expert Committee is
astonishing, for it will be recalled that, of the three studies which the
Committee quoted, the Brantford (City Health Department) study, far from being
carefully controlled, was not controlled at all. Furthermore, the control for
the Grand Rapids study was abandoned after only six years, at the crucial stage
of the trial when the first of the permanent teeth were erupting in the
children of the test city who had ingested fluoridated water throughout their
lives. Therefore the control , was abandoned before any assessment of caries
activity in those teeth could be made. In regard to the remaining study
mentioned by the Expert Committee, the Newburgh trial, after the unexplained
decreases in the DF rates for deciduous teeth, which were shown as having
occurred in Kingston, the control city, between 19456 and 1949, no further
caries rates for deciduous teeth were published. Also, the erratic changes
which were made in the methods used in this trial are not consistent with
careful planning, nor is the choice as the test and control areas of two cities
with water supplies which were of considerably different composition.
It has
been acknowledged for many years that one of the fundamental procedures in
planning an experiment is the establishment of a statistical design for the
procedures before work is commenced. The deficiencies in the basic statistical
requirements of a good experimental design are only too obvious in all these
studies. Therefore, it is surprising that the Expert Committee did not point
out these deficiencies, but, on the contrary, described the three studies which
it mentioned as carefully planned ones. The importance of these matters is
emphasized by the authoritative statement of Fisher (1951) that: "If the
design of an experiment is faulty, any method of interpretation which makes it
out to be decisive must be faulty too."
This
investigation of reports of these fluoridation trials was instituted when a
preliminary examination of the methods used revealed disturbing facts, and
solely because it was felt that, as Appleton (195 1) expressed it:
"Professionals and specialists have the duty of insisting upon a
scientific demonstration of a high probability that a proposed method will be
useful and safe, before it is recommended for general adoption. The maintenance
of this attitude is of paramount importance."
The
deficiencies of these trials not having been recognized, many cities have
already fluoridated their water supplies on advice which is based largely on
the results that have been considered. It is, therefore, an important and
urgent matter that a more accurate assessment of the efficacy of this process
should be obtained, but, unfortunately, it appears that little long-term
experimental evidence is available. Therefore, despite the limitations imposed
by the methods used in these studies, consideration should be given to a
careful and competent examination of the whole of the original data obtained in
them. The findings resulting from such an examination would be of assistance in
designing future fluoridation trials, and would provide a far more adequate
assessment of the results reported from these studies than it is possible to
obtain from an examination of the very limited data that have been published.
At least
until such a report is available for examination, it would be wise to maintain
an open mind in regard to the efficacy of artificial fluoridation.
SUMMARY
1.
Endorsements of the process of the mechanical fluoridation of public water
supplies rely mainly on five experimental trials.
2. The
controls used in these studies are considered.
3. The
reliability of the results reported is affected by:
(a) odd experimental and statistical methods;
(b) failure to consider random variation and examiner variability, and to
eliminate examiner bias;
(c) omission of relevant data;
(d) arithmetical errors;
(e) misleading comments.
4.
Controls were either doubtful or inadequate.
5. No
control was employed in one trial.
6. The
published data do not justify the statement that caries rates remained the same
in control cities.
7. The
sound basis on which the efficacy of a public health measure must be assessed
is not provided by these five crucial trials.
No comments:
Post a Comment