Limits of science and pseudoscience
Methods of scientific working
Motivation
In the public discussion of science, especially of controversial scientific research, the quality of individual scientific studies as well as the reputation of scientists is frequently criticised.
On the other hand, the overall high reputation of science and the scientific method as finding and expressing of natural and other phenomena, makes it appealing to appear “scientific”. Think of TV advertisements in which people try to come across as scientists by wearing a white lab coat and mentioning scientific studies (without references to the literature, of course) that claim that the toothpaste they want to sell you has positive effects.1
The motivation for this chapter is to identify the limits of science by defining how science progresses and coming up with a philosophical definition of what is science, and general criteria for defining good, bad and pseudoscience.
Limits of science
In the previous chapter on the scientific method we discussed that the scientific method acknowledges the limits of the scientific domain, i.e., the phenomena and questions that may or may not be accessible to investigation with the scientific method.
However, how can one find out where the limits of science and the scientific method are? Closely related is the question: What is science and what constitutes pseudoscience?
Philosophers of science have long thought about to differentiate between science and pseudoscience. This is well known as demarcation problem. In the 20th century, three important philosophers developed theories to define what constitutes proper science and differentiate it from pseudoscience:
- Karl Popper (1902-1994) focused on falsification as key approach to differentiate between science and pseudoscience.
- Thomas Kuhn (1922-1996) called scientific revolutions as the driver of scientific progress.
- Paul Feyerabend (1924-1994) stated that “Anything goes”. Accordingly, there is no unique scientific method and no real boundary between science and pseudoscience.
These theories will be outlined briefly in the following.
Important philosophers of science in the 20th century.
Falsification
For Karl Popper2, the main criterion to differentiate between science and pseudoscience were the following (Popper, 1934):
- No theory can be proven correct, they can only be falsified
- Scientific theories are falsifiable
The results in the challenge to develop a creative, audacious theory, which is falsifiable and then amenable for testing using empirical data. However, Popper’s theory has an important weakness: Scientists do not only want to show that certain theories are false. Instead they want to convince people that their own theories are true. For example, Albert Einstein responded in 1919 responding to the question of what he had done if measurements of his relativity theory would be false:
I would have been sorry for god - my theory is correct.
However, given the position of Popper that hypotheses can only be falsified, but not proven, we need to keep in mind that systematic attempts to falsify hypotheses is very efficient approach to scientific reasoning and that only hypotheses that are falsifiable are accessible to scientific investigation.
The demarcation problem
The demarcation problem addresses how to distinguish science from pseudoscience by developing criteria that are both universally accepted and practically applicable. This challenge has been debated since antiquity, because already present in Hippocratic texts (500 BCE) that differentiate genuine medical theory from quackery.
The term demarcation problem was introduced by Karl Popper in 1953 in a talk, and published in 1963 in his book Conjectures and Refutations. Heproposed falsifiability as the key criterion of scientific status: a theory is scientific only if it can, in principle, be refuted by empirical evidence. While influential and widely used by practicing scientists, this approach faces several difficulties.
It is often unclear whether contradictory results truly falsify a theory or merely reflect measurement or technical errors, and it requires that hypotheses specify in advance what evidence would count as disconfirming. Yet, the absence of such evidence does not guarantee truth. The saying goes that “Absence of evidence is not evidence of absence”.
Some false claims may also be unfalsifiable simply because no suitable evidence can be generated. Historical sciences such as cosmology, geology, or evolutionary biology cannot perform controlled experiments, which makes strict falsification possible: The tape can not be played again.
Moreover, Popper’s criterion paradoxically renders even absurd claims “scientific” if they specify improbable conditions under which they could be rejected. It implies that scientific theories can never be proven true, only be considered “not yet falsified,” which is counterintuitive for non-experts.
Subsequently, philosophers have therefore refined or broadened the concept, for example by considering multiple dimensions such as the amount of empirical evidence and theoretical understanding in a field, or by formulating local demarcation criteria tailored to specific domains, rather than seeking a single universal criterion for demarcation between science and pseudoscience.
The falsificability criterion leads to additional problems. As already mentioned because science is adversarial and competitive, one does not make a good careers refuting other people’s hypotheses, but to discover new phenomena about nature. In this, one has to be the first (for a given finding) and be more correct and competitors, e.g, by producing larger data sets.
Furthermore, science is quite expensive. Winners attract more money and become more powerful, wheres “losers”, e.g. researchers with creative, new but seemingly “fringe” theories are marginalised.
Therefore, there is a certain area in science, that new claims appear controversial at the beginning and are considered “fringe”, “exotic”, or even “esoteric”, until more evidence is produces. Such new claims could be considered as pseudoscience if they are difficult to prove using available means.
In summary this discussion shows that science is a social enterprise where unconvential, new ideas may not be successful in their adoption by the scientific community.
Scientific revolutions
The social aspect of science and very dynamic nature of scientific investigation is expressed in the following theory. The main hypothesis of the philosophy of Thomas Kuhn (Wikipedia) was that science does not proceed by falsifying theories, but by paradigm changes (Kuhn, 1962). Kuhn postulated the following steps of a scientific revolution:
- Scientific paradigm: There is an existing, widely accepted world view, the current paradigm.
- Normal science: Research is mainly aimed at confirming the existing paradigm.
- Anomaly: There is increasing number of discrepant results that are not consistent with the existing paradigm (e.g., “Outliers”)
- Crisis and scientific revolution: A new theory and a new paradigm are being developed.
A prototypical example of this philosophy is transition from geocentric to heliocentric planetary system, which was brought about by Johannes Kepler.
Kuhn’s philosophy does not provide a strong indicator about the limits of science or pseudo-science, but provides a criterion of the limits of the current scientific mainstream. Since his theory is also rather generic (i.e., general), it does not consider methodology and puts a great importance on the scientific community as gatekeeper of scientific versus non-scientific theories.
“Anything goes”
The philosopher Paul Feyerabend3 is known for his theory for which he used the catch phrase “Anything goes”. According to Feyerabend this is the only possible description of the historical course of scientific research for rationalists (i.e., people who believe in reason). In the history of science there are many instances of breaking existing rules of the sciences, which led to the further advancement of science. Feyerabend mentions Galileo Galilei as a prime example. In his book Against Method (1975) Feyerabend argues for epistemiological anarchism from the effort of Galileo Galilei to defend Kepler’s heliocentric cosmology against the teachings of the Catholic church, which was based on the geocentric view of Aristotle (Figure 4 (b)). According to Feyerabend, Galileo introduced new interpretations whenever empirical evidence was strongly in contrast with the rotation of Earth, against the existing dogma, thereby breaking the methodological rules (of the time), but defending the scientific method. For this reason, there is no rational and generally valid rule of what is allowed and forbidden in science, for which one can guarantee that it does not prohibit scientific progress. Therefore, the expression ‘anything goes’ indicates that a methodology that claims to be universally valid needs to be blank and useless when compared to the actual history of science.
In summary, Feyerabend claims that science is not a single unified discipline that is based on a common foundation of methods and concepts. However, such a view is not shared by many natural scientists, because it does not reflect the daily experience of practising scientists. For this reason, the German biologist Axel Meyer called Feyerabend a ‘Dadaist philosopher’ and a ‘cynic and provocative clown’ (Meyer, 2011).
In an opposing view, Feyerabend is ‘critical not of science itself, but of false and misleading images of the sciences’ and that he ‘warned his peers that mere abstract reflection on the sciences would produce only idealised fantasies of science, rather than workable models of it’ (Kidd, 2011).
Pseudoscience
In the following, we define pseudoscience somewhat more formally and give examples.
Pseudoscience refers to a collection of beliefs, practices, or methodologies that claim or appear to be scientific but lack the rigor, evidence, and systematic approach that characterize true science. Pseudoscience often relies on anecdotal evidence, fails to follow the scientific method, and is not open to testing or refutation. It typically lacks empirical support and is often incompatible with existing scientific knowledge.
Key characteristics of pseudoscience include:
- Lack of empirical evidence: Pseudoscientific theories are often not based on empirical evidence gathered through systematic observation or experimentation.
- Non-falsifiability: Pseudoscientific theories are often non-falsifiable, meaning they cannot be tested in a way that could potentially refute them.
- Reliance on anecdotal evidence: Pseudoscience often relies on personal stories and anecdotes rather than systematic research.
- Absence of peer review: Pseudoscientific claims are usually not subjected to peer review or scrutiny by the wider scientific community.
- Overreliance on confirmation: Pseudoscience tends to focus on evidence that supports its claims while ignoring evidence that contradicts them. This is also known as Cherry-picking.
- Lack of self-correction: Pseudoscience does not evolve in light of new evidence; it often clings to its original claims regardless of contradictory evidence.
- Unexplained jargon and complex terminology: Often uses scientific-sounding language that does not make logical sense or is used incorrectly.
One key problem with pseudoscience is that such explanations frequently are highly appealing for lay people love it and that reputable scientists do not want to deal with explaining the problems with pseudoscience to the general because they consider it to be distraction from their main activity, namely to conduct research.
There are several areas in which pseudoscience is important to this day:
- Alternative medicine
- Alternative medicine comprises numerous treatments that are not part of the evidence-based medical canon. An important example is homeopathy: There is no scientific evidence that this therapy works, but its proponents make many claims about as yet undiscovered scientific principles that may explain homeopathy. For comprehensive treatments of alternative medicine from a scientist’s point of view, see Singh and Ernst (2008).
- Climate change
- Many people, companies and institutions with vested interests (still) deny the existence of human-caused climate change by presenting their own facts. This strategy is well described in Oreskes and Conway (2011)4.
- Organic agriculture
- Since its origin, organic agriculture (at least in Germany) has become a kind of substitute religion. In particular, bio-dynamic agriculture, which today is marketed under the Demeter label (Figure 6). It was invented by Rudolf Steiner. Proponents of bio-dynamic agriculture claim that it is superior to other forms of agriculture because it also makes use of cosmic forces (Kirchmann, 1994).
- Genetic engineering in plants
- Critics of genetic engineering (particularly in plants) often propose that the direct manipulation of genes is unnatural (Zwart, 2009). However, by promoting a simplistic concept of naturalness, they often ignore scientific facts and promote their own views on how plants adapt to their environment. These views are often influenced by Lysenkoism (Wikipedia) that states that crops do not adapt by genetic, but epigenetic processes and therefore plant breeding and cultivation should follow this approach.
Lysenkoism
The Sovjet agriculturalist Trofim Lysenko funded Lysenkoism, which was a political movement that rejected natural selection and Mendelian genetics in the improvement of crop plants, and instead promoted the pseudscientific theory of acquired traits by exposing plants to particular environments, which stands in the tradition of Lamarckism (Figure 7).
It is a prime example of a politicied science, in which scientific (or pseudoscientific) ideas are promoted by the goverment.
The politication of science is not restricted to the 20th century. It can also be observed now with the long-running claim that vaccines cause autism in children (Figure 8). This claim is based on false science, but has gained substantial traction among conspiracy theorists and the anti-vaccine movement, to which the current secretary of health of the United States, Robert F Kennedy Jr. can also be aligned.
Biodynamic agriculture as an example for pseudoscience
An example for pseudoscience is biodynamic agriculture. Biodynamic agriculture is often criticized for being based on pseudoscientific principles because it integrates certain practices and beliefs that lack empirical support and are not grounded in scientific evidence.
Developed in the 1920s by Rudolf Steiner (Figure 9), this approach includes the use of preparations made from fermented manure, minerals, and herbs, which are believed to enhance soil health and plant growth in a spiritually and astrologically harmonious manner. For instance, planting and harvesting are scheduled according to lunar and astrological cycles, a practice that has no scientific backing in agronomy or plant sciences.
While some aspects of biodynamic farming, like its emphasis on biodiversity and ecological sustainability, align with conventional agricultural science, the foundational principles involving cosmic forces and rhythmic cycles are not supported by empirical research, making them pseudoscientific. This blend of spiritual and mystical elements with agriculture places biodynamic farming in stark contrast to evidence-based agricultural practices.
Nevertheless, proponents of biodynamic agriculture state on one hand to operate within the framework of the scientific method, on the other hand claim that they are also operating outside of it, because of the special cosmic qualities that are important and differentiate it from other organic agricultural system.
This world view is expressed in the following quote, that has been taken from an editorial in a scientific journal that reports on various research projects on biodynamic agriculture (Brock et al., 2019). It is worthwhile to quote the complete introduction to the editorial:
Biodynamic farming is a growing movement with a high reputation among farmers and consumers. It is rooted in anthroposophy, an ontological system introduced by Rudolf Steiner (1861-1925) in the early 20th century. As such, biodynamic farming has a strong spiritual component, and both processes and effects are assumed to occur not only on the physical, but also on a metaphysical level. For example, the effect of the biodynamic preparations - defined natural remedies that are considered a core element of biodynamic farming - is explained by a facilitation of cosmic forces. Physical effects of the preparations can be measured with scientific methods, but the presumed mode of effect is not comprehensible with natural sciences until now.
The situation of research in biodynamic farming there-fore has much in common with the situation of research on traditional ecological knowledge: In both cases, physical traits and effects are comprehensible by the western scientific knowledge system, while the explanation for the effects is provided by a particular or indigenous knowledge system. The epistemological differences between the two knowledge systems usually prevent an integrated application, but it is possible to co-produce knowledge in a complementary way (Berkes 2009). This is what is done in biodynamic food and farming research since the emergence of that production system. The examination and assessment of biodynamic food and farming with methods from the scientific framework of natural sciences is only one part of the research landscape, while the other part is connected to a particular, indigenous epistemology of anthroposophy. It is important to note that the epistemology of anthroposophy does not exclude, but include the methods of natural sciences. In other words: the epistemology of natural sciences is fully comprehensible by anthroposophy, but anthroposophy is not fully comprehensible by natural sciences. The co-production of knowledge with scientific and ‘indigenous’ knowledge systems is well-established in the biodynamic sector.
However, it is not surprising that scientific research in biodynamic food and farming has usually addressed physical traits and effects until now, even though there have been successful attempts to elaborate methods that both allow for a more holistic assessment of effects, and meet the requirements of scientific methodology.
This text allows to make several observations:
- The scientific method, primarily focused on physical effects, is not yet able to uncover the effects of biodynamic preparations, which purportedly rely on cosmic forces. It is argued that while anthroposophy comprehends the scientific method, the latter fails to grasp anthroposophy. This rhetoric seemingly links biodynamic agriculture with scientific methodology while also distinguishing them from each other.
- Comparing biodynamic agriculture to ecological research, particularly in terms of ‘indigenous knowledge’, the authors suggest this disconnect is not unique to biodynamics. They highlight the scientific method’s limitation to physical phenomena, contrasting it with ‘indigenous’ knowledge, which is exclusive to certain groups of people and eludes formal scientific description and causal explanation.
- The authors differentiate the concept of a ‘western scientific knowledge system’ from ‘indigenous’ knowledge systems. This challenges the scientific method’s claims to objectivity and realism. By framing biodynamic agriculture as rooted in a unique indigenous knowledge system, it undermines objectivity, being accessible only to those with special insights, akin to ‘illuminates’ found in various belief systems. Furthermore, it contests scientific realism and leans towards relativism, echoing postcolonial critiques of Western rationalism and its impact on indigenous knowledge systems.
- The authors of this editorial affiliated with organizations supporting anthroposophy, organic agriculture research, and biodynamic agriculture. It is notable that biodynamic agricultural products are marketed under the Demeter label (frequently at premium prices) for their added value, despite the lack of scientific validation.
In conclusion, the editorial clearly positions biodynamic agriculture as pseudoscientific, primarily due to its deviation from the principles of the scientific method and its rejection of falsiability based on the notion that “indigenous” forces of life can not be investigated with a reductionist approach that is ascribed to the scientific method.
Good Science, Bad Science and Pseudoscience
In addition to a philosophical definition of science and its limits, there are also definitions that allow to differentiate good (i.e. properly carried out) science from bad science or pseudoscience.
Cargo Cult Science
An interesting concept of pseudoscience is the concept of Cargo Cult Science that was described at the Caltech commencement speech in 1974 by the Nobel-prize winning physicist Richard Feynman. It is based on the notion of ‘cargo cults’ in anthropology.
Cargo cults refer to a religious practice in tribal societies that are aimed at achieving material wealth ('cargo') through magic and religious rituals.5 These cults developed after industrialization mostly on islands in the South Pacific and became frequent during and after World War II because US and Japanese troups brought a lot of cargo to islands. The natives on these islands (who were still living in archeal societies) believed the goods were produced by gods. After the war, flow of cargo ceased and the local societies built mock airstrips and airplanes in the hope to attract further goods (Figure 11).
The term cargo cult science of R. Feynman refers to a practice that seems to be scientific, but in reality is not because
a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty
is missing (Feynman, 1974).
He then goes on to define how to avoid cargo cult science:
- Doubt your own theories
- Dishonesty common in daily life, advertising and politics is inacceptable in science
- A high level of personal integrity is required
- Example of a cargo cult approach: Use results from other studies as control instead of designing your experiment such that it includes its own experimental control
A similar definition of science is the Baloney Detection Kit by the American physicist and science advocate Carl Sagan (1934-1996) which was described in his book “The Demon Haunted World” (Sagan, 1995). It can be viewed as a kind of ‘How To’ for scientific working and a guide to differentiate it from pseudoscience or bad science.
Similar versons of the baloney kit have been popularized as part of efforts to improve the public understanding of science. One example on a guide to recognizing bad science is outlined in this ‘cheat sheet’.
Statistical cargo cults
The metaphor of the cargo cult science is used in an article entitled “Cargo-cult statistics and scientific crisis” by two statisticians who claim that cargo cult science is very widespread (Stark and Saltelli, 2018). They claim that an inadequate statistical training of practicing scientists leads to the mechanical application of statistical procedures without a deeper understanding on which premises and models these tests are based, but whose use is greatly facilitated by modern statistical packages. For this reason, the quality of scientific results is low and they can not reproduced. They make various suggestions to improve the situation which includes:
- Open science: all data and methods need to be described sufficiently well and be publicly available
- Better statistical training of scientists
- Critical reviewing of statistical work in scientific papers by experts in statistics
- Direct involvement of statisticians together with other citizens in social and environmental projects of relevance for the society
Further Reading
- Hugh Gauch (2002) The Scientific Method in Practice, Chapter 2 - An overview of pseudoscience
- Feynman, Cargo Cult Science: The text of his commencement speech is available as PDF
- Stark and Saltelli (2018) - A very good explanation what a statistical cargo cult is. PDF
- Sagan (1995), Ch. 12 - The fine art of baloney detection. The classical description of how to spot pseudoscience. PDF
Summary
- Philosophers thought deeply about the demarcation problem, i.e., how to differentiate between proper science and pseudoscience
- Karl Popper focused on falsifiability as key criterion to differentiate between science and pseudoscience, whereas Thomas Kuhn called scientific revolutions as motor of progess in which anomalies are the cause of revolutions. Paul Feyerabend postulated that there is no unique scientific method and hence no differentiation between science and pseudoscience.
- Key characteristics of pseudoscience include a lack of empirical evidence, non-falsifiability, a reliance on anectodal evidence, an absence of peer review, overreliance on confirmation, a lack of self-correction and unexplained jargon and complex terminology.
- Current examples of pseudoscience are alternative medicine and biodynamic agriculture. In the discussion of climate change and genetic engineering in plants pseudoscientific arguments are frequently used.
- Cargo cult science can best be described as mock science and includes bad scientific practice as well as pseudoscience.
Key concepts
Study questions
- One criticism of Popper’s philosophy of falsiability is that scientists want to show that hypotheses are true, not that they are wrong. One could argue that this is more a psychological than a philosophical problem. What is your opinion on this?
- Think about the radical philosophical theory of Paul Feyerabend’s ‘Anything goes’: If there exist no unifying method and concepts in science, how can one evaluate whether scientific research is good or close to revealing truth about nature, or not?
- Why do you think practicing natural scientists may not care about philosophical definitions of the limits of science or pseudoscience?
- Can you think of a formal definition for cargo cult science?
- What are the dangers for a young scientist to fall into the trap of cargo cult science?
In class exercises
Changing the taste of apples by eurythmic treatments
We discuss the paper by @ in different groups.
Group 1: Hypothesis
- What is the hypothesis of the paper?
- What are the premises for the hypothesis?
- Are the premises well defined and justified? - What is your opinion about it?
Group 2: Materials and Methods
- Which methods have been used by the authors?
- Are they explained in sufficient detail?
- Do you find the selection and justification of the methods convincing?
- Would one be able to repeat and replicate the experiment?
Group 3: Experimental design
- What is the experimental design of the study?
- Are the experiments and analysis justified and appropriate to answer the hypothesis of the paper?
- Do you see that the main principles of experimental design are implemented and used, or do you recognize any confounding factors or potential biases?
- Does the design allow for a falsification of the hypothesis
Group 4: Presentation of the results
- Are the results presented in a clear manner and with sufficient detail?
- Does the presentations of the results support the message of the paper and of the authors?
- Do you consider the level of statistical significance appropriate and the effects observed relevant for the analysis?
Group 5: Discussion of the results
- Is the discussion supported by the results?
- Are the conclusions justified and what are the wider implications of the study?
- Do you see how any conflicts of interest in the study?
All groups: General criticism
- What are your general criticism of the study?
- Would you consider the study as an appropriate scientific study?
- Any other comments?
Similar studies
See also: