Limits of science and pseudoscience
Background
In the public discussion of science, especially of controversial scientific research, the quality of individual scientific studies as well as the reputation of scientists is frequently criticised.
On the other hand, the overall high reputation of science and the scientific method as finding and expressing of natural and other phenomena, makes it appealing to appear “scientific”. Think of TV advertisements in which people try to come across as scientists by wearing a white lab coat and mentioning scientific studies (without references to the literature, of course) that claim that the toothpaste they want to sell you has positive effects.1
The motivation for this chapter is to identify the limits of science by defining how science progresses and coming up with a philosophical definition of what is science, and general criteria for defining good, bad and pseudoscience.
Limits of science
In the class on the scientific method we discussed that the scientific method acknowledges the limits of the scientific domain, i.e., the phenomena and questions that may or may not be accessible to investigation with the scientific method. However, how can one find out where the limits of science and the scientific method are? Closely related is the question what is science and what constitutes pseudoscience.
Philosophers of science have long thought about to differentiate between science and pseudoscience. This is well known as demarcation problem. In the 20th century, three important philosophers developed theories to achieve this goal:
- Karl Popper (1902-1994) focused on falsification as key approach to differentiate between science and pseudoscience.
- Thomas Kuhn (1922-1996) called scientific revolutions as the motor of scientific progress.
- Paul Feyerabend (1924-1994) stated that “Anything goes”. Accordingly, there is no unique scientific method and no real boundary between science and pseudoscience.
These theories will be outlined briefly in the following.
Falsification
For Karl Popper (Wikipedia), the main criterion to differentiate between science and pseudoscience were the following (Popper, 1934):
- No theory can be proven, they can only be falsified
- Scientific theories are falsifiable
The results in the challenge to develop a creative, audacious theory, which is falsifiable and then tested using empirical data. However, Popper’s theory has an important weakness: Scientists do not only want to show that certain theories are false. Instead they want to convince people that their own theories are true. For example, Albert Einstein responded in 1919 responding to the question of what he had done if measurements of his relativity theory would be false:
I would have been sorry for god - my theory is correct.
However, given the position of Popper that hypotheses can only be falsified, but not proven, we need to keep in mind that systematic attempts to falsify hypotheses is very efficient approach to scientific reasoning and that only hypotheses that are falsifiable are accessible to scientific investigation.
Scientific revolutions
The main hypothesis of the philosophy of Thomas Kuhn (Wikipedia) was that science does not proceed by falsifying theories, but by paradigm changes (Kuhn, 1962). Kuhn postulated the following steps of a scientific revolution:
- Scientific paradigm: There is an existing, widely accepted world view, the current paradigm.
- Normal science: Research is mainly aimed at confirming the existing paradigm.
- Anomaly: There is increasing number of discrepant results that are not consistent with the existing paradigm.
- Crisis and scientific revolution: A new theory and a new paradigm are being developed.
A prototypical example of this philosophy is transition from geocentric to heliocentric planetary system, which was brought about by Johannes Kepler.
Kuhn’s philosophy does not provide a strong indicator about the limits of science or pseudo-science, but provides a criterion of the limits of the current scientific mainstream. Since his theory is also rather generic (i.e., general), it does not consider methodology and puts a great importance on the scientific community as gatekeeper of scientific versus non-scientific theories.
“Anything goes”
The philosopher Paul Feyerabend (Wikipedia) is known for his theory for which he used the catch phrase “Anything goes”. According to Feyerabend this is the only possible description of the historical course of scientific research for rationalists (i.e., people who believe in reason). In the history of science there are many instances of breaking existing rules of the sciences, which led to the further advancement of science. Feyerabend mentions Galileo Galilei as a prime example. For this reason, there is no rational and generally valid rule of what is allowed and forbidden in science, for which one can guarantee that it does not prohibit scientific progress. Therefore, the expression ‘anything goes’ indicates that a methodology that claims to be universally valid needs to be blank and useless when compared to the actual history of science.
In summary, Feyerabend claims that science is not a single unified discipline that is based on a common foundation of methods and concepts. However, such a view is not shared by many natural scientists, because it does not reflect the daily experience of practising scientists. For this reason, the German biologist Axel Meyer called Feyerabend a ‘Dadaist philosopher’ and a ‘cynic and provocative clown’ (Meyer, 2011).
However, in an opposing view of a philosopher, Feyerabend is ‘critical not of science itself, but of false and misleading images of the sciences’ and that he ‘warned his peers that mere abstract reflection on the sciences would produce only idealised fantasies of science, rather than workable models of it’ (Kidd, 2011).
Pseudoscience
In the following, we define pseudoscience somewhat more formally and give examples.
Pseudoscience refers to a collection of beliefs, practices, or methodologies that claim or appear to be scientific but lack the rigor, evidence, and systematic approach that characterize true science. Pseudoscience often relies on anecdotal evidence, fails to follow the scientific method, and is not open to testing or refutation. It typically lacks empirical support and is often incompatible with existing scientific knowledge.
Key characteristics of pseudoscience include:
- Lack of empirical evidence: Pseudoscientific theories are often not based on empirical evidence gathered through systematic observation or experimentation.
- Non-falsifiability: Pseudoscientific theories are often non-falsifiable, meaning they cannot be tested in a way that could potentially refute them.
- Reliance on anecdotal evidence: Pseudoscience often relies on personal stories and anecdotes rather than systematic research.
- Absence of peer review: Pseudoscientific claims are usually not subjected to peer review or scrutiny by the wider scientific community.
- Overreliance on confirmation: Pseudoscience tends to focus on evidence that supports its claims while ignoring evidence that contradicts them. This is also known as Cherry-picking.
- Lack of self-correction: Pseudoscience does not evolve in light of new evidence; it often clings to its original claims regardless of contradictory evidence.
- Unexplained jargon and complex terminology: Often uses scientific-sounding language that does not make logical sense or is used incorrectly.
One key problem with pseudoscience is that such explanations frequently are highly appealing for lay people love it and that reputable scientists do not want to deal with explaining the problems with pseudoscience to the general because they consider it to be distraction from their main activity, namely to conduct research.
There are several areas in which pseudoscience is important to this day:
- Alternative medicine
- Alternative medicine comprises numerous treatments that are not part of the evidence-based medical canon.
- An important example is homeopathy: There is no scientific evidence that this therapy works, but its proponents make many claims about as yet undiscovered scientific principles that may explain homeopathy.
- For comprehensive treatments of alternative medicine from a scientist’s point of view, see Singh and Ernst (2008).
- Climate change
- Many people, companies and institutions with vested interests (still) deny the existence of human-caused climate change by presenting their own facts.
- This strategy is well described in Oreskes and Conway (2011) (Wikipedia).
- Organic agriculture
- Since its origin, organic agriculture (at least in Germany) has become a kind of substitute religion.
- In particular, bio-dynamic agriculture (which was invented by the anthroposophic guru Rudolf Steiner and is today marketed under the Demeter label) claims that it is superior to other forms of agriculture because it also makes use of cosmic forces (Kirchmann, 1994).
- Genetic engineering in plants
- Critics of genetic engineering (particularly in plants) often propose that the direct manipulation of genes is unnatural (Zwart, 2009).
- However, by promoting a simplistic concept of naturalness, they often ignore scientific facts and promote their own views on how plants adapt to their environment.
- These views are often influenced by Lysenkoism (Wikipedia) that states that crops do not adapt by genetic, but epigenetic processes and therefore plant breeding and cultivation should follow this approach.
The ‘science wars’ - Relativism vs. Scientific Realism
In addition to pseudoscience there was also a movement coming out from social science that were dubbed ‘science wars’, which is essentially an attack of social scientists on natural scientists. The key criticism was that “Science is a social construction of reality”. The social scientists proposed that what is considered as ‘truth’ depends a lot on sociological processes (‘group dynamics’). This theory was advanced by social scientists who called their field “Science studies”.
One goal was to replace science’s objectivity and realism with relativism. There were various reactions by natural scientists, which were highly critical of this movement and are, for example, summarized by Gottfried and Wilson (1997):
Scientists eventually settle on one theory on the basis of imperfect data, whereas logicians have shown that a finite body of data cannot uniquely determine a single theory. Among scientists this rarely causes insomnia, but it has tormented many a philosopher…
However, many scientists refuted such a view with the following arguments.
Scientists are generally convinced that science has a firm grasp on reality for several compelling reasons. Firstly, the ability of science to make steadily improving predictions that are often unambiguous, precise, diverse, and sometimes even surprising, demonstrates its deepening understanding of the natural world. These predictions, when consistently validated, reinforce the reliability of scientific methods.
Secondly, the accumulation of increasingly accurate and extensive data underpins this confidence. As data collection methods become more sophisticated, the breadth and depth of information available for analysis expand, allowing for more nuanced understanding of complex phenomena.
Thirdly, the development of increasingly specific and comprehensive theories offers robust frameworks for explaining a wide array of phenomena, linking disparate observations into coherent models.
Fourthly, the integration of evidence from diverse fields of study provides a cross-verification of findings, strengthening the overall picture of reality that science provides.
Fifthly, the undeniable progress over time in describing and explaining nature is a testament to the effectiveness of the scientific method. This progress is not just theoretical but is also evidenced in practical applications, as seen in the reproducibility of experiments. The ability to replicate findings under different conditions and by different researchers is a cornerstone of scientific reliability.
Lastly, the success of science-based technology that works in everyday life – from smartphones to medical treatments – serves as a tangible demonstration of science’s ability to understand and manipulate reality effectively.
This convergence of factors contributes to the strong conviction among scientists that science indeed has a good grip on the reality of the natural world.
Biodynamic agriculture as an example for pseudoscience
An example for pseudoscience is biodynamic agriculture. Biodynamic agriculture is often criticized for being based on pseudoscientific principles because it integrates certain practices and beliefs that lack empirical support and are not grounded in scientific evidence.
Developed in the 1920s by Rudolf Steiner, this approach includes the use of preparations made from fermented manure, minerals, and herbs, which are believed to enhance soil health and plant growth in a spiritually and astrologically harmonious manner. For instance, planting and harvesting are scheduled according to lunar and astrological cycles, a practice that has no scientific backing in agronomy or plant sciences.
While some aspects of biodynamic farming, like its emphasis on biodiversity and ecological sustainability, align with conventional agricultural science, the foundational principles involving cosmic forces and rhythmic cycles are not supported by empirical research, making them pseudoscientific. This blend of spiritual and mystical elements with agriculture places biodynamic farming in stark contrast to evidence-based agricultural practices.
Nevertheless, proponents of biodynamic agriculture state on one hand to operate within the framework of the scientific method, on the other hand claim that they are also operating outside of it, because of the special cosmic qualities that are important and differentiate it from other organic agricultural system.
This world view is expressed in the following quote, that has been taken from an editorial in a scientific journal that reports on various research projects on biodynamic agriculture (Brock et al., 2019). It is worthwhile to quote the complete introduction to the editorial:
Biodynamic farming is a growing movement with a high reputation among farmers and consumers. It is rooted in anthroposophy, an ontological system introduced by Rudolf Steiner (1861-1925) in the early 20th century. As such, biodynamic farming has a strong spiritual component, and both processes and effects are assumed to occur not only on the physical, but also on a metaphysical level. For example, the effect of the biodynamic preparations - defined natural remedies that are considered a core element of biodynamic farming - is explained by a facilitation of cosmic forces. Physical effects of the preparations can be measured with scientific methods, but the presumed mode of effect is not comprehensible with natural sciences until now.
The situation of research in biodynamic farming there-fore has much in common with the situation of research on traditional ecological knowledge: In both cases, physical traits and effects are comprehensible by the western scientific knowledge system, while the explanation for the effects is provided by a particular or indigenous knowledge system. The epistemological differences between the two knowledge systems usually prevent an integrated application, but it is possible to co-produce knowledge in a complementary way (Berkes 2009). This is what is done in biodynamic food and farming research since the emergence of that production system. The examination and assessment of biodynamic food and farming with methods from the scientific framework of natural sciences is only one part of the research landscape, while the other part is connected to a particular, indigenous epistemology of anthroposophy. It is important to note that the epistemology of anthroposophy does not exclude, but include the methods of natural sciences. In other words: the epistemology of natural sciences is fully comprehensible by anthroposophy, but anthroposophy is not fully comprehensible by natural sciences. The co-production of knowledge with scientific and ‘indigenous’ knowledge systems is well-established in the biodynamic sector.
However, it is not surprising that scientific research in biodynamic food and farming has usually addressed physical traits and effects until now, even though there have been successful attempts to elaborate methods that both allow for a more holistic assessment of effects, and meet the requirements of scientific methodology.
This text allows to make several observations:
- The scientific method, primarily focused on physical effects, is not yet able to uncover the effects of biodynamic preparations, which purportedly rely on cosmic forces. It is argued that while anthroposophy comprehends the scientific method, the latter fails to grasp anthroposophy. This rhetoric seemingly links biodynamic agriculture with scientific methodology while also distinguishing them from each other.
- Comparing biodynamic agriculture to ecological research, particularly in terms of ‘indigenous knowledge’, the authors suggest this disconnect is not unique to biodynamics. They highlight the scientific method’s limitation to physical phenomena, contrasting it with ‘indigenous’ knowledge, which is exclusive to certain groups of people and eludes formal scientific description and causal explanation.
- The authors differentiate the concept of a ‘western scientific knowledge system’ from ‘indigenous’ knowledge systems. This challenges the scientific method’s claims to objectivity and realism. By framing biodynamic agriculture as rooted in a unique indigenous knowledge system, it undermines objectivity, being accessible only to those with special insights, akin to ‘illuminates’ found in various belief systems. Furthermore, it contests scientific realism and leans towards relativism, echoing postcolonial critiques of Western rationalism and its impact on indigenous knowledge systems.
- The authors of this editorial affiliated with organizations supporting anthroposophy, organic agriculture research, and biodynamic agriculture. It is notable that biodynamic agricultural products are marketed under the Demeter label (frequently at premium prices) for their added value, despite the lack of scientific validation.
In conclusion, the editorial clearly positions biodynamic agriculture as pseudoscientific, primarily due to its deviation from the principles of the scientific method and its rejection of falsiability based on the notion that “indigenous” forces of life can not be investigated with a reductionist approach that is ascribed to the scientific method.
Good Science, Bad Science and Pseudoscience
In addition to a philosophical definition of science and its limits, there are also definitions that allow to differentiate good (i.e. properly carried out) science from bad science or pseudoscience.
Cargo Cult Science
An interesting concept of pseudoscience is the concept of Cargo Cult Science that was described at the Caltech commencement speech in 1974 by the Nobel-prize winning physicist Richard Feynman. It is based on the notion of ‘cargo cults’ in anthropology.
Cargo cults refer to a religious practice in tribal societies that are aimed at achieving material wealth ('cargo') through magic and religious rituals.2 These cults developed after industrialization mostly on islands in the South Pacific and became frequent during and after World War II because US and Japanese troups brought a lot of cargo to islands. The natives on these islands (who were still living in archeal societies) believed the goods were produced by gods. After the war, flow of cargo ceased and the local societies built mock airstrips and airplanes in the hope to attract further goods (Figure 2).
The term cargo cult science of R. Feynman refers to a practice that seems to be scientific, but in reality is not because
a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty
is missing (Feynman, 1974).
He then goes on to define how to avoid cargo cult science:
- Doubt your own theories
- Dishonesty common in daily life, advertising and politics is inacceptable in science
- A high level of personal integrity is required
- Example of a cargo cult approach: Use results from other studies as control instead of designing your experiment such that it includes its own experimental control
A similar definition of science is the Baloney Detection Kit by the American physicist and science advocate Carl Sagan (1934-1996) which was described in his book “The Demon Haunted World” (Sagan, 1995). It can be viewed as a kind of ‘How To’ for scientific working and a guide to differentiate it from pseudoscience or bad science.
Similar versons of the baloney kit have been popularized as part of efforts to improve the public understanding of science. One example on a guide to recognizing bad science is outlined in this ‘cheat sheet’.
Statistical cargo cults
The metaphor of the cargo cult science is used in an article entitled “Cargo-cult statistics and scientific crisis” by two statisticians who claim that cargo cult science is very widespread (Stark and Saltelli, 2018). They claim that an inadequate statistical training of practicing scientists leads to the mechanical application of statistical procedures without a deeper understanding on which premises and models these tests are based, but whose use is greatly facilitated by modern statistical packages. For this reason, the quality of scientific results is low and they can not reproduced. They make various suggestions to improve the situation which includes:
- Open science: all data and methods need to be described sufficiently well and be publicly available
- Better statistical training of scientists
- Critical reviewing of statistical work in scientific papers by experts in statistics
- Direct involvement of statisticians together with other citizens in social and environmental projects of relevance for the society
Further Reading
- Hugh Gauch (2002) The Scientific Method in Practice, Chapter 2 - An overview of pseudoscience
- Feynman, Cargo Cult Science: The text of his commencement speech is available as PDF
- Stark and Saltelli (2018) - A very good explanation what a statistical cargo cult is. PDF
- Sagan (1995), Ch. 12 - The fine art of baloney detection. The classical description of how to spot pseudoscience. PDF
Summary
- Philosophers thought deeply about the demarcation problem, i.e., how to differentiate between proper science and pseudoscience
- Karl Popper focused on falsifiability as key criterion to differentiate between science and pseudoscience, whereas Thomas Kuhn called scientific revolutions as motor of progess in which anomalies are the cause of revolutions. Paul Feyerabend postulated that there is no unique scientific method and hence no differentiation between science and pseudoscience.
- Key characteristics of pseudoscience include a lack of empirical evidence, non-falsifiability, a reliance on anectodal evidence, an absence of peer review, overreliance on confirmation, a lack of self-correction and unexplained jargon and complex terminology.
- Current examples of pseudoscience are alternative medicine and biodynamic agriculture. In the discussion of climate change and genetic engineering in plants pseudoscientific arguments are frequently used.
- Cargo cult science can best be described as mock science and includes bad scientific practice as well as pseudoscience.
Key concepts
Study questions
- One criticism of Popper’s philosophy of falsiability is that scientists want to show that hypotheses are true, not that they are wrong. One could argue that this is more a psychological than a philosophical problem. What is your opinion on this?
- Think about the radical philosophical theory of Paul Feyerabend’s ‘Anything goes’: If there exist no unifying method and concepts in science, how can one evaluate whether scientific research is good or close to revealing truth about nature, or not?
- Why do you think practicing natural scientists may not care about philosophical definitions of the limits of science or pseudoscience?
- Can you think of a formal definition for cargo cult science?
- What are the dangers for a young scientist to fall into the trap of cargo cult science?
In class exercises
Changing the taste of apples by eurythmic treatments
We discuss the paper by @ in different groups.
Group 1: Hypothesis
- What is the hypothesis of the paper?
- What are the premises for the hypothesis?
- Are the premises well defined and justified? - What is your opinion about it?
Group 2: Materials and Methods
- Which methods have been used by the authors?
- Are they explained in sufficient detail?
- Do you find the selection and justification of the methods convincing?
- Would one be able to repeat and replicate the experiment?
Group 3: Experimental design
- What is the experimental design of the study?
- Are the experiments and analysis justified and appropriate to answer the hypothesis of the paper?
- Do you see that the main principles of experimental design are implemented and used, or do you recognize any confounding factors or potential biases?
- Does the design allow for a falsification of the hypothesis
Group 4: Presentation of the results*
- Are the results presented in a clear manner and with sufficient detail?
- Does the presentations of the results support the message of the paper and of the authors?
- Do you consider the level of statistical significance appropriate and the effects observed relevant for the analysis?
Group 5: Discussion of the results
- Is the discussion supported by the results?
- Are the conclusions justified and what are the wider implications of the study?
- Do you see how any conflicts of interest in the study?
All groups: General criticism
- What are your general criticism of the study?
- Would you consider the study as an appropriate scientific study?
- Any other comments?
Similar studies
See also: