Selection bias occurs when a sample is not chosen completely at random and is therefore no longer representative. Selection Bias means that the distribution of characteristics is not the same as in the population.

What is the Selection Bias?

Selection bias refers to a bias in the composition of samples that can lead to the distortion of data in surveys or studies. Therefore, one must be careful when interpreting such data. Selection bias is not necessarily directly apparent but only becomes apparent upon close examination of the sample determination process.

A perfect sample is one in which every person in the population has the same probability of being represented. If this circumstance is not given, the sample is biased. The bias can be stronger or weaker, depending on the situation.

What are some examples of Selection Bias?

Suppose we want to find out how much money people spend on consumption on average. However, surveying all German adults would be too time-consuming and costly. Therefore, we decide to take a random sample and go to Munich’s city center to do so and survey random passers-by.

Due to the random selection of participants, we assume that our sample is meaningful. However, the publication of the results has been met with massive criticism. The accusation: selection bias! The following problems arise with our selection:

1. Does every German adult really have the same probability of appearing in the sample? What about adults from Berlin or Hamburg?
2. Is the income level of Munich comparable to the German average? If not, what impact does that have on our sample?
3. What about adults who generate their consumer spending primarily on the Internet and e-commerce? How would these individuals change the results?
4. What age groups do we encounter on a Friday afternoon in Munich? Which age groups might not be represented at that time?

The sampling errors do not always have to be as obvious as in our example. Sometimes they cannot even be prevented and must be included in the interpretation of the results.

Another example of selection bias for many is the choice of profession. If one relies only on experiences and opinions from one’s close family and friends, one is already subject to bias. This selection covers only a certain range of professions and is not representative of all possible professions. This distorts the result because people do not obtain any information at all or only very little information about certain occupational groups.

The following examples are also subject to bias:

• Surveys: People can decide for themselves whether or not to participate in a survey. This inevitably leads to bias because a certain group of people, those who do not participate in surveys, are not present in the sample.
• Direct Questioning: The way the results of the sample were collected can also have an impact on the bias. Most people will probably be uncomfortable admitting in a direct interview that they have ever driven drunk. In a written survey, on the other hand, more participants might answer honestly.

What are the types of Selection Bias?

There are a variety of reasons for selection bias. Here we have listed only the types of sampling bias that are most common:

• Attrition bias occurs when participants have dropped out of the study or survey prematurely and are not counted in the final results for this reason. It is important not to make the mistake of simply removing these subjects from the sample because, for example, the treatment did not work for them.
• A similar phenomenon is the so-called volunteer bias, where the bias arises because the participants actively agree to be part of the sample. Consenting to participate can already be a characteristic that distinguishes the sample from the population and thus distorts the result. In reality, this bias is often difficult to prevent. However, it should be taken into account when interpreting the results.
• Social bias occurs when the type of survey or study makes it highly likely that people will not answer truthfully. This can lead to the problem that not a truthful answer is given, but one that is socially acceptable or that puts the respondent in a better light.

Why does the sampling bias occur?

In addition to selection bias, other factors can lead to sampling bias. This chapter lists some points that should be checked before an experiment to minimize the risk of bias.

• Poor study design: The selection of the sample and the study design should always be questioned as to whether it is truly representative of the overall population being studied. This population group should then be precisely defined and narrowed down so that a good sampling procedure can be defined.
• Insufficient sample size: A sample that is too small increases the risk of an unrepresentative study. Of course, size is not everything, but there should be a balance between sample size and participant catchment radius depending on the budget and time frame. For example, if you want to make a statement about young people, you should not only survey students at your university, as this involves little effort.
• Incorrect data collection methods: The design and implementation of the questions or experiments should also be scrutinized, for example, to prevent interview bias due to suggestive questions. In addition, there may be distortions in the answers, for example to the question of how much sport a person does. Here, respondents often tend to give more optimistic answers to possibly look better socially. Such things should be taken into account when selecting and formulating questions.
• Self-selection of participants: Another reason for bias in the study is the self-selection of participants. In a survey on the political situation, for example, citizens who are interested in politics anyway and therefore came across the study are more likely to participate voluntarily. If possible, the research team should actively approach the participants and select them specifically to avoid a self-selection bias.
• Exclusion criteria: When selecting participants, bias can also occur if targeted groups or individuals are left out. For example, people with a pre-existing condition may not participate in certain studies. However, this exclusion should be taken into account when interpreting the results and their general validity.

There are many pitfalls and problems that can lead to bias in the results of a study. It is therefore important to scrutinize the study design and, above all, the selection of participants in order to minimize the risk of bias from the outset. However, it is not always possible to prevent the points mentioned. They should therefore be mentioned in the study results themselves in order to make the problems clear to the reader and to make it clear that the results may not be representative of the population as a whole. Concealing the risks of bias often makes such studies unnecessarily vulnerable.

What problems does selection bias cause?

Selection bias can have several consequences. First, it can lead to results that do not accurately reflect the population studied, leading to misleading conclusions or recommendations based on flawed data. Second, if a sample is not representative of the population as a whole, the results may not be generalizable to other populations. This may limit the applicability of the results and reduce the ability to make broader conclusions or recommendations. Third, selection bias may reduce the statistical power of a study, making it more difficult to detect significant differences or associations between variables.

Finally, evidence of selection bias can reduce confidence in the results and undermine the validity of the study or analysis. Overall, selection bias can have a significant impact on the accuracy, generalizability, and validity of study results, which can affect the ability to make informed decisions or recommendations based on the results.

How can Selection Bias be prevented?

The most important point in preventing selection bias is first of all the awareness of possible problems in one’s own experimental setup. In addition, some sampling biases simply cannot be prevented. If you want to conduct a large-scale study, for example in the medical field, you have to rely on volunteer participants and volunteer bias cannot be prevented.

Thus, no general tips can be given on how to avoid selection bias, as this depends strongly on the individual case. The only important thing is, to be honest when publishing the results and to provide as much information as possible about the sample creation. It is always helpful to be open about possible problems and to be transparent.

This is what you should take with you

• Selection bias, or sampling bias, occurs when a sample is not chosen completely at random and is therefore no longer representative.
• There are many different types of selection bias, such as volunteer bias or attrition bias, which can occur depending on the experiment.
• Possible strategies to prevent sampling bias depend on the individual case. However, it is important to be transparent about how samples were created when publishing results.
• Selection bias can significantly affect the accuracy, generalizability, and validity of study results.
• It can lead to misleading conclusions, reduced statistical power, and lower confidence in the results.
• Causes of selection bias can include sampling bias, nonresponse bias, and survival bias.
• Awareness of selection bias and its potential consequences is critical to ensuring the validity and reliability of research results.

Other Articles on the Topic of Selection Bias

• The University of Oxford has published a collection of biases here.