Sources Of Bias & Designing Experiment

Unit: Collecting Data

Chapter: Sources of Bias & Designing Experiment.

Reference: – Types of Bias & their sampling methods, non-responsive bias, Minimization, Under coverage bias, Voluntary response bias, Response bias, Experimental design, Controlled Experiments, Causation vs Correlation, Sampling techniques & Validity, Ethical considerations, Bias in surveys & Experiments

After studying this chapter, you should be able to:

• Types of Bias & their Sampling methods.
• Non – responsive Bias & Their minimization
• Causation vs Correlation & Ethical considerations.
• Sampling techniques & Validity.

Bias in Sampling methods

1. Random Sampling:
   1. Random sampling is a method where each member of the population has an equal chance of being selected for the sample.
   2. It helps reduce bias by ensuring that every individual has a fair opportunity to be included.
2. Systematic Sampling:
   1. Systematic sampling involves selecting every nth individual from the population.
   2. It can introduce bias if there's a pattern in the population that aligns with the sampling interval.
3. Stratified Sampling:
   1. In stratified sampling, the population is divided into distinct subgroups (strata) based on certain characteristics.
   2. It helps ensure representation from different groups, reducing bias by preventing underrepresentation.
4. Cluster Sampling:
   1. Cluster sampling involves dividing the population into clusters, then randomly selecting entire clusters for the sample.
   2. Bias can arise if clusters are not representative of the population or if there's a wide variability within clusters.
5. Convenience Sampling:
   1. Convenience sampling involves selecting individuals who are readily available or easy to reach.
   2. This method can introduce bias because it may not represent the entire population's characteristics.
6. Bias in Non-Random Sampling:
   1. Non-random sampling methods (e.g., convenience or judgment sampling) can lead to selection bias, where certain groups are overrepresented or underrepresented.
7. Voluntary Response Bias:
   1. Voluntary response bias occurs when individuals self-select to participate in a survey or study.
   2. The sample may not accurately represent the population because those with strong opinions are more likely to respond.
8. Under coverage Bias:
   1. Under coverage bias happens when certain segments of the population have a lower chance of being included in the sample.
   2. This can lead to results that don't accurately reflect the entire population.
9. Non-Response Bias:
   1. Non-response bias occurs when individuals selected for the sample do not respond or participate in the study.
   2. The non-respondents might have different characteristics, leading to a biased sample.
10. Random Sampling Error:
    1. Even with random sampling, there is a chance of random sampling error, where the sample's characteristics differ from the population's due to chance.
11. Sample Size and Bias:
    1. Smaller sample sizes are more susceptible to bias as chance variations can have a larger impact.
12. Sample Frame:
    1. The list or database from which the sample is drawn is the sample frame.
    2. A biased sample frame can lead to bias in the sample.
13. Overcoming Bias:
    1. Random selection and appropriate sampling methods can help mitigate bias.
    2. Stratified and cluster sampling can address biases related to specific population characteristics.
14. Bias vs. Variability:
    1. Bias refers to a consistent deviation from the true value, while variability refers to how much the data points differ from each other.
15. Representativeness:
    1. The goal of sampling is to create a representative sample that accurately reflects the population's characteristics, minimizing bias.

Types of Sampling & Their Explanation

1. Simple Random Sampling:
   • Explanation: Every member of the population has an equal chance of being selected for the sample. This is often done using random number generators or drawing lots.
   • Example: Selecting 50 students from a school by assigning each student a unique number and then using a random number generator to pick the numbers.
2. Stratified Sampling:
   • Explanation: The population is divided into subgroups or strata based on certain characteristics, and then a random sample is taken from each stratum in proportion to its size in the population.
   • Example: Dividing a city's population into age groups (e.g., 0-18, 19-35, 36-50, 51 and above) and then randomly selecting individuals from each age group.
3. Systematic Sampling:
   • Explanation: A starting point is chosen randomly, and then every nth member of the population is selected for the sample.
   • Example: Selecting every 10th customer entering a store to participate in a survey about their shopping habits.
4. Cluster Sampling:
   • Explanation: The population is divided into clusters (groups or areas), and a random sample of clusters is selected. All individuals within the selected clusters are included in the sample.
   • Example: Selecting a few schools from different districts and surveying all students within the selected schools.
5. Convenience Sampling:
   • Explanation: Individuals who are easiest to reach or are readily available are included in the sample.
   • Example: Conducting a survey of customers who visit a store on a particular day.
6. Voluntary Response Sampling:
   • Explanation: Individuals self-select to be part of the sample, often in response to an open invitation.
   • Example: Setting up an online poll where people can choose to participate by clicking a link.
7. Judgmental (or Purposive) Sampling:
   • Explanation: The researcher uses personal judgment to select individuals who are considered representative of the population.
   • Example: Selecting specific patients for a medical study based on their unique medical conditions.

Non-Responsive Bias & their Minimization

Non-Response Bias:

• Non-response bias occurs when individuals selected for a survey or study do not participate, leading to potential distortion in results.
• Underrepresentation:
• Non-respondents may differ systematically from respondents, leading to underrepresentation of certain groups or perspectives.

Example:

• In a political survey, if supporters of a particular candidate are less likely to respond, the survey results may be biased.
• Impact on Results:
• Non-response bias can lead to inaccurate estimates, distorted trends, and decreased generalizability of findings.
• Non-Random Non-Response:
• When non-response is related to the study's variables, it can lead to bias that is difficult to correct.

Causes of Non-Response:

• Factors such as time constraints, lack of interest, privacy concerns, or survey complexity can contribute to non-response.

Minimization Strategies:

• Ensuring clear and concise survey questions can reduce respondent burden and encourage participation.

     Pre-Survey Communication:

• Informing potential participants about the survey's importance and confidentiality can increase response rates.

Incentives:

• Offering incentives like small rewards can motivate individuals to participate and reduce non-response.

Follow-Up Efforts:

• Contacting non-respondents with reminders or additional survey opportunities can increase participation.

Non-Response Weighting:

• Assigning different weights to respondents and non-respondents based on known characteristics can help correct bias.

Imputation Techniques:

• Using statistical methods to estimate missing values based on responses from similar participants can mitigate non-response bias.

Non-Response Analysis:

• Analyzing characteristics of respondents and non-respondents can help identify potential biases and adjust for them.

Multiple Data Sources:

• Collecting information from various sources can provide a more complete picture and reduce reliance on a single biased sample.

Transparency and Reporting:

• Clearly documenting non-response rates and efforts to address bias enhances the study's credibility and allows for better interpretation of results.

Causation vs Correlation & Ethical Considerations

Causation vs. Correlation:

1. Causation:
   • Causation implies a cause-and-effect relationship where changes in one variable directly influence changes in another.
2. Correlation:
   • Correlation indicates a statistical relationship between two variables, but it does not necessarily imply a causal connection.
3. Third Variable Confounding:
   • Correlation between two variables can be influenced by a third variable that affects both, creating a spurious correlation.
4. Direction of Causation:
   • Establishing which variable causes the other can be challenging based solely on correlation.
5. Reverse Causation:
   • In some cases, the causation might be reversed, meaning changes in one variable are caused by changes in the other.
6. Coincidence:
   • Correlation can sometimes occur by chance, leading to a false perception of causation.
7. Experimentation and Causation:
   • Well-designed experiments can provide stronger evidence of causation by controlling variables and randomizing treatments.
8. Observational Studies:
   • Observational studies may reveal correlations but cannot definitively establish causation due to potential confounding variables.
9. Temporal Order:
   • Causation requires the cause to precede the effect in time, while correlation does not have this temporal requirement.
10. Magnitude of Association:
    • Strong correlation does not necessarily indicate strong causation; other factors must be considered.
11. Spurious Correlation:
    • Spurious correlations are false relationships caused by coincidental or unrelated factors.
12. Common-causal Variables:
    • Causation might be due to a common cause affecting both variables, creating a correlation.
13. Randomized Controlled Trials:
    • Randomly assigning treatments in experiments helps establish causation by minimizing confounding variables.
14. Strength of Evidence:
    • Causation requires rigorous evidence beyond just a strong correlation.
15. Scientific Theory and Mechanism:
    • A well-defined theoretical framework explaining how one variable influence another can provide stronger support for causation.

Ethical Considerations in AP Statistics:

1. Informed Consent:
   • Respecting participants' autonomy by providing clear information about the study's purpose and potential risks.
2. Privacy and Confidentiality:
   • Safeguarding participants' personal information and ensuring anonymity to maintain privacy.
3. Voluntary Participation:
   • Participants should not be coerced or pressured into taking part in a study.
4. Beneficence:
   • Ensuring that the study benefits participants or contributes to scientific knowledge.
5. Minimizing Harm:
   • Taking steps to minimize any potential physical, psychological, or emotional harm to participants.
6. Debriefing:
   • Informing participants about the study's true nature and purpose after data collection, especially in studies involving deception.
7. Respect for Vulnerable Populations:
   • Special consideration for individuals who may be at increased risk or unable to provide informed consent.
8. Avoiding Bias:
   • Ensuring that the study design and implementation are free from bias that could affect participants or results.
9. Data Handling and Security:
   • Protecting collected data from unauthorized access and ensuring secure storage.
10. Fair Representation:
    • Striving for fair representation of diverse groups in studies to avoid bias in results.
11. Transparency and Honesty:
    • Clearly and honestly communicating study methods, results, and limitations to participants and the public.
12. Long-Term Impact:
    • Considering the potential long-term consequences of the study on participants and society.
13. Respecting Cultural Norms:
    • Being sensitive to cultural norms and practices when designing and conducting studies.
14. Conflict of Interest:
    • Disclosing any potential conflicts of interest that could influence the study's objectivity.
15. Peer Review:
    • Submitting research for peer review to ensure ethical standards are met and research is sound.

Example: Investigating Bias in Survey Sampling

Scenario: A student is conducting a study to estimate the average amount of time high school students spend on social media each day. The student wants to ensure the survey design minimizes bias.

1. Potential Source of Bias: Self-Selection Bias
   • Issue: Students who choose to participate in the survey might have different social media usage patterns than those who do not participate.
   • Solution: Implement a random sampling method to select participants. Assign each student in the school a unique number and use a random number generator to select a sample of participants. This helps ensure that all students have an equal chance of being selected, reducing self-selection bias.
2. Potential Source of Bias: Non-Response Bias
   • Issue: Only a portion of selected students might respond to the survey, and their responses might differ from those who do not respond.
   • Solution: Follow up with non-respondents and encourage their participation. Alternatively, use techniques such as weighting to adjust for potential differences between respondents and non-respondents.
3. Potential Source of Bias: Volunteer Bias
   • Issue: Students who are willing to participate might have different social media habits than those who are not willing to participate.
   • Solution: Implement random sampling as mentioned earlier to reduce the likelihood of volunteer bias. Also, consider offering incentives to encourage participation without revealing the study's topic.
4. Potential Source of Bias: Under coverage Bias
   • Issue: The survey is conducted only within one school, which might not represent the broader population of high school students.
   • Solution: Randomly select a diverse set of schools to participate in the study, and then randomly sample students from each selected school. This helps ensure a more representative sample of high school students.
5. Potential Source of Bias: Response Bias
   • Issue: Students might underreport or overreport their social media usage due to social desirability bias or other reasons.
   • Solution: Use anonymous surveys to encourage honest responses. Additionally, consider using techniques such as randomized response methods to mitigate response bias.
6. Potential Source of Bias: Interviewer Bias
   • Issue: If the survey is administered in person by interviewers, their behavior, tone, or appearance might influence respondents' answers.
   • Solution: Provide interviewers with standardized training to ensure consistent administration of the survey. Consider using technology (e.g., online surveys) to minimize interviewer bias.
7. Potential Source of Bias: Sampling Frame Bias
   • Issue: The list of students from which the sample is selected might not be accurate or up-to-date.
   • Solution: Verify and update the sampling frame before conducting the survey to ensure all eligible students are included.

Key Points

Planning a Study

1. Research Objective: Clearly define the research question or objective that you want to address in your study.
2. Population: Identify the entire group or population you wish to study, ensuring it is well-defined and relevant to your research.
3. Sample: Determine a representative subset of the population, known as the sample, from which you will collect data.
4. Variables: Identify the variables of interest—those that you want to measure or analyze in your study.
5. Data Collection Method: Choose appropriate methods to collect data, such as surveys, experiments, observations, or existing records.
6. Bias Considerations: Be aware of potential sources of bias that could affect your study's results and take steps to minimize or account for them.
7. Ethical Considerations: Ensure that your study adheres to ethical guidelines, respects participant privacy, and obtains necessary approvals.

Sampling Methods:

1. Simple Random Sampling: Every member of the population has an equal chance of being selected for the sample.

1. Stratified Sampling: Divide the population into distinct subgroups (strata) and then randomly sample from each stratum.

1. Systematic Sampling: Select every nth element from the population to create the sample.

1. Cluster Sampling: Divide the population into clusters, randomly select some clusters, and then sample all elements within the selected clusters.

1. Convenience Sampling: Choose participants who are readily available or easy to reach, often leading to non-representative samples.

1. Voluntary Response Sampling: Individuals self-select to be part of the sample, introducing potential bias.

1. Judgmental (Purposive) Sampling: Select specific individuals or elements based on the researcher's judgment, which may introduce subjectivity.

1. Randomization: Use randomization techniques, such as random assignment or random selection, to minimize bias and enhance the validity of your study.