What makes an experiment a fair comparison?
You will be able to: Describe random assignment, comparison, replication, control and masking in context.
What makes an experiment a fair comparison?
A teacher compares two feedback formats. Giving the new format only to the strongest class would mix format effects with prior skill.
A useful starting point: Why can a huge survey still mislead? →
Words and symbols before equations
- Random assignment
- A chance mechanism allocates treatments to experimental units.
- Control group
- A reference treatment group for comparison.
- Replication
- More than one independent experimental unit receives each treatment.
- Blinding or masking
- Keeping treatment identity unknown to participants or assessors where feasible.
- Placebo
- An inactive treatment resembling the active one.
What this picture assumes
24 synthetic units: IDs 1–12 are beginners and 13–24 experienced. A and B receive 12 units each. Skill is a potential influence on the response. No outcomes or treatment effect are simulated.
Read the picture in three steps
- Read the axes and labels first. Identify what each symbol and line represents. Read the units and fixed conditions before comparing quantities.
- Treatment A: 3, 5, 7, 11, 13, 16, 18, 19, 20, 21, 22, 23. A has 4 beginners and 8 experienced units; B has the remaining units. Allocation does not simulate an effect.
- Check what the picture assumes below. Use the Explore task to predict one change before moving a control.
Connect the picture to the mathematics
Use chance to assign units, compare at least two conditions, include enough units in each treatment, and keep other relevant conditions consistent. Random assignment balances uncontrolled factors in expectation, not perfectly in every experiment.
A control may be the standard method rather than no treatment. A placebo can separate the effect of expecting treatment from the active ingredient. The placebo effect compares response to placebo with response to no treatment.
Single masking hides treatment identity from one relevant group, such as participants or assessors; double masking hides it from participants and interacting research staff. A teaching format may be impossible to hide from students, but anonymous scoring by assessors can still reduce bias. Consent and privacy remain necessary.
A worked example, step by step
Plan a study of two feedback formats with 40 volunteers.
- Label volunteers and randomly assign 20 to each format using a shuffled list.
- Keep task, time, instructions and testing conditions consistent.
- Use the standard format as a comparison and 20 independent volunteers per group as replication.
- Have scorers assess anonymized work without knowing format; compare scores with a method that accounts for chance, and limit population claims because recruitment used volunteers.
Repeating a measurement on one person is not the same as replication with independent experimental units.
Does random assignment guarantee the two groups match exactly?
Compare with an explanation
No. It prevents systematic allocation based on characteristics, but chance imbalance can remain.
Predict. Change one thing. Explain.
Switch from assigning by prior skill to random assignment. Inspect the skill mix in both groups across seeds. Explain why random assignment helps without guaranteeing exact balance.
On narrow screens, swipe or scroll diagrams sideways to read all labels.
Treatment A: 3, 5, 7, 11, 13, 16, 18, 19, 20, 21, 22, 23. A has 4 beginners and 8 experienced units; B has the remaining units. Allocation does not simulate an effect.
| Group | Selected IDs / treatment A |
|---|---|
| 1 | 3, 5 |
| 2 | 7, 11 |
| 3 | 13, 16, 18 |
| 4 | 19, 20, 21, 22, 23 |
24 synthetic units: IDs 1–12 are beginners and 13–24 experienced. A and B receive 12 units each. Skill is a potential influence on the response. No outcomes or treatment effect are simulated.
Explain what you noticed: Answer the investigation prompt above. State one observation and explain it using the data values, graph scales, summary statistics or study-design conditions. Identify what the representation cannot tell you.
Apply the idea to a fresh problem Practice →Show what you understand.
Two original questions are a starting check, not proof of mastery. Explain your choice before revealing the answer.
Original written challenge
4 points · self-check · not an official AP questionThirty seedlings will receive fertilizer A or B. Propose a controlled, replicated randomized comparison.
This response is not submitted or saved. Copy it before leaving.
Compare with the answer and four-point rubric
- 1 point: Randomly allocate 15 independent seedlings to A and 15 to B.
- 1 point: Keep light, water, pot size and growth period consistent.
- 1 point: Measure a pre-specified response such as growth in centimeters.
- 1 point: Use masked measurement where feasible and assess chance variation before attributing a difference to fertilizer.
Accept equivalent correct methods and explanations. This is a Refresh Kid teaching rubric, not an official AP scoring guideline.
Retrieve it before you reveal it.
RECALL 1What is replication within a study?
Several independent units receive each treatment.
RECALL 2Must a control group receive nothing?
No; it may receive a standard comparison treatment.
RECALL 3What does random assignment reduce?
Systematic confounding between treatment and other characteristics.
Revisit these tomorrow and a week later. Try a fresh problem and explain why the method applies.
What makes an experiment a fair comparison?
- Compare, randomize, replicate and control.
- Mask treatment identity where feasible.
- Do not infer a true effect from any observed difference alone.
Remember: Repeating a measurement on one person is not the same as replication with independent experimental units.
Conditions: 24 synthetic units: IDs 1–12 are beginners and 13–24 experienced. A and B receive 12 units each. Skill is a potential influence on the response. No outcomes or treatment effect are simulated.
Refresh Kid · AP Statistics Unit 1 · Objectives 1.13.A, 1.13.C, 1.13.D · Review edition
Framework, scope and review status
Mapped to College Board, AP Statistics CED, Topic 1.13, objectives 1.13.A, 1.13.C, 1.13.D. Framework effective Fall 2026, checked September 17, 2026. Unit 1 includes one-variable data and data collection; it is part of the revised five-unit course.
Examples and datasets are synthetic, independently authored teaching material. Quartile calculations state a median-of-halves convention. Outlier screens identify values to investigate, not data to discard. Random selection and random assignment have different inferential roles. These lessons introduce design and descriptive reasoning; formal inference comes in later units.
The Organic Chemistry Tutor companion title and destination were checked; the full video was not reviewed. Khan Academy’s destination was checked, but its lesson content was not fully readable by the research tool. OpenStax provides optional reference reading. No provider scripts, questions or graphics were copied. Refresh Kid is not affiliated with these providers.
GitHub’s 3D website collection informed optional spatial inspection. Our original sampling model uses self-hosted Three.js with its MIT license. It shows labeled units in four groups; camera rotation does not change the sampling procedure. Quantitative graphs remain 2D to avoid perspective distortion. Complete labeled diagrams, selected IDs and explanations remain available without 3D.
Independent teacher review and observation of students remain pending. Technical checks do not certify statistical accuracy, accessibility or learning effectiveness. This is a review edition.
Released AP Statistics questions and scoring guides are optional. Older exams use the earlier framework, so check alignment before selecting parts. All practice on this page is original, not official AP material.
Learn → Explore → Practice → Review is informed by the IES learning guide. This implementation has not yet been evaluated with learners.
Want to work through this with a tutor?
Bring your question about What makes an experiment a fair comparison? Your explanation and answers remain free to access.