What changes when treatment labels are shuffled?
You will be able to: Distinguish a randomization distribution from sampling a population.
What changes when treatment labels are shuffled?
Four outcomes are 1,2,4,5 points. An experiment assigned two units to A and two to B. To explore no treatment effect, hold the outcomes fixed and reassign the labels.
A useful starting point: Why can averages look bell-shaped when individual values do not? →
Words and symbols before equations
- Randomization distribution
- Statistic values generated by repeatedly reallocating outcomes according to a study’s assignment rule.
- Null model
- A specific no-effect assumption used for a chance comparison.
- Difference in means
- Mean of A minus mean of B, with order stated.
- Reallocation
- A new allowed assignment of fixed outcomes to treatment groups.
What this picture assumes
Four fixed outcomes 1, 2, 4 and 5 are reassigned into two groups of two. All six complete assignments are equally likely in this simplified completely randomized design under a no-effect model. Outcomes stay fixed; labels move. Other designs require different valid shuffles.
Read the picture in three steps
- Read the axes and labels first. Identify what each symbol and line represents. Read the units and fixed conditions before comparing quantities.
- Selected difference -3. 2 of 6 equally likely reassignments have an absolute difference at least this large: 0.333. This compares outcomes under the stated no-effect model; it is not the probability that the model is true.
- Check what the picture assumes below. Use the Explore task to predict one change before moving a control.
Connect the picture to the mathematics
For a simple two-versus-two assignment, all six choices of which two units receive A are equally likely. Compute mean(A)−mean(B) for each and retain the sign.
The outcomes stay fixed; labels move. This differs from drawing new observations from a population. Under a sharp no-treatment-effect model consistent with random assignment, the reallocation distribution describes differences caused by assignment alone.
Respect the actual design: paired or blocked experiments require restricted shuffles. This lesson introduces distribution construction and comparison; formal hypothesis-test decisions come later. A small tail frequency is not the probability that a no-effect model is true.
| Feature | Sampling distribution | Randomization distribution |
|---|---|---|
| What varies | A fresh random sample | Allowed treatment-label assignments |
| What stays fixed | Population and sampling procedure | Observed outcomes under the no-effect model |
| One result | A statistic from one sample | A statistic from one valid reassignment |
A worked example, step by step
Enumerate differences for outcomes 1,2,4,5 when two receive A and two receive B.
- A={1,2} gives 1.5−4.5=−3; A={1,4} gives 2.5−3.5=−1.
- A={1,5} and A={2,4} each give 0.
- A={2,5} gives 1; A={4,5} gives 3.
- Thus the exact randomization distribution is −3,−1,0,0,1,3 with assignment probabilities 1/6 each; equal statistic values combine.
Shuffling is meaningful only under the stated no-effect model and the actual assignment mechanism. It does not make observational data randomized.
Should a matched-pairs experiment use unrestricted shuffling across everyone?
Compare with an explanation
No. Reallocation must respect the paired assignment rule, such as swapping treatment labels within pairs.
Predict. Change one thing. Explain.
Choose an observed A pair and inspect its signed difference. Compare its magnitude with all six allowed assignments. Explain which quantities stay fixed while labels change.
On narrow screens, swipe or scroll diagrams sideways to read all labels.
Selected difference -3. 2 of 6 equally likely reassignments have an absolute difference at least this large: 0.333. This compares outcomes under the stated no-effect model; it is not the probability that the model is true.
| Group A outcomes | Group B outcomes | Mean difference |
|---|---|---|
| 1, 2 | 4, 5 | -3 ← selected |
| 1, 4 | 2, 5 | -1 |
| 1, 5 | 2, 4 | 0 |
| 2, 4 | 1, 5 | 0 |
| 2, 5 | 1, 4 | 1 |
| 4, 5 | 1, 2 | 3 |
Four fixed outcomes 1, 2, 4 and 5 are reassigned into two groups of two. All six complete assignments are equally likely in this simplified completely randomized design under a no-effect model. Outcomes stay fixed; labels move. Other designs require different valid shuffles.
Explain what you noticed: Answer the investigation prompt above. State one observation and explain it using the probability values, reference groups, graph scales or model assumptions. Identify what the representation cannot tell you.
Apply the idea to a fresh problem Practice →Show what you understand.
Two original questions are a starting check, not proof of mastery. Explain your choice before revealing the answer.
Original written challenge
4 points · self-check · not an official AP questionFor outcomes 0,2,4,6 assigned two per group, find the difference when A gets 0,2 and when A gets 0,6. Explain the reallocation idea.
This response is not submitted or saved. Copy it before leaving.
Compare with the answer and four-point rubric
- 1 point: For A={0,2}, mean A=1 and mean B=5, so difference=−4.
- 1 point: For A={0,6}, both means are 3, so difference=0.
- 1 point: The values and group sizes stay fixed while labels are reassigned.
- 1 point: The reallocation rule must match the randomized experiment and a stated no-effect assumption.
Accept equivalent correct methods and explanations. This is a Refresh Kid teaching rubric, not an official AP scoring guideline.
Retrieve it before you reveal it.
RECALL 1What is reallocated?
Treatment labels, according to the allowed assignment scheme.
RECALL 2What does each distribution entry record?
The same statistic computed after one allowed reallocation.
RECALL 3Is a tail frequency the probability a hypothesis is true?
No; it is calculated under a specified model, not a probability of that model.
Revisit these tomorrow and a week later. Try a fresh problem and explain why the method applies.
What changes when treatment labels are shuffled?
- Keep outcomes fixed; reallocate labels using the original design.
- Recompute the same statistic for every allocation.
- Use absolute differences only when comparing both directions.
Remember: Shuffling is meaningful only under the stated no-effect model and the actual assignment mechanism. It does not make observational data randomized.
Conditions: Four fixed outcomes 1, 2, 4 and 5 are reassigned into two groups of two. All six complete assignments are equally likely in this simplified completely randomized design under a no-effect model. Outcomes stay fixed; labels move. Other designs require different valid shuffles.
Refresh Kid · AP Statistics Unit 2 · Objectives 2.12.A · Review edition
Framework, scope and review status
Mapped to College Board, AP Statistics CED, Topic 2.12, objectives 2.12.A. Framework effective Fall 2026, checked September 17, 2026. Unit 2 includes probability, random variables, probability models and introductory sampling distributions; it is part of the revised five-unit course.
Examples and datasets are synthetic, independently authored teaching material. Assumptions about independence, replacement and equal likelihood are stated before calculations. Simulation estimates fluctuate. Discrete probability is summed; continuous probability is area. Sampling distributions and randomization distributions use different repetition mechanisms. Formal inference comes in later units.
The Organic Chemistry Tutor companion title and destination were checked; the full video was not reviewed. Khan Academy’s destination was checked, but its lesson content was not fully readable by the research tool. OpenStax provides optional reference reading. No provider scripts, questions or graphics were copied. Refresh Kid is not affiliated with these providers.
GitHub’s 3D website collection informed optional spatial inspection. Our original three-toss outcome cube uses self-hosted Three.js with its MIT license. Eight corners represent eight equally likely sequences of three fair independent tosses. Conditioning removes ineligible sequences; camera rotation never changes probabilities. Quantitative graphs remain 2D to avoid perspective distortion. Complete labeled diagrams, outcome lists and explanations remain available without 3D.
Independent teacher review and observation of students remain pending. Technical checks do not certify statistical accuracy, accessibility or learning effectiveness. This is a review edition.
Released AP Statistics questions and scoring guides are optional. Older exams use the earlier framework, so check alignment before selecting parts. All practice on this page is original, not official AP material.
Learn → Explore → Practice → Review is informed by the IES learning guide. This implementation has not yet been evaluated with learners.
Want to work through this with a tutor?
Bring your question about What changes when treatment labels are shuffled? Your explanation and answers remain free to access.