AQA Syllabus focus:
'Probability and significance, including statistical tables, critical values, Type I errors and Type II errors.'
In inferential testing, psychologists judge whether findings are likely to be genuine or simply due to chance. This requires understanding probability, significance levels, statistical tables, critical values, and decision-making errors.
Probability and significance
When psychologists analyze results, they ask how likely it is that the pattern they found happened by chance alone. This is the idea of probability.
Probability is the likelihood that an event or result will occur by chance.
In statistical testing, a result is judged against a significance level, usually 5% in AQA Psychology.
This figure shows a standard normal distribution with two critical cutoffs defining the rejection regions in both tails. It helps you see how a two-tailed significance level (e.g., ) allocates a small area to the extreme outcomes regarded as too unlikely under the null hypothesis. The critical values mark the boundaries between “likely by chance” and “unlikely by chance” results. Source
If the probability of the result occurring by chance is small enough, the result is described as statistically significant.
Statistical significance means that the probability of a result occurring by chance is small enough to be considered unlikely, usually .
A significance level of 5% means there is a 5% or lower probability that the result happened by chance. Put another way, psychologists are usually willing to accept a small risk of being wrong in order to decide that a result is unlikely to be random.
This does not mean a result is definitely true. It means the result is unlikely enough to have occurred by chance that the researcher treats it as meaningful evidence. Statistical significance is therefore about probability, not certainty.
A result can also be not significant. This means the result is not unusual enough to rule out chance as a reasonable explanation. It does not prove that there is no effect; it simply means the evidence is not strong enough at the chosen significance level.
Statistical tables and critical values
Psychologists do not decide significance by guesswork. They use statistical tables for the test they have carried out. These tables show the cutoff points needed for a result to count as significant.
The cutoff point used to make this decision is called the critical value.
This diagram highlights a critical value on a Student’s distribution and labels it in terms of the confidence/probability level and degrees of freedom. It reinforces the idea that critical values come from statistical tables and depend on parameters like (df) and the chosen significance level. Visually, the critical value is the threshold that separates typical outcomes from rare outcomes under the null model. Source
A critical value is the value from a statistical table that a test result must reach or exceed, depending on the test, in order to be significant at a chosen level.
Statistical tables are important because they take into account factors such as the chosen significance level and, for some tests, the size of the data set. This makes the judgment more objective and standardized.
The basic decision process is:
carry out the appropriate statistical test
obtain the calculated or observed value from that test
find the correct critical value in the statistical table
compare the observed value with the critical value
decide whether the result is significant
The exact comparison rule depends on the test being used. For some tests, the observed value must be equal to or greater than the critical value. For others, it must be equal to or less than the critical value. The key idea is that the critical value marks the boundary between a result that is likely enough to be due to chance and one that is unlikely enough to be called significant.
Because statistical tables are based on probability, they help researchers make decisions in a consistent way rather than relying on personal judgment.
Type I and Type II errors
Even when statistical procedures are followed correctly, decisions about significance can still be wrong. These mistakes are called Type I and Type II errors.
This diagram illustrates the idea of classification mistakes: false positives and false negatives. In hypothesis testing language, a Type I error corresponds to a false positive (concluding there is an effect when there isn’t one), and a Type II error corresponds to a false negative (missing a real effect). The visual separation makes it easier to map each error type to the decision outcome. Source
A Type I error happens when a researcher concludes that a result is significant when it is actually due to chance.
Type I error is a false positive, where a researcher decides there is a significant effect when no real effect exists.
This means the researcher has incorrectly treated a chance finding as meaningful. Type I errors matter because they can lead psychologists to believe they have found an effect, difference, or relationship that is not really there.
A different problem can also occur when a real effect is missed.
Type II error is a false negative, where a researcher decides a result is not significant when a real effect does exist.
In this case, the study fails to identify something genuine. Type II errors matter because potentially useful findings may be overlooked.
These two errors show that inferential decisions always involve some risk. A significance level such as 5% helps control that risk, but it cannot remove it completely. If a researcher uses a stricter significance level, such as 1%, this reduces the chance of a Type I error, but it may make Type II errors more likely because the standard for significance becomes harder to reach.
Interpreting significance carefully
Psychologists must be careful not to overstate what significance means.
A significant result means:
the finding is unlikely to be due to chance alone
the result has passed the chosen significance threshold
the researcher has evidence to support a genuine finding
A significant result does not mean:
the finding is definitely true
the effect is large or important
errors are impossible
Likewise, a non-significant result does not automatically mean that nothing is happening. It only means the evidence is not strong enough at the chosen level of probability.
Understanding probability, significance, statistical tables, critical values, Type I errors, and Type II errors allows psychologists to make more accurate and more cautious decisions about research findings.
Practice Questions
What is meant by a Type I error in inferential testing? (2 marks)
1 mark for stating that it is a false positive or an incorrect significant result.
1 mark for stating that the researcher concludes there is a real effect or difference when the result is actually due to chance.
Explain how psychologists use statistical tables and critical values to decide whether a result is significant. Refer to one Type I error and one Type II error in your answer. (6 marks)
1 mark for stating that the researcher obtains a calculated or observed test value.
1 mark for stating that the correct statistical table must be used.
1 mark for identifying that the critical value is taken from the table at the chosen significance level.
1 mark for explaining that the observed value is compared with the critical value to decide significance.
1 mark for correctly outlining a Type I error as deciding a result is significant when it is actually due to chance.
1 mark for correctly outlining a Type II error as deciding a result is not significant when a real effect exists.
FAQ
A 1% significance level is stricter than 5%. It means the researcher is only willing to accept a 1% chance that the result is due to random variation.
This may be chosen when:
the consequences of a false positive are serious
the claim being tested is especially important
the researcher wants stronger evidence before calling a result significant
The trade-off is that it becomes harder to reach significance, so genuine effects may be missed more often.
In formal statistical decision-making, a result is either significant at the chosen level or it is not. “Almost significant” is not a proper statistical category.
However, researchers sometimes use phrases such as:
“approached significance”
“showed a trend”
These phrases should be treated cautiously. They do not mean the result has crossed the required threshold. In exams, it is safer to say the result is either significant or non-significant based on the critical value and chosen level.
Critical values can change with sample size because the probability of getting certain results by chance can depend on how much data you have.
In general:
small samples are more affected by random variation
larger samples often give more stable patterns
Because of this, statistical tables may include different critical values for different sample sizes. This helps keep the significance decision fair and mathematically appropriate for the amount of data collected.
Small sample sizes can increase the risk of a Type II error because real effects are harder to detect when there is less data.
With a larger sample:
estimates are usually more stable
random fluctuations have less impact
genuine effects are easier to identify
This means larger samples often improve the chance of detecting a true effect, reducing the likelihood of incorrectly deciding that a real result is non-significant.
A significant result only tells you that the finding is unlikely to be due to chance in that particular study. It does not guarantee the result will appear again.
Replication matters because it helps show that:
the finding is reliable
the result was not a one-off outcome
the effect can be observed under similar conditions
If repeated studies find similar significant results, confidence in the finding increases. If replications fail, the original result may have been misleading.
