The calculation of the NAP (short for “Non-Overlap of All Pairs”) is a straightforward method for evaluating the effectiveness of interventions in school and educational settings. In this article, we will explain what the NAP is, which data you need, how the calculation works, and how to interpret the results. We will also show you digital options for performing the calculation.
1. What is the NAP?
The NAP is a statistical measure that assesses the effectiveness of interventions based on the overlap of data points from different phases. It is commonly used in single-case diagnostics, for example in A-B-A or A-B designs. These are studies in which data is collected for a period of time without any educational intervention (Baseline), and then compared with a phase in which the intervention is implemented. In both phases, the same behavior (e.g., “disrupting the class”) is observed and later compared. The NAP indicates how often data points from the intervention phase are higher or lower than those from the baseline phase. The relative overlap is then expressed as a percentage:
- Weak effects: 0 to 0.65
- Medium effects: 0.66 to 0.92
- Large or strong effects: 0.93 to 1.0
But be careful! The interpretation of the NAP value depends on the target direction of the observed behavior. If the goal is to reduce a certain behavior (e.g., unwanted behavior), then lower values in the intervention phase should be considered progress. Conversely, if a behavior is supposed to increase (e.g., desired behavior), then higher values in the intervention phase should be considered progress. A misalignment here can cause misunderstandings when evaluating the intervention.
2. Which data do I need?
To calculate the NAP, you need measurement data from at least two phases:
- Baseline phase: Data points that represent the behavior before the intervention.
- Intervention phase: Data points that capture the behavior during the intervention.
The data should be collected as precisely and regularly as possible to ensure a valid analysis. Ideally, each phase includes at least 5 to 12 measurement points — although this can vary depending on the behavior in question. If you are looking at learning-related behavior, fewer measurement points are often sufficient for valid results. In the domain of emotional/social behavior, more measurement points are typically required.
3. How do I calculate the NAP?
The NAP value is calculated by comparing all possible pairs of data points from two conditions. Expressed as a formula:
$$ \text{NAP} = \frac{\text{Number of non-overlapping pairs} + 0.5 \times \text{Number of equal-value pairs}}{\text{Total number of pairs}} $$
1. Form data pairs
Take the data points from both conditions:
- Baseline (A): e.g. [2, 3, 5]
- Intervention (B): e.g. [4, 6]
Create all possible pairs by combining each value from Baseline (A) with each value from the Intervention (B).
_____________________________
2. Compare the pairs
Compare each pair (consisting of a value from A and a value from B) using the following rules:
- Non-overlapping: The value from A is smaller than the value from B (e.g., 2 < 4).
- Overlapping: The value from A is greater than the value from B (e.g., 5 > 4).
- Equal-value: The value from A is equal to the value from B (e.g., 3 = 3).
_____________________________
3. Calculate the values
Now calculate the following values:
- Non-overlapping pairs: The number of pairs in which the A-value is smaller than the B-value.
- Equal-value pairs: The number of pairs in which the A-value equals the B-value.
- Total number of pairs: The total number of all possible pairs, calculated as the number of values in A multiplied by the number of values in B.
Baseline (A): [2, 3, 5]
Intervention (B): [4, 6]
- Form pairs:
All possible pairs: (2, 4), (2, 6), (3, 4), (3, 6), (5, 4), (5, 6)- Compare pairs:
- (2, 4): 2 < 4 → Non-overlapping
- (2, 6): 2 < 6 → Non-overlapping
- (3, 4): 3 < 4 → Non-overlapping
- (3, 6): 3 < 6 → Non-overlapping
- (5, 4): 5 > 4 → Overlapping
- (5, 6): 5 < 6 → Non-overlapping
So we arrive at the following result:
- Number of non-overlapping pairs: 5
- Number of overlapping pairs: 1
- Number of equal-value pairs: 0
Plug into the formula:
- Total number of pairs: $$ N_{total} = 3 \times 2 = 6 $$
$$ \text{NAP} = \frac{5 + 0.5 \times 0}{6} = \frac{5}{6} \approx 0.83 $$
4. Interpretation of the NAP
When interpreting the NAP value, it is crucial to clearly define the type of behavior being measured, since different target behaviors (e.g., disruptive behavior vs. learning-enhancing behavior) lead to different expectations:
Decreasing behavior:
If the goal is to reduce a behavior such as disruptive behavior in class, the NAP value should show that the values in the intervention phase are lower than in the baseline phase. In this case, a pair is considered non-overlapping if the intervention-phase value is smaller than the baseline value.
$$(b_j < a_i)$$
Increasing behavior:
Conversely, if the goal is to foster a behavior such as learning-enhancing behavior, the values in the intervention phase should be higher than in the baseline phase. Here, a pair is considered non-overlapping if the intervention-phase value is greater than the baseline value.
$$(b_j > a_i)$$
Adjustment for overlap:
It is crucial to adapt the logic behind the definition of “overlap” according to the objective. A misdefinition of overlap may lead to an incorrectly interpreted NAP value, making it impossible to draw the correct conclusions about the effectiveness of the intervention.
4. Digital Options for Calculation
Conveniently, the calculation can also be done via the internet platform singlecaseresearch.org or via apps such as PlusDaily. In the PlusDaily app, you can calculate the NAP directly and visualize the data. The app synchronizes data via Firebase and allows for straightforward interpretation. There are also specific R packages that can perform the calculations for you.