Sample designed to support the two goals while considering other factors
Our sample was designed to support these two main goals while considering several other factors.
NCIER®
Report page 152
Claims on this page
First consideration: enough sample given uncertainty about how many schools had BTAM programs as of
The first consideration was researchers needed to ensure sufficient sample to unpack differences in schools’ BTAM programs with some uncertainty about how many schools nationally had BTAM programs as of January 2025.
Using April 2024 School Pulse Panel data, estimated about eight in ten sampled schools would have a
Using data collected from the Department of Education’s School Pulse Panel in April 2024, we estimated that about eight in ten sampled schools would have a BTAM team (NCES, 2025).
Second consideration: sample size to estimate means and proportions across three state policy contex
A second consideration was ensuring sufficient sample size to estimate means and proportions across the three state policy contexts of interest without substantially increasing the survey design effect.
Final consideration: availability of panelists across state policy contexts; about 6,000 active pane
A final consideration was the availability of panelists across state policy contexts within the ASLP to be sampled. There were about 6,000 active panelists in the ASLP at the time of sampling.
Target of 1,650 completed surveys across three state policy contexts
The final sample was chosen to target 1,650 completed surveys distributed across the three state policy contexts.
Initial goal: 50 percent early legislation, 25 percent other states, 25 percent encouraged-only
Sample (no n: a percentage change, not a share of a sample) · a percentage that is not a share of a sample (e.g. a percentage-point change or a weighted estimate of the population)
The initial goal was to sample 50 percent from the early legislation states (as it was anticipated that BTAM programs would be most common in these states), 25 percent from each of the other states, and 25 percent from the encouraged BTAM use only states.
At a 30 percent response rate, sample 1,375 in other and encouraged states and 2,750 in early legisl
Sample (no n: a percentage change, not a share of a sample) · a percentage that is not a share of a sample (e.g. a percentage-point change or a weighted estimate of the population)
As shown in Table B.2, assuming a 30 percent response rate (which is in line with historical ASLP surveys), this would require sampling 1,375 principals in both the other states and encouraged BTAM use only states and 2,750 principals in early legislation states.
Too few panelists in other states; extra principals sampled from encouraged-only states
Sample (no n: a percentage change, not a share of a sample) · a percentage that is not a share of a sample (e.g. a percentage-point change or a weighted estimate of the population)
There were not enough panelists in the other states in the ASLP at the time of sampling to meet the desired sample. Therefore, we sampled some additional principals from the encouraged BTAM use only states to maintain the 50 percent/50 percent distribution with principals from early legislation states.
5,500 ASLP members invited using probability-based sampling
RAND researchers used probability-based sampling to invite 5,500 ASLP members to complete the survey using the ASLP as the sampling frame.
Of 5,500 sampled, 1,665 fully completed and 181 partially completed
Of the 5,500 members sampled, 1,665 principals fully completed the survey, and 181 principals partially completed the survey.
38 sampled principals were ineligible
Meanwhile, 38 sampled principals were determined ineligible for the survey.
1,746 cases received a weight, a 32.0 percent completion rate (AAPOR RR6)
Sample (no n: a percentage change, not a share of a sample) · a percentage that is not a share of a sample (e.g. a percentage-point change or a weighted estimate of the population)
Excluding these 38 principals and the cases in which respondents did not complete a significant portion of the survey, a total of 1,746 cases received a weight. This corresponds to a 32.0 percent completion rate using the American Association for Public Opinion Research’s Response Rate 6 definition (American Association for Public Opinion Research, 2023).
Anticipated about 80 percent of 1,650 completes (n = 1,320) on full survey, 20 percent (n = 330) on
Sample (n = 1320 of 1650) · Targeted number of completed surveys (design target)
We anticipated that about 80 percent of the 1,650 completes would receive the full survey and be asked detailed questions about their BTAM program (n = 1,320) and 20 percent would receive the shorter survey (n = 330).
Underestimated principals reporting another school-based team that is not called a BTAM team
However, we underestimated the number of principals who would indicate that their school had another “school-based team that makes decisions regarding serious student behavioral concerns that is not referred to as a behavioral assessment and management (BTAM) team” and receive the longer survey pathway.
About 250 schools reported a school-based team not referred to as a BTAM team
About 250 schools told us that they have a school-based team that makes decisions regarding serious student behavioral concerns that is not referred to as a BTAM team.
Virtually all schools got the longer survey; only about 50 got the shorter pathway
This means that virtually all schools received the longer survey, and only about 50 schools received the shorter pathway intended to accommodate schools that were not doing BTAM.
Other states: desired completes 412, desired sample 1,375, panelists 1,307, final sample 1,307, fina
Sample (n = 423 of 1307) · Table B.2 final sample, other states
Other states | 412 | 1,375 | 1,307 | 1,307 | 423 | 32.4
Encouraged BTAM use only: desired completes 412, desired sample 1,375, panelists 1,794, final sample
Sample (n = 481 of 1443) · Table B.2 final sample, encouraged BTAM use only states
Encouraged BTAM use only | 412 | 1,375 | 1,794 | 1,443 | 481 | 33.3
Early legislation states: desired completes 826, desired sample 2,750, panelists 3,219, final sample
Sample (n = 842 of 2750) · Table B.2 final sample, early legislation states
Early legislation states | 826 | 2,750 | 3,219 | 2,750 | 842 | 30.6
All: desired completes 1,650, desired sample 5,500, panelists 6,320, final sample 5,500, final compl
Sample (n = 1746 of 5500) · Table B.2 final sample, all states
All | 1,650 | 5,500 | 6,320 | 5,500 | 1,746 | 31.7
Top
Find the Perfect Training Class For You