The Academy cohort on the Getty overlook, the Los Angeles basin behind them

The cohort, at the overlook

BusinessTech Academy sealStructured evidence synthesis

Career academies: what the research shows

This is the technical companion to the plain-language summary. It states each study’s method, what its comparison group actually received, and how much weight the finding can carry.

Conflict of interest. This review was compiled by a school that operates a California Partnership Academy.
How to read this

The method decides how much a finding can carry

Every study below names its method, so the list runs strongest first. None of these words is doing anything clever — each one describes how the researchers found something to compare against.

Lottery, or randomised
More students applied than there were seats, so places were drawn at random. The two groups were alike to begin with, which is the whole reason a difference afterwards can be credited to the program rather than to who signed up. This is the strongest kind of evidence here.
Discontinuity
The program used a cutoff — a test score, a grant line. Students just above and just below that line are near-identical, so comparing them works almost like a lottery. Nearly as strong.
Matched comparison, or quasi-experimental
Researchers pair each student in the program with a similar student who is not, on the things they can see: prior grades, income, school. Useful, but only as good as the list of things they thought to match on.
Observational
Compares whoever enrolled with whoever did not, adjusting for what happens to be recorded. The weakest kind here, because students who choose a program differ from students who do not in ways nobody measured. That is what unobserved differences means wherever it appears below.
Significant
The result was big enough that plain luck is an unlikely explanation. Note the opposite carefully: “no effect detected” is not the same as “no effect”. A study with few students can miss a real difference, which is why the certainty section matters as much as the findings.
The 2024 evaluation

What a lottery of this size can resolve

MDRC’s lottery-based evaluation reported no significant effect on graduation, college readiness or enrolment. These are its seven estimates.

Finished high school
4 years
Both groups identical
Ready for a state university
4 years
Finished high school
5 years
Ready for a state university
5 years
Went to a four-year university
5 years
Went to community or trade school
5 years
Went anywhere after high school
5 years

Every band crosses zero, so nothing is statistically significant. Every band also reaches past +10 points, an effect that would be educationally large. The study cannot distinguish those two cases. With 241 students in the comparison group, that is a question of power, not of evidence for absence.

241 students in the comparison group; the report states oversubscription was limited and comparison groups were smaller than expected.
Power was computed for employment outcomes (detectable effect size 0.16–0.17), not for these interim outcomes.
COVID cancelled or postponed work-based learning for most of the treatment group, which the report says could dilute effects.
Survey response rates differed by 11 points (75% treatment against 64% comparison).
Earnings, the outcome the 2008 trial moved, is not reported until 2028 and 2032.
Findings by outcome

Grouped by what was measured, never pooled

The studies estimate different quantities against different comparison groups, so an average across them would not describe anything. Direction is carried by shape and words, not colour alone.

Academic achievement

Test scores, course grades.3 findings
no effect detected
David Stern 2012
regression, prior scores
Most academy coefficients not statistically significant; some significantly positive, some significantly negative.
increase
SRI Education 2020
quasi-experimental matched comparison
8.9 more credits accumulated overall; 15.5 more for students with low prior achievement.
no effect detected
Rasinski & Pedlow 1998
observational, student-characteristic controls
Vocational coursetaking showed only small effects on mathematics, science and reading achievement.

High school graduation

On-time or within the follow-up window.10 findings
no effect detected
Kemple with Willner 2008
randomised
More than 90% of both groups graduated or earned a GED.
no effect detected
MDRC 2024
randomised
92.9 against 92.9.
no effect detected
MDRC 2024
randomised
93.9 against 92.7.
descriptive only
Charles Dayton 2011
descriptive, self-reported program data
95% of CPA seniors graduated against 85% statewide — counted only among students still enrolled in senior year.
increase
Eric Brunner 2023
discontinuity
young men
Male students 10 percentage points more likely to graduate.
increase
SRI Education 2020
quasi-experimental matched comparison
3.1 percentage points more likely to graduate overall; 7.6 points for students with low prior achievement.
increase
SRI Education 2020
quasi-experimental matched comparison
2.1 percentage points less likely to drop out overall; 4.5 points for students with low prior achievement.
increase
Rasinski & Pedlow 1998
observational, student-characteristic controls
The widely quoted figure: students in vocational programmes 8 to 10 times less likely to drop out in their third and fourth years than in a general programme.
increase
Rasinski & Pedlow 1998
observational, student-characteristic controls
The authors’ own corrected estimate, after redesigning the analysis to remove the classification artefact: a dropout reduction of about 12 percent.
increase
Sade Bonilla 2020
discontinuity
“Dropout rates declined by 23 percent in treatment districts but were more pronounced for females than males.”

College readiness

Completion of California a–g or equivalent course requirements.7 findings
no effect detected
MDRC 2024
randomised
60.2 against 55.5.
no effect detected
MDRC 2024
randomised
58.9 against 54.5.
increase
MDRC 2024
randomised
young women
+12 percentage points.
increase
MDRC 2024
randomised
low-income and struggling
+13 percentage points.
descriptive only
Charles Dayton 2011
descriptive, self-reported program data
57% of CPA graduates met a–g against 36% statewide, as reported by the academies.
no effect detected
SRI Education 2020
quasi-experimental matched comparison
Pathway students completed 0.9 more a–g semester courses, but were equally likely to complete the full a–g suite (0.22, SE 0.20, not significant). Mathematics was the only significant subject-level gain. The a–g suite null held for every subgroup tested.
increase
Warner 2018
quasi-experimental matched comparison
Choosing a major: high school courses sparked the interest +0.51 (p<.01); a counsellor or adult encouraged it +0.56 (p<.05); worked in field-related settings +0.61 (p<.01).

Postsecondary enrollment

Any college, trade school, or four-year university.11 findings
no effect detected
Kemple with Willner 2008
randomised
About half of both groups completed a postsecondary credential.
no effect detected
MDRC 2024
randomised
Four-year university 32.7 against 28.0.
no effect detected
MDRC 2024
randomised
Community college or trade 36.2 against 36.9.
no effect detected
MDRC 2024
randomised
Any postsecondary 68.9 against 65.0.
descriptive only
Sarah Bohn 2018
descriptive
Close to 800,000 California high school students (45%) enrolled in a CTE course in 2016–17; business and finance was 6.4% of high school CTE enrollment.
no effect detected
Eric Brunner 2023
discontinuity
young men
Semesters enrolled in college −0.479 (SE 0.190, p<0.05) in the early window — a delay rather than a standing trade-off. The difference becomes statistically insignificant by age 23; students who enter the labour market first appear to return.
increase
Rosen and colleagues 2023
lottery
5 percentage points more likely to have completed an associate’s degree seven years in.
increase
Rosen and colleagues 2023
lottery
young men
13% of young men completed an associate’s degree against 3% in the comparison group.
no effect detected
SRI Education 2020
quasi-experimental matched comparison
“Certified pathway students were as likely as similar peers in traditional high schools to enroll in college.” Conditional on enrolling anywhere, equally likely to enrol at a four-year college and to persist to a second year.
descriptive only
California LAO 2025
descriptive administrative
18% of 2021–22 graduating high school students completed a CTE pathway; about 70% completed at least one CTE course.
no effect detected
California LAO 2025
descriptive administrative
CTE awards rose 50% since 2015–16, but non-CTE awards rose faster — “the increase in awards likely reflects factors beyond SWP”.

Earnings and employment

Wages, hours, employment stability after high school.14 findings
increase
Kemple with Willner 2008
randomised
+11% (+$2,088 a year), $16,704 over 8 years, 2006 dollars.
increase
Kemple with Willner 2008
randomised
young men
Young men +17% (+$3,731 a year), nearly $30,000 over 8 years.
increase
Eric Brunner 2023
discontinuity
young men
Male students 32% higher quarterly earnings.
no effect detected
Eric Brunner 2023
discontinuity
young women
“Little evidence that attending a technical high school affects the outcomes of female students”; appendix estimates for the female sample are small and statistically insignificant.
increase
Eric Brunner 2023
discontinuity
young men
Restricting to students aged 23 and older, the earnings difference is still 33% — so the gain is not merely early earnings from skipping college. It is 43% at 22 and younger, and the authors attribute about half that difference to time the comparison group spent in college.
decrease
SRI Education 2020
quasi-experimental matched comparison
One year out, pathway students were less likely to be employed — 34 percent against 43 percent — with no difference in compensation.
increase
Ann Huff Stevens 2019
quasi-experimental, individual fixed effects
“Average returns to CTE certificate and degrees that range from 14 to 45 percent”, largest in healthcare; non-health programmes 15 to 23 percent.
no effect detected
California LAO 2025
descriptive administrative
All three earnings measures have improved for former CTE students, and improved at similar rates for non-CTE students. The LAO’s own heading: “Impact of Program on Students’ Labor Market Outcomes Is Unclear.”
increase
Warner 2018
quasi-experimental matched comparison
Job quality. “Immediately after high school, the skills gained by Linked Learning students translated into jobs better than those of their peers.” Health insurance +0.53 (SE 0.22, p<.05) and paid vacation +0.51 (SE 0.26, p<.05) in logits; sick days +0.57 was not significant.
increase
Warner 2018
quasi-experimental matched comparison
Employed pathway students were more likely to have had help finding the job from a working professional they met in high school.
decrease
Warner 2018
quasi-experimental matched comparison
Pathway students were less likely to be employed one year out, 34% against 43%.
no effect detected
Warner 2018
quasi-experimental matched comparison
No difference in compensation, in the complexity of skills used on the job, or in autonomy.
increase
Walter Ecton 2023
observational
“CTE concentration is associated with a large increase in initial earnings ($1,792 in the first year after high school), that persist even 7 years after high school ($3,359 in annual earnings).”
descriptive only
Walter Ecton 2023
observational
“Substantial differences in outcomes for students in fields as diverse as healthcare, IT, and construction” — averages hide sector variation.

Student experience

Belonging, mentorship, perceived preparedness.1 finding
increase
MDRC 2024
randomised
More personalized attention, more collaboration, greater perceived preparedness; +29 points industry certification; +21 points work-based learning.
Read these with them

Study notes

One note per study, stated once rather than repeated under every outcome it touches.

SRI Education 2020
quasi-experimental matched comparison

Matched comparison, not random assignment: unobserved differences between pathway students and their matches can remain. Findings are for pathways that met a certification standard, not all pathways. Two things must travel with the employment figure: employed pathway students were more likely to report jobs offering sick days, paid vacation and health insurance, and more likely to have had job-finding help from a professional they met in high school; and SRI frames one year out as too early to read as a labour-market outcome. Note also that the convergence with MDRC 2024 holds on a–g and enrolment but not on graduation, where SRI found a positive effect and MDRC found none.

Rasinski & Pedlow 1998
observational with student-characteristic controls

The source discredits its own headline number on the following page. Because programme membership is defined from transcripts, “students who drop out may not be in school long enough to qualify for membership in either the academic or vocational programs. By default they drop into the general high school program category, with the result that dropout rates in this category are artificially inflated while dropout rates in the vocational and academic categories are artificially depressed.” The authors then build an alternative analysis to remove that artefact and get about a 12 percent reduction — not 8 to 10 times. They attribute most of the shrinkage to class rank, describe the effect as possibly indirect, and write that “the results are not conclusive”. Separately, the CDE page misstates the grade span: the source says the third and fourth years, not grades ten and eleven, and it names no “high-risk” subgroup.

Charles Dayton 2011
descriptive, self-reported program data

Three defects, all documented in the report itself. (1) The 95% is a grade-12 rate: Appendix 2 defines it as graduates at year’s end divided by students enrolled in grade 12 at the start, so everyone who left before senior year is excluded from both the top and bottom of the fraction. (2) The 85% statewide comparator uses California’s superseded counting method — 404,987 graduates rather than the 386,222 the four-year adjusted cohort method gives — which the report explains it chose “to be consistent with earlier years.” It therefore cannot be set beside any CDE cohort graduation rate. (3) The report states its own limit: “This report describes CPAs and students’ performance, but cannot establish causal impact.” The underlying figures are unaudited academy self-reports.

Eric Brunner 2023
discontinuity

The authors note earnings effects may partly reflect general skills: male students also had higher attendance and test scores, and industry fixed effects explain less than a third of the earnings gain. Cite 32%: the 2019 preprint said 31% and the published 2023 version says 32%. It is a percentage of quarterly earnings, not a dollar figure, and must not be converted into a Kemple-style annual sum. Applicant records matched to wage records at a rate of 72.3%, so roughly a quarter of applicants are unobserved in the earnings data. Two presentation traps. The “−8 percentage points ever enrolled” figure that circulates comes from the 2019 preprint; in the published paper it survives only as an in-text, explicitly unreported full-sample estimate, so it must not be cited as a table result. And do not write that CTE effects run through attainment — the paper does not establish mediation and argues the opposite emphasis.

Sade Bonilla 2020
discontinuity

District-level intent-to-treat on grant money, so it shows what happens when a district gets funded — not a per-completer premium, and it cannot be quoted as one. Its value is that it is California, it is causal, and its female-skewed effect matches the 2024 partnership-academy evaluation, which is two California studies pointing the same way against three out-of-state studies favouring young men.

Warner 2018
quasi-experimental matched comparison

The authors put the limit plainly: “these students were only 1 year out of high school, so although it is helpful to understand pathway graduates’ early workforce experiences, it is far too early to assess the overall labor market outcomes of participation in a Linked Learning pathway.” Point estimates are in logits, so they are directions and significance rather than percentages, and must not be reported as percentage points. The employment rate and the job-quality findings belong together: fewer pathway students were working, and those who were had better jobs.

Rosen and colleagues 2023
lottery

Costs were not significantly different from other high schools in the community.

Ann Huff Stevens 2019
quasi-experimental with individual fixed effects

Postsecondary community-college certificates and degrees. This is not high school pathway completion and must never be presented as one. Included because it is the strongest California earnings evidence in the CTE family and because leaving it out while keeping weaker favourable numbers would be selective.

California Legislative Analyst’s Office 2025
descriptive administrative

Community college CTE, not high school — its figures must never be read as a high-school completer premium. It is kept because it argues against the case: a CTE-versus-other gap that also appears among non-CTE students is not evidence CTE caused it, and that is the state’s own analyst saying so. It also notes the job-relatedness figure comes from “a survey to which only a small share of former CTE students respond”. Correction: we previously recorded this as showing CTE earnings gains slightly lower than non-CTE. That is not what it says. All three measures improved for CTE students; they improved at similar rates for everyone. The state’s own analyst also records the gap: these outcomes “are not always publicly available. In the case of other CTE objectives, such as career exploration, the state does not collect any specific outcome data.” The completer figures are survey-based.

Walter Ecton 2023
observational

The authors write associated with, not caused by, and the design is selection on observables — so these figures are descriptive and must be labelled as such wherever they appear. The version read is the November 2021 EdWorkingPaper; the published EEPA article may differ and was not opened. The sector reversals recorded elsewhere in this dataset come from the reported summary of this paper and the specific per-sector magnitudes have not been checked line by line against it.

Certainty by outcome

Every downgrade named

Low certainty does not mean a finding is wrong. It means the evidence cannot yet carry the weight placed on it.

Academic achievement
moderate certainty
No meaningful effect
  • Consistent across a value-added design with two years of prior scores.
  • Downgraded for indirectness: one state, one year.
High school graduation
moderate certainty
No effect
  • Two randomised trials agree, thirty years apart.
  • A ceiling makes this close to unmeasurable: both groups exceed 90%.
College readiness
low certainty
Unresolved, possibly positive
  • Downgraded for imprecision: 241 comparison students; every interval crosses zero and reaches past +10.
  • Downgraded for indirectness: 79% of the comparison group took a CTE course anyway.
  • Two subgroup effects are significant and large.
Postsecondary enrollment
low certainty
No effect detected
  • Same imprecision and counterfactual contamination as readiness.
  • The 2008 trial agrees at eight years.
Earnings and employment
moderate certainty
Large positive effect, concentrated in young men
  • Two independent designs agree: a randomised trial in the 1990s and a Connecticut discontinuity study published in 2023 using state wage records.
  • Different teams, different states, different decades, and the second was not run by MDRC.
  • Downgraded because both concentrate the effect in young men, and no study has shown it for young women.
  • California’s own earnings readouts land in 2028 and 2032.
Student experience
moderate certainty
Positive effect
  • Large, consistent differences across several survey measures.
  • Downgraded for risk of bias: self-report, with an 11-point gap in response rates.
Where the evidence fights itself

Four places the studies disagree

Both headline results come from the same organisation.
Kemple 2008 and the 2024 evaluation are both MDRC. Agreement between them is worth less than agreement between independent teams, because they can share a common error. Independent replication from other states is the missing piece.
Effects appear for young men in one state and young women in another.
The 2024 California study found a 12-point college-readiness gain for young women and nothing for young men; Kemple’s earnings gains were concentrated among young men. There is now evidence for a sector explanation rather than a gender one. In Massachusetts data, earnings advantages concentrate in construction, transportation, manufacturing and healthcare and are near zero in hospitality, agriculture and communications; on the college side healthcare and education concentrators gain while transportation and construction concentrators lose, almost entirely through four-year attendance. Health academies enrol mostly women and building trades mostly men, and CASN found 106 academies significantly more female against 71 more male. So the male-earnings, female-college split may be a composition effect of which industries a programme teaches. That is a hypothesis with support and not a finding: the authors discount their own magnitudes as covariate-sensitive, and nobody has estimated academy effects by sector directly.
The comparison group was already receiving much of the treatment.
In the 2024 study 79% of comparison students took a CTE course anyway and 36% had work-based learning. Studies of separate technical high schools have a far cleaner contrast. If effect size tracks how different the comparison actually was, these studies may be answering different questions rather than disagreeing.
Students select into academies, and not only downward.
CASN found academies enrol more low-income and Latino students and fewer special-education students — and that academy tenth-graders more often had higher ninth-grade scores than non-academy students at the same school. That positive selection is why simple comparisons overstate the effect, and it was documented by the academy movement’s own research centre.
Excluded

Ten numeric claims, zero citations

Career Technical Education facts for school leaders · cde.ca.gov

Ten numeric claims, zero citations. Two have now been traced to the CASN 2009–10 Profile, and how they are built is why they cannot be quoted as outcomes: the graduation figure counts only students who reached senior year, so anyone who left in grade 10 or 11 is in neither the top nor the bottom of the fraction, and the statewide figure it is set against is not computed that way. Both rest on numbers the academies reported about themselves. No method identity, so no cell.

The 95% graduation figure deserves its own sentence, because every defect in it is documented in the source the state declined to cite. It counts only students who reached grade twelve. It is set against a statewide number computed by a method California had already replaced, chosen “to be consistent with earlier years.” It rests on unaudited figures the academies reported about themselves. And the report carries its own warning: “This report describes CPAs and students’ performance, but cannot establish causal impact.”

The 2024 lottery does not contradict that 95%. It explains it. When students are assigned by chance instead of by application, the graduation gap does not shrink — it disappears, 93% against 93%. A difference that survives selection and vanishes under randomisation was selection all along.

Those are the most widely quoted statistics about partnership academies in California. Four were on this site until their sources were sought and none was found.

Not yet read

Named so the gaps are visible rather than silent.

The Effect of Career and Technical Education on Human Capital Accumulation: Causal Evidence from Massachusetts
Shaun Dougherty · Education Finance and Policy 13(2):119–148, 2018 · quasi-experimental admissions cutoff

Not yet read at source. Reported to raise on-time graduation at Massachusetts regional vocational technical high schools by roughly 7 to 10 percentage points, with larger effects for low-income students. A second separate-institution counterfactual alongside Connecticut.

Method

A structured synthesis, not a systematic review

No pre-registered protocol and no exhaustive search. The initial searches were seeded with studies expected to support our case, which biases what was found, and we state it rather than let the format imply otherwise.

Declare each study’s estimand before its estimate.
Record what its comparison group actually received.
Refuse to pool estimates of different quantities.
Rate certainty per outcome with named downgrades.
Exclude claims with no stated method rather than counting them as weak evidence.
Sources
MDRC 2024 evaluation (PDF)Stern, Dayton & Raby 2010Stern, Saroyan & Hester 2012PPIC: CTE in California, 2018 (PDF)All sources

Compiled by BusinessTech Academy, Santa Fe High School. Corrections are welcome and will be made in public.

When the 2028 and 2032 results arrive, they go on this page either way

Interest