
The cohort, at the overlook
This is the technical companion to the plain-language summary. It states each study’s method, what its comparison group actually received, and how much weight the finding can carry.
Every study below names its method, so the list runs strongest first. None of these words is doing anything clever — each one describes how the researchers found something to compare against.
MDRC’s lottery-based evaluation reported no significant effect on graduation, college readiness or enrolment. These are its seven estimates.
Every band crosses zero, so nothing is statistically significant. Every band also reaches past +10 points, an effect that would be educationally large. The study cannot distinguish those two cases. With 241 students in the comparison group, that is a question of power, not of evidence for absence.
The studies estimate different quantities against different comparison groups, so an average across them would not describe anything. Direction is carried by shape and words, not colour alone.
One note per study, stated once rather than repeated under every outcome it touches.
Matched comparison, not random assignment: unobserved differences between pathway students and their matches can remain. Findings are for pathways that met a certification standard, not all pathways. Two things must travel with the employment figure: employed pathway students were more likely to report jobs offering sick days, paid vacation and health insurance, and more likely to have had job-finding help from a professional they met in high school; and SRI frames one year out as too early to read as a labour-market outcome. Note also that the convergence with MDRC 2024 holds on a–g and enrolment but not on graduation, where SRI found a positive effect and MDRC found none.
The source discredits its own headline number on the following page. Because programme membership is defined from transcripts, “students who drop out may not be in school long enough to qualify for membership in either the academic or vocational programs. By default they drop into the general high school program category, with the result that dropout rates in this category are artificially inflated while dropout rates in the vocational and academic categories are artificially depressed.” The authors then build an alternative analysis to remove that artefact and get about a 12 percent reduction — not 8 to 10 times. They attribute most of the shrinkage to class rank, describe the effect as possibly indirect, and write that “the results are not conclusive”. Separately, the CDE page misstates the grade span: the source says the third and fourth years, not grades ten and eleven, and it names no “high-risk” subgroup.
Three defects, all documented in the report itself. (1) The 95% is a grade-12 rate: Appendix 2 defines it as graduates at year’s end divided by students enrolled in grade 12 at the start, so everyone who left before senior year is excluded from both the top and bottom of the fraction. (2) The 85% statewide comparator uses California’s superseded counting method — 404,987 graduates rather than the 386,222 the four-year adjusted cohort method gives — which the report explains it chose “to be consistent with earlier years.” It therefore cannot be set beside any CDE cohort graduation rate. (3) The report states its own limit: “This report describes CPAs and students’ performance, but cannot establish causal impact.” The underlying figures are unaudited academy self-reports.
The authors note earnings effects may partly reflect general skills: male students also had higher attendance and test scores, and industry fixed effects explain less than a third of the earnings gain. Cite 32%: the 2019 preprint said 31% and the published 2023 version says 32%. It is a percentage of quarterly earnings, not a dollar figure, and must not be converted into a Kemple-style annual sum. Applicant records matched to wage records at a rate of 72.3%, so roughly a quarter of applicants are unobserved in the earnings data. Two presentation traps. The “−8 percentage points ever enrolled” figure that circulates comes from the 2019 preprint; in the published paper it survives only as an in-text, explicitly unreported full-sample estimate, so it must not be cited as a table result. And do not write that CTE effects run through attainment — the paper does not establish mediation and argues the opposite emphasis.
District-level intent-to-treat on grant money, so it shows what happens when a district gets funded — not a per-completer premium, and it cannot be quoted as one. Its value is that it is California, it is causal, and its female-skewed effect matches the 2024 partnership-academy evaluation, which is two California studies pointing the same way against three out-of-state studies favouring young men.
The authors put the limit plainly: “these students were only 1 year out of high school, so although it is helpful to understand pathway graduates’ early workforce experiences, it is far too early to assess the overall labor market outcomes of participation in a Linked Learning pathway.” Point estimates are in logits, so they are directions and significance rather than percentages, and must not be reported as percentage points. The employment rate and the job-quality findings belong together: fewer pathway students were working, and those who were had better jobs.
Costs were not significantly different from other high schools in the community.
Postsecondary community-college certificates and degrees. This is not high school pathway completion and must never be presented as one. Included because it is the strongest California earnings evidence in the CTE family and because leaving it out while keeping weaker favourable numbers would be selective.
Community college CTE, not high school — its figures must never be read as a high-school completer premium. It is kept because it argues against the case: a CTE-versus-other gap that also appears among non-CTE students is not evidence CTE caused it, and that is the state’s own analyst saying so. It also notes the job-relatedness figure comes from “a survey to which only a small share of former CTE students respond”. Correction: we previously recorded this as showing CTE earnings gains slightly lower than non-CTE. That is not what it says. All three measures improved for CTE students; they improved at similar rates for everyone. The state’s own analyst also records the gap: these outcomes “are not always publicly available. In the case of other CTE objectives, such as career exploration, the state does not collect any specific outcome data.” The completer figures are survey-based.
The authors write associated with, not caused by, and the design is selection on observables — so these figures are descriptive and must be labelled as such wherever they appear. The version read is the November 2021 EdWorkingPaper; the published EEPA article may differ and was not opened. The sector reversals recorded elsewhere in this dataset come from the reported summary of this paper and the specific per-sector magnitudes have not been checked line by line against it.
Low certainty does not mean a finding is wrong. It means the evidence cannot yet carry the weight placed on it.
Ten numeric claims, zero citations. Two have now been traced to the CASN 2009–10 Profile, and how they are built is why they cannot be quoted as outcomes: the graduation figure counts only students who reached senior year, so anyone who left in grade 10 or 11 is in neither the top nor the bottom of the fraction, and the statewide figure it is set against is not computed that way. Both rest on numbers the academies reported about themselves. No method identity, so no cell.
The 95% graduation figure deserves its own sentence, because every defect in it is documented in the source the state declined to cite. It counts only students who reached grade twelve. It is set against a statewide number computed by a method California had already replaced, chosen “to be consistent with earlier years.” It rests on unaudited figures the academies reported about themselves. And the report carries its own warning: “This report describes CPAs and students’ performance, but cannot establish causal impact.”
The 2024 lottery does not contradict that 95%. It explains it. When students are assigned by chance instead of by application, the graduation gap does not shrink — it disappears, 93% against 93%. A difference that survives selection and vanishes under randomisation was selection all along.
Those are the most widely quoted statistics about partnership academies in California. Four were on this site until their sources were sought and none was found.
Named so the gaps are visible rather than silent.
Not yet read at source. Reported to raise on-time graduation at Massachusetts regional vocational technical high schools by roughly 7 to 10 percentage points, with larger effects for low-income students. A second separate-institution counterfactual alongside Connecticut.
No pre-registered protocol and no exhaustive search. The initial searches were seeded with studies expected to support our case, which biases what was found, and we state it rather than let the format imply otherwise.
Compiled by BusinessTech Academy, Santa Fe High School. Corrections are welcome and will be made in public.