The apprenticeship measurement framework

Section 1

Project overview

Google.org has spent a decade building apprenticeship programs, funding grantees, publishing best practices, and convening ecosystem actors. The GP2 North Star Goal commits to expanding that influence. The challenge is proving it.

North Star Goal (GP2)

Advance economic and workforce development through expanding technology-focused apprenticeships and access to job-ready skills and meaningful opportunities.

The goal has two components. The first is Google-run and Google-funded programs: measurable, documented, and already tracked. The second is ecosystem influence: the programs others start or scale because of Google.org's funding, content, research, convening activity, and field engagement. The second component has no validated measurement framework. This engagement produces one.

Why current measurement falls short

The four approved GP2 metrics (HM, AM1, AM2, AM3) are almost entirely program-facing. They measure completion rates and job placement for Google-run cohorts. The only ecosystem metric, AM3 (ten thought leadership engagements per year), is tagged "requires work to measure" in the goal tree and was itself flagged by Nina at Design Session 1 as a placeholder.

Three structural gaps drive the problem:

• Soft channels score zero. Content, field engagement, and convenings contribute nothing countable under a headcount-only model, even though they are central to the strategy.
• Attribution is unresolved. Google.org influences outcomes but rarely solely causes them. Other funders, governments, and employers are always in the room. Without a methodology, contribution claims are either inflated or invisible.
• Cross-pillar activity is uncounted. KSL and other-pillar grants that touch apprenticeship work are not included in GP2 reporting, so the goal understates the portfolio.

Engagement scope

Social Return Advisory was engaged from May to July 31, 2026 to deliver:

• Metric recommendations with attribution logic, covering all six channels of ecosystem influence
• 3-year target-setting report (2026 to 2028), sequenced correctly relative to the 2026 baseline census
• Measurement operational plan, including required changes to the Annual Cycle Reporting Form
• Executive summary and presentation for internal audiences

The scope confirmed at Design Session 1 (June 22, Seattle) expands the original brief: all four GP2 metrics are in scope for revision, not AM3 alone. The headline metric and AM1 through AM3 will each be revisited.

Design Session 1 takeaways (June 22, Seattle)

Session held in person with Nina Ong, Claire Conneely, and Chris Taylor. Key conclusions:

• Full metric scope confirmed. Nina confirmed that GP2-HM, AM1, AM2, and AM3 all need revision, not only AM3. The current headline metric (85% of apprentices securing a meaningful opportunity) reflects program outcomes, not ecosystem ambition.
• "Technology-focused" eligibility clarified. The phrase is intentionally broad: it includes construction and trades apprenticeships where AI and digital skills are embedded, not only tech-sector apprenticeships. Technology-focused is the cross-portfolio language across Google.org pillar goals.
• Scope is squarely apprenticeships. Google certificates, Coursera credentials, and short-term credentials are adjacent but out of scope for this framework, a later-stage policy question.
• Grantee intermediaries included. The JFF "Apprenticeships Unlocked" grant (convincing 100 companies to create new apprenticeships using Google tools) is a clear in-scope grantee case, likely reportable through the KSL survey channel and requiring explicit cross-pillar de-duplication.
• Index framing is understood, not yet settled. Nina acknowledged the ecosystem half currently scores zero. She noted the Index may be abstract for government affairs (GAP) audiences, who respond to stories and assets, not composite scores. A filterable dashboard concept was raised.
• Third-party validation should carry weight. Nina and Chris both noted that external validation (government agencies listing the toolkit, peer citations) should be weighted in the index, distinct from self-reported activity.

Session 2 target: week of July 13 to 15 (moved from July 6 to 10 to allow more material review time). Kate is deputy while Claire is on leave from July 9.

Open questions as of v1.0

Open items only. Resolved decisions from Session 1 are listed below for reference.

Resolved at Session 1:

Section 2

Current challenges

The measurement problem has three layers. Each one compounds the next.

Layer 1: The attribution problem

Google.org influences apprenticeship outcomes through six channels but rarely causes them unilaterally. Governments decide policy. Employers fund programs. Intermediaries recruit and train. When enrollment grows, any one of dozens of actors can claim partial credit. Without a validated methodology, Google.org faces two bad options: claim too much (inflated, indefensible) or claim nothing (invisible contribution, unfit for a North Star).

Layer 2: Soft channels score zero

If the unit of measurement is apprentices, then content, field engagement, and convenings produce nothing countable. A workforce agency that lists the Google.org toolkit as a resource does not immediately generate apprentices. A convening that produces employer commitments does not immediately generate apprentices. Under a headcount-only model, these channels are invisible in the annual report despite being central to the strategy.

This is not a data problem. It is a unit problem. The solution is a second measurement instrument whose unit is milestone events rather than apprentices, so that all six channels produce a non-zero annual figure.

Layer 3: The 3-year target cannot be set yet

The goal cycle is 2026 to 2028. A credible headcount target requires knowing the baseline: how many apprentices are currently in Google.org-influenced programs, with what attribution share, in which regional systems? That data comes from the 2026 census, which has not yet been completed.

Setting a headcount number before the baseline can create challenges later if the goal is not close to being reached. The framework avoids this by separating what can be set now (the influence index target, which rests on forecastable activity) from what must wait (the headcount target, which rests on census data).

Layer 4: Reporting infrastructure gaps

Section 3

Current North Star metrics

The approved GP2 metrics as of November to December 2025. All four are now in scope for revision per Design Session 1.

Current approved metric tree

Why all four metrics need revision

Nina confirmed at Design Session 1 that the current metric set is disproportionately aligned to program outcomes and does not reflect ecosystem ambition. In her framing: "our metrics are about program confidence, but it has nothing to do with the ecosystem impact we want to have."

The goal statement is explicitly ecosystem-oriented. The metric tree is not. This is the primary revision target.

Decision confirmed: full metric revision in scope

The engagement scope now includes proposed revised language for all four GP2 metrics. This does not mean all four will change: GP2-HM and GP2-AM1 may survive in modified form as the program quality spine, with ecosystem metrics added alongside. That is a decision for Nina and Research and Evaluation to take at Design Session 2.

Section 4

Six pathways to ecosystem change

Google.org influences apprenticeship ecosystems through six distinct channels. Each produces verifiable milestone events as influence travels from activity to uptake to adoption to scale. The four-stage pipeline is common to all six, the evidence standards differ by channel.

The four-stage pipeline

Apprentices enter the count only at adoption, when a named program reports enrollment. Left of that gate, activity is tracked as Index points and conditions movement, never as headcount.

Channel 1

Grantmaking

The most direct pathway. Google.org makes a grant to an apprenticeship intermediary or program operator, who launches or expands a program. Enrollment in that program is countable with a defined attribution share and funging adjustment.

Example: JFF National Apprentice Fund. Google.org funds JFF to administer financial support to apprentices facing hardship barriers to completion. The program design, enrollment data, and outcomes are tracked through JFF evaluation. Attribution share: high, given Google.org is the primary funder and the program was designed to Google.org specifications.

Example: JFF "Apprenticeships Unlocked." New 2026 grant. JFF will convince 100 companies to create new apprenticeships using Google best practices and AI tools. Cross-pillar: likely flows through KSL survey. De-duplication with KSL required.

Channel 2

Incubating and piloting

Google.org designs or seeds a model, which a partner then pilots and potentially replicates. The seeded model may operate independently of continued Google.org funding. Attribution is stronger here than for pure thought leadership, because Google.org designed the model, but weaker than direct grantmaking because the partner operates it.

Example: FAME USA / Manufacturing Institute. Google.org has supported FAME USA expansion into AI-skills-embedded manufacturing apprenticeships. If the FAME model spreads to sites Google.org did not fund, those sites earn stage credit through process tracing.

Example: AI Opportunity Fund. Google.org-seeded programs that blend AI credentials with apprenticeship pathways. Adoption by external program operators earns adoption-stage credit, spread to peer institutions earns scale-stage credit.

Channel 3

Funded research

Google.org commissions or funds research on apprenticeship models, barriers, or outcomes. The research influences practice and policy when cited, adopted into program design, or referenced in government guidance.

Illustration (from SRA Reference Exhibit A): A Google.org-funded study on apprenticeship completion barriers is published and downloaded widely. A national skills agency references it in a public consultation (uptake). The agency revises its funding rules to include the supportive-services model the study recommended (adoption). Programs under the new rules enroll 4,000 apprentices. Attribution share applied: 25% (agency made and funded the decision). Funging adjustment: 70% (share that would have happened anyway). Net attributed: approximately 700, medium confidence.

The JFF National Apprentice Fund outcomes study is the primary near-term research asset. Its design and interim findings are a critical input to the grantmaking headcount calculation.

Channel 4

Open-source content

Google.org creates and publishes free-access toolkits, curricula, and best practices that other organizations can embed in their own programs. The AI Essentials curricula are the primary example currently. The FAME AI transformation toolkit is another.

Example raised by Nina in session: When the toolkit was released, the UK government listed it as a recommended resource for AI transformation. That government listing is an uptake event, documentable by date and citation. If a program formally embeds the curriculum, that is adoption. If the curriculum becomes a sector norm, that is scale.

What counts is external uptake, not download volume alone. A download threshold signals uptake, a named program embedding the content is adoption.

Channel 5

Field engagement

Direct engagement with governments, employers, and workforce organizations: advisory sessions, testimony, advisory board participation, peer-learning exchanges. The influence travels through the recommendation: if an engaged actor cites the recommendation, adopts a practice change, or brings it to peer institutions, each stage is countable.

Example: Google.org engages a state workforce board on apprenticeship expansion. The board cites the engagement in a funding application (uptake). The board adopts a new employer engagement model recommended in the session (adoption). Three peer boards adopt the same model at a national conference (scale, via process tracing).

Key distinction noted in session: GAP (government affairs) audiences value stories and assets, not index scores. Field engagement events should be documented with narrative evidence and linked to concrete outputs (published guidance, employer announcements) that GAP can bring to government counterparts.

Channel 6

Convenings

Hosted gatherings of ecosystem actors: employers, government agencies, intermediaries, practitioners. Convenings produce commitments, working groups, partnerships, and occasionally coalitions that sustain beyond Google.org's involvement.

Example: JFF Horizons convening, Washington DC, July 2026. Google.org is a convening partner. Commitments made by employer attendees are uptake events. A partnership formed between two convening participants that delivers a new program is an adoption event. A standing coalition that operates independently after the convening is a scale event.

Third-party validation raised by Chris Taylor in session: when an external actor independently credits Google.org's convening role, that validation is stronger evidence than self-reporting and should carry higher confidence in the Index.

The count gate

Section 5

Proposed approach to measure and track ecosystem impact

The framework rests on two parallel instruments that answer different questions. The decision of which one leads is the central open question for Design Session 2.

The three-panel model

Reporting rests on three panels of equal standing, not a single headline number. This lets a slow-channel year read as progress rather than failure.

The Apprenticeship Ecosystem Influence Index (AEII)

The AEII is the instrument that makes Panels 2 and 3 quantified and trackable. It counts dated, verifiable milestone events across all six channels, weights each by stage and confidence, and sums to one annual number.

Event score = stage weight × confidence multiplier

Four controls guard against gaming: steep weighting toward adoption and scale so cheap uptake cannot dominate, a required evidence artifact per event, a sampled annual audit, and the standing rule that the Index is always reported next to the Panel 1 headcount.

Illustrative scoring: soft channels in one year

The three channels a headcount-only model scores at zero contribute approximately 42 auditable points in a single year under draft weights.

Vintage accounting

Every activity is tagged by the year it began and tracked until it resolves. A 2026 convening that becomes a 2031 program is credited to the 2026 vintage when those apprentices land, and contributes leading Index points in between. Activity that stalls past its expected lag is moved to dormant so the pipeline does not inflate with influence that never matured.

The three-year target commits to banked headcount plus pipeline advancement, the two things that can move in three years. The large eventual yield lives on the vintage maturation curve as a forecast, reported with a widening confidence band.

Attribution methodology by channel

Regional variance architecture

Three distinct regional contexts require adaptation of the common core model.

Section 6

Potential North Star metric revisions

Design Session 1 confirmed that all four GP2 metrics are in scope for revision. What follows presents the proposed approach and, where the approach is not yet settled, two or three candidate options for Nina and Research and Evaluation to decide at Design Session 2.

Headline metric: three candidate configurations

Alternative metrics: potential revisions

These apply regardless of the headline configuration chosen.

GP2-AM1 · Revised

From: completion rate  ·  To: completion rate plus ecosystem quality contribution

GP2-AM1 currently tracks program completion for Google-run cohorts. The revised version retains that but adds a second track for ecosystem quality contribution: the marginal improvement in completion rate attributable to Google.org-recommended practices in non-Google programs.

Rationale from session: Spencer raised the marginal quality model (Swiss example) where a Google.org-recommended change, such as pairing a mentor with each apprentice, moves completion rates from 80% to 85%. That 5-point lift is attributable and defensible without claiming to have created the program.

GP2-AM2 · Revised

From: apprentice confidence  ·  To: meaningful opportunity rate, with ecosystem track

Nina noted that the 90% confidence metric "is typically used as a program health metric" and was flagged as a potential swap-out if an ecosystem metric could be added. Proposed revision retains confidence as a program health indicator (internal use) and promotes meaningful opportunity rate to the alternative metric slot with an ecosystem extension.

GP2-AM3 · Replacement

From: thought leadership engagement count  ·  To: AEII annual score

AM3 replaces the provisional 10 engagements/year count with the AEII annual score, which subsumes field engagement (stage activity) as one of six counted channels. Two transition options:

GP2-HM · Revised headline (if Option A or B chosen)

If the AEII is selected as the headline (Option A above), the program-facing outcome moves to AM2-A. The revised headline reads:

If the headcount is selected as the headline (Option B above), the proposed headline reads:

What must be decided at Design Session 2

• Which headline configuration (A, B, or C) leads the GP2 metric tree
• Whether GP2-AM1 carries an ecosystem quality track or remains program-only
• Whether GP2-AM2 carries the meaningful opportunity rate or reverts to confidence as the alternative metric
• Replace or transition GP2-AM3 (and if transition, mint as AM4 or record as named replacement)
• Confirm stage weights (0/2/5/10) and confidence multipliers (1.0/0.7/0.4) or adjust
• Who owns weight revision governance as evidence arrives

Section 7

Critical path and next steps

Highest-priority outstanding inputs

These four inputs are blocking build decisions. They are listed in order of leverage.

Design Session 2: week of July 13 to 15

Session 2 is the decision session. The build proceeds after Session 1 on best available information, Session 2 either confirms decisions or adjusts the architecture. Kate is the deputy for Claire, who is on leave from July 9.

Session 2 agenda (draft):

• Review of AEII draft event log and channel matrix with real Google.org activity
• Headline metric configuration decision (Options A, B, or C)
• Revised metric language review for all four GP2 metrics
• Reporting-form field additions (with Kate if Claire is unavailable)
• Target calibration: Index multiple and coverage threshold
• Cross-pillar de-duplication governance

Engagement timeline

Social Return Advisory  ·  Google.org Apprenticeship NSG  ·  v1.0  ·  June 2026