Context & Challenge

Innovate UK's BridgeAI programme set out to accelerate AI adoption across sectors where uptake has been slow. In the construction industry, that challenge is acute: leaders often lack the conceptual grounding to evaluate AI solutions, commission data strategies, or manage AI-driven change. The client needed more than a training course — they needed evidence that the programme would produce genuine, transferable capability, not just completion statistics.

The measurement risk here was real. Without a clear construct definition of "AI literacy for leaders," any assessment would be measuring the wrong thing — course familiarity rather than applied competence. Weak measurement design would also undermine the programme's credibility with Innovate UK and the BridgeAI Skills Working Group, who required alignment with an established competency framework and defensible evidence of impact.

Approach

• Defined AI literacy as a multi-dimensional construct for the specific population (senior leaders in construction), distinguishing between conceptual understanding, strategic application, and organisational leadership — informed by the latest systematic review in the field (Pinski & Benlian, 2023)

• Designed a layered assessment architecture combining knowledge checks (multiple-choice items at module level), practical application scoring (applied "pet projects" scored on a 0–100 rubric), and self-reported learning gain surveys based on validated feedback methodology (Kember & Ginns, 2012)

• Mapped course content to the AI Skills for Business Competency Framework, ensuring each of seven defined learning outcomes could be traced to a specific assessment component and competency level

• Developed a participant feedback instrument with items targeting perceived gains in ethical reasoning, ROI communication, and AI application — dimensions that written tests alone would miss

• Advised on construct validity throughout curriculum design, flagging where proposed content addressed surface familiarity rather than the deeper strategic reasoning the programme intended to build

• Produced documentation for stakeholder review by the Innovate UK Skills Working Group, including a clear rationale for assessment choices and criteria for programme success

Outcomes

• A coherent measurement framework that connected each learning outcome to a concrete assessment method — making programme evaluation defensible to a government funder

• Course content structured around a validated construct model, reducing the risk that the programme would train on AI vocabulary without building transferable decision-making capability

• A practical scoring rubric for applied projects that allowed Social Machines facilitators to assess learning consistently across participants from different organisation sizes and roles

• Clearer accountability for impact: completion data, knowledge gain, and practical application were each tracked separately, giving the client distinct levers to diagnose and improve the programme

• A programme design credibly aligned with a national competency framework, positioning it for integration with broader UK AI skills infrastructure

If your organisation is building or procuring AI tools that need to assess human skills, and you need the measurement to hold up under scrutiny, get in touch.