Module 5 Overview

Theme

Evaluation, uncertainty, and error analysis

Essential Question

How do we know whether an AI system is useful, robust, and honest?

Module Components

• Book prose: conceptual framing, domain scenario, methods, and failure modes

• Assignment: evidence-backed production of a specific artifact

• Slides: presentation sequence for seminar or lecture delivery

• Narration: spoken version of the slide flow

• Instructor notes: facilitation plan, discussion prompts, and grading cues

• Rubric: criteria for evaluating the module artifact

• Notebook: executable lab aligned with the module theme using synthetic system evidence including task features, model outputs, confidence signals, and review outcomes

Module Artifact

AI system review package with architecture, evidence, limitations, and deployment recommendation focused on evaluation, uncertainty, and error analysis: Build an evaluation table with accuracy, error slices, and uncertainty notes.

Professional Setting

Students work as if advising an AI review team evaluating a proposed applied AI system before pilot deployment. Their work must be intelligible to technical lead, domain owner, governance reviewer, and end-user representative.