Generative UX research to inform checklist design and functionality

Role: Lead Researcher

Timeframe: 4 weeks

Context

As part of Amazon's transition from its legacy learning management system (Knet), the company was building Amazon Learn, a new enterprise-scale B2B training platform designed to serve internal employees globally. This platform intended to centralize all learning, compliance, skills validation, and on-the-job training activities across Amazon’s diverse businesses from logistics to data centers to satellite manufacturing.

Amazon Learn is not a typical LMS; it is an enterprise-grade, custom-built system designed for scale, flexibility, and global compliance. The product was expected to support hundreds of thousands of employees across business units, handling localized content, regulatory training, and real-time skill validation in high-stakes environments.

One of the key features being developed was the observation checklist, used by administrators, validators (observers), and driver trainers to assess employees’ competencies in hands-on training environments. These checklists are critical for validating job readiness, compliance, and training completion.

This generative UX research aimed to uncover pain points, workflow challenges, and unmet user needs in the current checklist experience within Knet, and inform product specifications for a simplified and scalable checklist experience in Amazon Learn.to

Research Questions

The research revealed significant pain points across the end-to-end checklist lifecycle:

  • Complex creation process involving multiple disjointed steps and interfaces.

  • Manual entry of checklist items from external documents (e.g., Excel), with no bulk upload capability.

  • Lack of editing flexibility, requiring complete recreation of checklists to make changes.

  • Cumbersome validation workflows, especially when validators need to mark multiple learners or items.

  • Limited search/filtering, particularly around location-based access.

  • Mixed-device environments, where mobile, tablet, and paper each have trade-offs depending on the use case.

  • Low adoption due to inefficiencies, especially in high-volume training environments.

  • Translation workflows are external and poorly integrated, creating friction in maintaining localized content.

Approach

To explore these issues, I conducted qualitative research using 1:1 semi-structured interviews with nine stakeholders:

  • 4 Administrators (checklist creators)

  • 2 Validators (observers/verifiers)

  • 3 Driver Trainers (instructors and evaluators)

These participants represented key roles in the checklist lifecycle and came from various businesses within Amazon. Interviews were analyzed using thematic analysis to identify patterns and pain points.

Insights

Current Setup was Fragmented and involved three disconnected steps, causing confusion and errors. Admins want a single flow to create checklists, assign competencies, and define permissions.

Data Entry and Editing Are Painful

  • No bulk upload; all checklist items must be entered manually.

  • Editing checklists post-publishing is nearly impossible, creating version control issues.

  • Reordering competencies is unintuitive (no drag-and-drop).

Validation Is Inefficient and Manual

  • Validators must open each learner’s profile individually.

  • Bulk validation is technically possible but non-intuitive and underutilized.

  • Validation becomes a bottleneck in high-volume contexts like AMZL.

Search and Filtering Are Inadequate

  • Validators are overwhelmed by global learner lists.

  • No way to filter by location or team, hindering usability.

Hybrid Device Needs: Print + Tablet

  • Driver Trainers prefer tablet-based digital checklists but also need printable versions due to environmental constraints in Data Centers.

  • Current workaround involves transferring PDFs manually—time-consuming and error-prone.

Incomplete Support for Translations and Retakes

  • Translations require offline workflows with no system support for uploading localized content.

  • Retakes and audit trails are poorly tracked.

  • Learner feedback and validation loop is not well-connected.

Impact

This research directly influenced the following product and design decisions for Amazon Learn:

  • Unified checklist builder: A single, intuitive interface to create checklists, add competencies, and assign permissions in one place.

  • Bulk import/export: Support for uploading competencies using standardized templates (e.g., CSV).

  • Drag-and-drop UI for reordering checklist items.

  • Bulk validation workflows for validators to validate multiple learners or competencies in a single interaction.

  • Location-based filters to improve search and assignment accuracy.

  • Digital + printable checklist support, giving flexibility depending on work environment.

  • Improved reporting for audit trails, retake tracking, and validation performance.

  • Translation management integration, including side-by-side version viewing and bulk upload of localized content.

Learnings and Takeaways

  • Design for real-world workflows: Validators and trainers often operate in high-pressure environments where speed and accuracy are paramount. UX should support hybrid offline/online models.

  • One-size-doesn’t-fit-all: Different businesses have vastly different validation needs (e.g., AMZL vs. Kuiper vs. Data Centers), requiring a modular, configurable checklist experience.

  • Simplicity increases adoption: Complexity in checklist setup and validation directly correlates with lower usage and delayed certification.

  • Support the ecosystem: Features like translation, printable versions, and retake tracking are often considered "edge cases" but are essential for operational excellence.

  • Device context matters: Avoid assuming digital is always better; support safer, familiar workflows like printed checklists when appropriate.