Voice-First Safety Reporting

The Problem

Why Traditional Safety Reporting Fails

Aviation safety management requires reporting incidents, hazards, and safety observations for trend analysis, regulatory compliance, and safety improvement. But safety reporting has two conflicting requirements: immediate capture (reports filed quickly while details are fresh) and structured risk assessment (safety management systems need risk data for prioritization).

Traditional forms try to satisfy both simultaneously with dropdown menus, severity ratings, narrative boxes, and risk assessment matrices. This fails when it matters most.

Forms demand structured thinking and keyboard access when reporters are in reactive mode, trying to capture observations quickly in crew rooms, hangars, on ramps. Risk assessment requires expertise reporters don't have. The difference between "Major Accident" and "Catastrophic Accident" potential requires understanding safety management concepts. Forms present these as dropdown options with no guidance.

Worst of all, forms force premature classification. Dropdown menus ask "What was the severity?" before the reporter has articulated what happened. Reporters think "we had hydraulic fluid near the APU, found it during inspection, maintenance cleaned it up," not "this was a Level 2 hazard with Limited barrier effectiveness."

The cognitive load is backwards. Forms prioritize database structure over reporter workflow.

The Solution

We rebuilt safety reporting as a voice conversation that captures incident narratives naturally, then automatically performs structured risk assessment using aviation safety methodology.

TrustFlight's Quality SMS product suite already had a mature iPad-based form system with an established data model for safety reporting. Rather than replace it entirely, we asked: what if reporters could speak their reports instead of typing them?

Voice-First, Entirely Conversational

Reporters speak their account naturally, as if describing what happened to a colleague. No keyboard, no dropdown menus, no navigating between form fields.

Complete voice reporting workflow from initial incident description through automated risk assessment and confirmation

A maintenance technician might say: "During preflight inspection on aircraft G-ABCD, I noticed hydraulic fluid residue near the auxiliary power unit. Small amount, maybe a few drops. Equipment is still operational. Maintenance has been notified."

As they speak, the system is extracting structured data in real-time—incident type (maintenance observation), aircraft registration (G-ABCD), equipment involved (APU), operational status (still functional), actions taken (maintenance notified). All the fields that would normally require separate form inputs are being populated automatically from the natural description.

Then the system asks contextual follow-ups via voice, only for details that weren't in the initial account: "What time was this?" "Did you take any photos?" The questions adapt to what's already been captured, so reporters aren't repeating themselves or answering irrelevant questions.

Automated Risk Assessment

Traditional systems force reporters to classify risk manually: picking severity levels and barrier effectiveness ratings from dropdown menus before they've even finished describing what happened. We inverted this. The system listens to the narrative first, extracts risk factors as the reporter speaks, then applies formal safety management methodology automatically.

Real-time risk profile generation and automated safety assessment

A pilot reports: "At cruise altitude, we got a collision warning directing us to descend immediately. We followed it, descended 500 feet, and the other aircraft passed 300 feet above us. Air traffic control hadn't warned us about the traffic."

As the reporter speaks, the system builds a risk profile. It recognizes "collision warning" as a near-miss event, "300 feet above" as dangerously close proximity, "followed immediately" as effective crew response, and "ATC hadn't warned us" as a procedural failure. The system infers what could have happened: mid-air collision (Major Accident), and what prevented it: automated warning system and crew training (Effective barriers). It calculates Risk Score 80: High Risk requiring management review.

Then it confirms via voice: "I've classified this as high risk—a collision avoidance event with effective barriers that prevented escalation. Risk score 80. Does that sound right?" If the assessment feels wrong, reporters can adjust it and the system learns from the correction.

Inference Before Asking

The system tries to understand what happened before asking explicit questions. If the narrative contains enough information—incident severity, barriers present, outcome—it performs the assessment automatically. Only when details are ambiguous does it probe deeper with targeted follow-ups: "Did you maintain full control?" "Was there any point where things could have gotten worse?"

This approach reduces reporter burden significantly. They describe what happened naturally, and risk assessment emerges from that description rather than being forced through a classification matrix. The cognitive load shifts from "what dropdown option matches this situation?" to simply "what happened?"

Flight Context Integration

For pilots filing reports about recent flights, the system already knows the operational context. If a pilot recently flew from London to Manchester on BA1234, the system doesn't make them manually enter flight number, route, aircraft registration, and timing. Instead, it prompts via voice: "Were you reporting about flight BA1234 from London to Manchester today?"

System automatically pulling recent flight data and confirming with the reporter

The reporter confirms or corrects, and the system pre-populates everything it already knows from operational data. This turns what would be five minutes of manual data entry into a five-second confirmation.

Photo Attachments with Visual Context Analysis

We designed for multi-modal input from the start: voice, photos, and operational data working together to accelerate reporting. The goal was to meet reporters wherever they were in the workflow, whether that meant speaking first or leading with a photo.

Visual context analysis and multi-modal reporting workflow demonstration

When a photo is uploaded first, the system analyzes it while pulling in flight context if the reporter recently operated an aircraft. A pilot who just landed uploads a windshield chip photo. The conversation begins with aircraft registration, flight number, and timing already populated, plus an initial observation: "I can see damage to the windshield. Was this discovered during preflight or did it occur in flight?"

By combining voice, visual analysis, and operational context, the system reduces what reporters need to manually input. The faster they can file a complete report, the more likely they are to file it at all.

Results & Impact

What We Learned