Can We Trust What It Builds? Testing Speech‑to‑Text for Healthcare

This article was first published by Quality IT Resourcing on LinkedIn on October 19, 2026, as Healthcare AI Can’t Afford to Be Almost Right.

From “Can It Build?” to “Can We Trust It?”

AI has changed how fast software gets made. Work that used to take a team weeks can reach a working prototype in days. That speed raises a harder question for any organization putting AI into real work: how do you know the result is reliable?

I watched large language models go from struggling to write working code to writing code that rivals experienced engineers, in what felt like no time at all. If AI could build this fast, the important question was no longer “can it build?” but “can we trust what it builds?”

I studied Computer Science at the University of Manitoba, specializing in AI and machine learning. On the Zokforce Research Team, I build agentic AI systems with LangGraph, MCP, retrieval-augmented generation (RAG), FastAPI and Google Cloud.

A Federal Healthcare Client

One of my projects was evaluating a speech-to-text system for a federal healthcare client. The system had to transcribe conversations in English and French, and get them right through medical terms, different accents, background noise and speakers switching between the two languages.

Medical termsOne wrong word can change a record

Bilingual speech‑to‑textConversations in English and French

AccentsMany speakers, many ways of speaking

Background noiseSound that is not the speaker

Language switchingEnglish and French in one conversation

In healthcare, a transcription error isn’t a cosmetic issue. It can change the meaning of a record.

The client needed evidence that the system was reliable before accepting it, not just a vendor’s word that it “worked well.”

Test Audio With a Known Answer

The usual way to test a system like this is by hand: people read scripts aloud, and someone checks the transcripts by eye. It is slow, two testers rarely judge alike, and no round can be repeated exactly.

I helped design a different approach. Instead of relying on people reading test scripts aloud, we generated test audio with text-to-speech. Because we wrote the script, we knew exactly what should come out.

01Write the scriptThe text we expect back

02Generate the audioText-to-speech reads the script aloud

03TranscribeThe system under test turns it back into text

04ScoreWord and character error rate against the script

RepeatableEvery round runs again the same way after each change, in both languages

Each generated clip ran through the speech-to-text system, and each transcript was scored against its script with two standard measures: word error rate (WER) and character error rate (CER). The test set could grow to cover both languages and the hard cases, and every round could be run again the same way after each change. The client’s acceptance team got repeatable, measured evidence of how the system performed, not impressions.

Finding the Cause, Not Just the Error

Automation tells you quickly that something is wrong. It does not tell you why. When a transcript came back wrong, the hard part wasn’t spotting the error. It was working out why. Was it the audio, the accent, a domain term, the language switch, or the system itself?

Spotting it is the easy partA transcript comes back wrong

The audio

The accent

A domain term

The language switch

The system itself

The hard partUnderstand why it failedThen stop the same failure from coming back

That distinction matters for anyone putting AI into sensitive work. Catching an error is where the work starts. A system becomes dependable when you understand why it failed and stop the same failure from coming back.

Verify as Fast as You Build

For me, testing is part of building AI, not a step after it. My own agent projects, built from LangGraph agents, MCP tools and retrieval, carry an automated test suite that runs on every change.

The problem I see most is that organizations can now build AI fast but can’t verify it fast. Most still evaluate AI by vibes: someone tries ten prompts, the answers look good, and it ships.

By vibesA few prompts and a good feeling

• Someone tries ten prompts
• The answers look good
• It ships

Verify as fast as you build

By evidenceTested on every change

• Correct answers defined in advance
• The difficult cases in every round
• Automated tests after every change
• People review where the stakes are high

Four Habits for Leaders

My advice to enterprise leaders:

1. Define correctness in business terms. Decide what a right answer means for your work before you measure anything.
2. Build evaluation sets around the difficult cases. The common case usually works; test the cases that hurt most when they go wrong.
3. Test automatically after every change. A new model, prompt or data source can break what worked yesterday.
4. Keep people in charge where the stakes are high. Human review stays wherever a mistake has real consequences.

LLM systems fail quietly and unevenly. They do well on the common case and break on the edge cases that matter most.

As AI agents take on more of the work, evaluation will decide which systems move past an impressive demo into dependable daily use. AI can now build code and products faster than ever, but human judgment doesn’t scale at that speed. People get tired, and review is where mistakes slip through. Being able to verify quality at scale may become one of the most valuable capabilities an organization can build.

The same thinking runs through our AI harness engineering work at Zokforce: test cases built with your specialists around the high-risk edge cases, every update graded automatically before release, and every answer checked before anyone relies on it.

References

• Quality IT Resourcing, “Healthcare AI Can’t Afford to Be Almost Right,” LinkedIn, October 19, 2026.
• Aiden Mak, AI Engineer, Zokforce Research Team: Aiden Mak on LinkedIn.