An AI feature inside a product is not a chat demo. Users expect it to behave like the rest of the software: predictable, quick to respond, and never the reason the page broke.
Ask for structure, then validate it
Free text is hard to use downstream. Ask the model for a typed object and validate it before anything else touches it.
import { generateObject } from 'ai';
import { z } from 'zod';
const Summary = z.object({
title: z.string().max(120),
points: z.array(z.string()).max(5),
});
const { object } = await generateObject({ model, schema: Summary, prompt });If validation fails, retry once with the error in the prompt, then fall back. Never render unvalidated model output into the interface.
Design for slow and down
- Stream responses where the user is watching, so the first words arrive quickly.
- Move anything that is not interactive to a queue and a background worker, with retries and a timeout.
- Keep a second provider behind the same interface; switching models is a config change, not a rewrite.
- Show a useful non-AI fallback when every provider is unavailable, rather than an error.
Keep a human in the loop where it matters
For anything a user will send, publish or act on, present the model’s output as a draft they confirm. Log inputs and outputs so the team can review quality and catch regressions when a model version changes.
Treated this way, the model becomes one more dependency with known failure modes, and the feature stays as reliable as the product around it.
TechSea engineering team