Build a local AI feature
Test the local AI feature
Test deterministic behavior directly and exercise the real model with a small evaluation instead of making every test depend on local inference.
Split verification into two layers.
Unit tests should cover the deterministic parts:
- prompt construction
- schema and output validation
- timeout and error mapping
- fallback sentence
- provider adapter contract
These tests run without Ollama. They should be fast and repeatable.
Integration checks exercise the installed model. Keep a small set of representative activity records and run them against the exact model tag used by the application.
Do not assert one exact generated sentence. Check the contract: valid shape, supported facts, maximum length, acceptable latency, and no invented activity.
Record failures beside the input and model version. A later model upgrade must pass the same cases before becoming the default.
The model is nondeterministic. Your acceptance boundary should not be.
Lesson completed