Eval infrastructure
eval-harness-template
@ai-native-agency/eval-harness
Live on GitHubApache 2.0
Production-grade evaluation harness scaffold for LLM workflows. TypeScript + Python bindings, gating thresholds, regression detection, CI integration.
What it does
Lets you define labelled test sets, expected outputs, scoring rubrics, and gating thresholds — and runs them in CI on every prompt or model change.
Why it exists
Every engagement we ship requires an eval harness. We open-sourced the template so clients can fork it and operate it themselves at exit.
MCP / tool registry
mcp-server-template
@ai-native-agency/mcp-server
Live on GitHubApache 2.0
Production-ready Model Context Protocol server scaffold with auth, rate limiting, audit logging, and standard tool registry patterns.
What it does
Spin up an MCP server in 10 minutes with the security defaults regulated mid-market needs: auth, rate limiting, structured audit log, schema validation.
Why it exists
MCP is becoming the 2026 standard for tool registries. We needed a hardened starter — sharing it openly accelerates the ecosystem.
Compliance / governance
audit-log-spec
@ai-native-agency/audit-log-spec
Live on GitHubApache 2.0
JSON Schema for AI inference audit logs. WORM-compatible, regulator-ready. Maps cleanly to HIPAA, FINRA, GDPR retention requirements.
What it does
Specifies the exact fields every AI inference must log to be defensible to an auditor: input fingerprint, retrieval bundle hash, model version, prompt hash, output, downstream action, reviewer disposition.
Why it exists
There is no industry-standard schema for AI audit logs. We published ours so other agencies and in-house teams can converge on a defensible baseline.
Cost / latency optimization
claude-multi-model-router
@ai-native-agency/claude-multi-model-router
Live on GitHubApache 2.0
Cost-optimized router for Claude Opus / Sonnet / Haiku — picks the right model per request based on complexity, latency budget, and cost ceiling.
What it does
Wraps the Anthropic SDK with a router that downsizes routine calls to Haiku, escalates complex reasoning to Opus, and caches deterministically. Cuts mid-market inference bills by 40-70%.
Why it exists
Every engagement we run uses multi-model routing to keep Run costs in check. The router was tribal knowledge — now it's a public package.
Prompt engineering
prompt-library
@ai-native-agency/prompt-library
Live on GitHubApache 2.0
Versioned, evaluated prompt patterns for production workflows: customer service triage, document extraction, compliance review, lead qualification.
What it does
Battle-tested prompt patterns for common mid-market workflows, each with a labelled eval set and benchmark scores against Claude/GPT/Gemini.
Why it exists
Most prompt libraries are personal collections without eval data. We ship ours with the evals attached so you can verify before adopting.