Pytest-LLM
by Community / Open Source · free · Last verified 2026-03-17
Pytest-LLM is a pytest plugin that brings structured evaluation and assertion capabilities to LLM-powered applications, enabling developers to write deterministic and behavioral tests for AI outputs. It integrates with standard pytest workflows to assert on semantic correctness, tone, factuality, and safety of language model responses.
https://github.com/pytest-llm/pytest-llm ↗C
C—Below Average
Adoption: CQuality: B+Freshness: ACitations: CEngagement: F
Specifications
- License
- MIT
- Pricing
- free
- Capabilities
- LLM output assertions, semantic testing, pytest integration, behavioral evaluation, regression testing
- Integrations
- pytest, OpenAI, Anthropic, LangChain, LiteLLM
- Use Cases
- CI testing for LLM apps, regression detection in model upgrades, tone and safety assertion, semantic similarity checks
- API Available
- No
- SDK Languages
- python
- Deployment
- local, ci-cd
- Rate Limits
- No limits — open source
- Data Privacy
- No data transmitted; runs locally
- Tags
- testing, llm, pytest, evaluation, python, qa
- Added
- 2026-03-17
- Completeness
- 100%
Index Score
42.4Adoption
45
Quality
72
Freshness
80
Citations
40
Engagement
0