ContractNLI
by Koreeda & Manning / Stanford NLP · free · Last verified 2026-03-17
ContractNLI is a dataset for natural language inference (NLI) focused on contract understanding. It challenges models to determine if a hypothesis about a contract is entailed, contradicted, or not mentioned by the contract text. This simulates real-world legal document review, testing a model's ability to reason over complex legal language.
https://stanfordnlp.github.io/contract-nli/ ↗B
B—Above Average
Adoption: BQuality: AFreshness: BCitations: B+Engagement: F
Specifications
- License
- Apache-2.0
- Pricing
- free
- Capabilities
- Natural Language Inference (NLI), Legal Text Understanding, Document-level Reasoning, Text Classification, Contract Clause Analysis, Information Extraction from Legal Documents, Benchmarking Legal AI Models, Few-shot Learning Evaluation
- Integrations
- Use Cases
- [object Object], [object Object], [object Object], [object Object]
- API Available
- No
- Evaluated Models
- gpt-4o, claude-opus-4, deberta-v3-large, roberta-large
- Metrics
- accuracy, f1-score
- Methodology
- 17 hypothesis templates tested against 607 non-disclosure agreements. Three-way classification (entailment / contradiction / not-mentioned) evaluated at the span level. Models are fine-tuned or prompted zero-shot.
- Last Run
- 2025-11-05
- Tags
- legal, nli, contract, document-understanding, classification, dataset, benchmark, legal-tech, natural-language-processing, text-classification
- Added
- 2026-03-17
- Completeness
- 0.9%
Index Score
60.8Adoption
63
Quality
84
Freshness
65
Citations
75
Engagement
0