Measuring What Matters -- or What's Convenient?: Robustness of LLM-Based Scoring Systems to Construct-Irrelevant Factors

6 Steps

1. 1

   Acknowledge Construct-Irrelevant Factor Vulnerability: Understand that your LLM-based scoring system, regardless of its performance metrics, is susceptible to construct-irrelevant factors (CIFs) that can bias scores without reflecting the true underlying construct.

2. 2

   Identify Domain-Specific CIFs: Brainstorm and document specific CIFs relevant to your application. For instance, in educational essay scoring, these could include politeness, specific stylistic choices, or irrelevant content length, if they don't contribute to the core skill being assessed.

3. 3

   Design & Execute Sensitivity Tests: Create test cases that systematically vary identified CIFs while keeping the core, intended construct constant. For example, provide identical content with different stylistic wrappers (e.g., polite vs. rude tone, verbose vs. concise irrelevant introductions).

4. 4

   Implement Interpretability Tools: Utilize AI interpretability tools (e.g., LIME, SHAP, attention visualization) to analyze which parts of the input an LLM prioritizes when making scoring decisions. This helps reveal if the model is over-relying on CIFs.

5. 5

   Conduct Adversarial Testing: Develop or employ adversarial techniques to deliberately probe for CIF vulnerabilities. Generate or modify inputs to trick the LLM into assigning incorrect scores based on irrelevant cues rather than actual merit.

6. 6

   Report Limitations & Iterate: Document identified CIFs, their impact on scoring, and implemented mitigation strategies. Establish a continuous monitoring and refinement process to ensure ongoing robustness, fairness, and validity of your LLM scoring system.