Kimi K3.1 Pro: Claim-Level Hallucination Check in Python

Moonshot AI shipped Kimi K3.1 Pro on October 23, 2026, and one number from its launch post is getting passed around everywhere: a claim-level hallucination rate of 8.3% on Kimi Hallu Bench Hard, down from 23.4% for Kimi K3. That figure is company-reported, on a benchmark Moonshot built, and your documents are not that benchmark.

This guide shows you how to measure it yourself. You will build a small Python harness that asks the model a question about a document, splits the answer into atomic claims with strict structured output, checks every claim against the source, and reports the unsupported rate at each reasoning_effort level. By the end you will have a number for your own data, plus a feel for what K3.1 Pro's API quirks cost you if you ignore them.