Phase 2 · Multi-AI Calibration · Live
Where four vision-language models sit relative to plural human readings.
Four models, five prompt conditions, thirty-two clips. Each model's output is scored against the plural human distributions Phase 1 established (Kim, 2026a, accepted at Curieux Academic Journal). The plan was frozen before analysis began. This page shows what the running system is producing right now — the reading of it stays with the pre-registration.
AI readings complete
last run 10h ago
32%
of the plan
Mean JSD · running · per model
Lower is closer to the plural human reading. Values move as new clips complete each day.
Live · running mean across 617 completed runs
Current metrics · per model
Numbers move as runs complete
Gemini 2.5 Pro
Progress
Mean JSD
0.41
Modal agree
42%
Claude Sonnet 4.5
Anthropic
Progress
Mean JSD
0.61
Modal agree
28%
GPT-4o multimodal
OpenAI
Progress
Mean JSD
0.55
Modal agree
26%
Qwen 3.8 27B
Alibaba · open-weight
Progress
Mean JSD
0.52*
Modal agree
27%*
* n < 20 — interpret with caution.
Methodology · at a glance
A 4 × 5 × 32 × 3 design, pre-registered before the runs.
Models
4
Gemini · Claude · GPT-4o · Qwen (open-weight)
Conditions
5
Zero-context → human-calibrated
Clips
32
The frozen Phase 1 corpus
Runs per cell
3
Uniform N — total 1,920 API calls
Every combination is a cell — 4 × 5 × 32 = 640 unique cells. Each cell is run N = 3 times (independent samples with the same prompt), so the total run budget is 640 × 3 = 1,920 API calls. The primary metric is Jensen–Shannon divergence between the model's distribution and the human distribution Phase 1 measured for the same clip; lower = closer.
No clip labels reach the model in the zero- and perspective-taking conditions. The uniform N = 3 per cell also gives us within-model / within-condition variance at every point in the grid — no separate stochasticity subset needed.
Where to go from here