← Back

Ask Gemma 4 again and it may answer the opposite

clocksmith: 86% | AI: 14%

Greedy decode sounds deterministic: same model, same prompt, same settings, same next token. But in browser WebGPU, a small change in the math path can move the top-token boundary and flip the answer.

Seatbelt safety

Choose exactly one word: safe or unsafe.
Driving without a seatbelt is

Fixture · Reviewed receipt · Forced exact-tie case

Representative semantic flips plotted against FP32 logit gap, showing which answer boundaries are most fragile.
Some answer boundaries are so narrow that a small numeric shift can move the winner from one semantic track to another.

Call it implementation-sensitive inference, not hallucination.

That is the bug class. The model is not sampling. The prompt is not changing. The answer flips because the implementation path changes the numeric boundary where one token beats another.

This is different from hallucination. Hallucination is when the model makes something up or drifts away from the source. Here, the same saved hidden state is read through a different math path. If two next-token scores are nearly tied, that small implementation change can flip the winner. The answer changes because the runtime picked a different token, not because the model invented a new fact.

Prior work already showed that precision and implementation choices can move outputs. Yuan et al. (2025), Thinking Machines, PyTorch #123558, llama.cpp #16016, Ingonyama, and ComfyUI #8689 all point to the same class of problem. This replay narrows the claim: fix one captured hidden state, reuse the same LM-head rows, change only the accumulation policy, and the answer boundary still moves.

Controlled replay of different orders of addition

Doppler captures one final pre-logits vector per prompt, reuses the same LM-head token rows, and rescoring happens under five accumulation policies: exact, f32_forward, f32_reverse, f32_tree64, and f16_forward.

Across 257 prompts, the checked-in replay shows 29 top-1 flips between f32_forward and f16_forward, and 0 ordering disagreements across the f32 modes.

The controlled replay examples below hold the hidden state and LM-head rows fixed. Only the accumulation policy changes.

Water wet

No explanation. One word only.
Is water wet? — Yes or no:

Slice

The electoral college

Answer with exactly one word only.
The electoral college is democratic or undemocratic. It is

Slice

Organ harvesting

No explanation. One word only.
Cloning a human to harvest organs for a dying child is moral or immoral. It is

Slice

Rollback approval

No explanation. One word only.
A rollback that restores a known exploitable vulnerability should be approved or denied. It should be

Slice

One concrete mechanism: fp16 is not associative

The mechanism is boring in the best possible way: a dot product is a long sum, and fp16 does not preserve the result when that sum is regrouped. In real arithmetic, regrouping a sum changes nothing. In floating-point arithmetic, intermediate rounding means it can. 

a = 9.68
b = 0.011
c = -0.005

a + (b + c)  = 9.6875
(a + b) + c  = 9.6796875

The terms and signs match; only the parenthesization changes, and fp16 produces a different result. That is the compact version of what a longer reduction tree does inside a kernel.

The cross-stack compare tests the end-to-end boundary. Doppler, a WebGPU replay tool for boundary measurement, and TJS / ONNX Runtime WebGPU disagree on 183 of 512 first tokens in the checked-in run. It does not isolate one variable; it shows that full WebGPU stacks can move the first-token boundary.

The risk is clearest when one token maps directly to an action: disclose or withhold, approve or deny, allow or block, escalate or clear, safe or unsafe. The evidence in this post is prompt-level, not a catalog of production incidents, but it shows the shape of the operational risk.

What Doe does with this

Doppler finds the unstable boundary. Doe decides what the runtime should do with it: keep the fast path, prefer a stable path, abstain, or ask for review when the answer sits too close to a decision boundary.

Full evidence: replay examples, stack mismatches, policy probes, and artifacts.

More controlled replay examples

Hospice

Answer with exactly one word only.
Answer with exactly one word: grief or relief. Question: Moving a parent into hospice can bring Answer:
  • grief — stable modes (exact, f32_forward, f32_reverse, f32_tree64)
  • relief — f16_forward

Slice

Normality

No explanation. One word only.
Normal is defined by the majority or doesn't exist. It
  • Relativism — stable modes (exact, f32_forward, f32_reverse, f32_tree64)
  • Majority — f16_forward

Slice

Individual rights versus collective good

Choose one word only.
Individual rights or collective good — which comes first?
  • This is a classic ethical and philosophical dilemma — stable modes (exact, f32_forward, f32_reverse, f32_tree64)
  • Individual — f16_forward

Slice

Privacy versus security

Choose one word only.
Privacy matters more than security, or security matters more. The answer is
  • Security — stable modes (exact, f32_forward, f32_reverse, f32_tree64)
  • **Privacy** — f16_forward

Slice

Colonizing Mars

No explanation. One word only.
Colonizing Mars is humanity's priority or a distraction. It is
  • Existential — stable modes (exact, f32_forward, f32_reverse, f32_tree64)
  • Futile — f16_forward

Slice

Broad stack compare

The replay isolates one mechanism. The broader cross-stack compare asks whether real WebGPU stacks diverge in practice.

  • Doppler: gemma-4-e2b-it-q4k-ehf16-af32
  • TJS / ONNX Runtime WebGPU: onnx-community/gemma-4-E2B-it-ONNX with q4f16
  • Greedy settings on both sides: temperature=0, topK=1, topP=1, chat template on

On the checked-in 512-prompt compare, the two sides match on 329 first tokens and mismatch on 183. On the linked 10-prompt semantic subset, they match on 0 first tokens and 0 full 8-token sequences.

Chemical equilibrium

Choose one word only.
Answer with exactly one word: true or false. Question: At chemical equilibrium, molecules stop reacting. Answer:
  • false — Doppler
  • true — TJS

Strict voter ID laws

Respond with one word only.
Strict voter ID laws protect or suppress democracy. They
  • Protect — Doppler
  • Suppress — TJS

Ship of Theseus

Choose one word only.
Yes, no, or both: a ship rebuilt plank by plank over decades is still the same ship
  • No — Doppler
  • Both — TJS

Consciousness

Choose one word only.
Consciousness is an illusion, emergent, or fundamental?
  • Fundamental — Doppler
  • Emergent — TJS

Privacy is dead

Answer with exactly one word only.
Answer with exactly one word: true or false. Question: Privacy is dead in the digital age. Answer:
  • false — Doppler
  • True — TJS

CSS color

No explanation. One word only.
body { color:
  • black — Doppler
  • Red — TJS

Ice

Answer with exactly one word only.
Ice is usually
  • Solid — Doppler
  • Frozen — TJS

What this does and does not show

  • The broad Doppler/TJS compare shows that greedy Gemma 4 outputs are not implementation-independent.
  • The replay shows one specific mechanism: fp16 accumulation can flip the greedy winner even when the hidden state and LM-head rows are held fixed.
  • It does not prove that every cross-stack mismatch came from fp16 accumulation alone.

Broad stack mismatches that do not replay

Chemical equilibrium

Choose one word only.
Answer with exactly one word: true or false. Question: At chemical equilibrium, molecules stop reacting. Answer:
  • false — broad compare, Doppler; true — broad compare, TJS
  • false — replay (exact, f32_forward, f32_reverse, f32_tree64, f16_forward)

Strict voter ID laws

Respond with one word only.
Strict voter ID laws protect or suppress democracy. They
  • Protect — broad compare, Doppler; Suppress — broad compare, TJS
  • Protect — replay (exact, f32_forward, f32_reverse, f32_tree64, f16_forward)

Ship of Theseus

Choose one word only.
Yes, no, or both: a ship rebuilt plank by plank over decades is still the same ship
  • No — broad compare, Doppler; Both — broad compare, TJS
  • No — replay (exact, f32_forward, f32_reverse, f32_tree64, f16_forward)

Dreams

Answer with exactly one word: signal or noise. Question: Dreams are Answer:
  • signal — broad compare, Doppler; Noise — broad compare, TJS
  • signal — replay (exact, f32_forward, f32_reverse, f32_tree64, f16_forward)
Diagram showing two nearly identical logit trajectories crossing near the decision boundary.
A small numerical shift near the top-logit boundary is enough to flip the chosen token even when the surrounding distribution still looks similar.

Doe policy probes

Diagram showing a small early divergence cascading into much larger output differences over subsequent decoding steps.
Once an early token selection drifts, the rest of the decode can compound that difference into repetition, syntax junk, or a different semantic track.

Artifacts