H2O.ai releases H2O Lightning 4B, an Apache-2.0 decision model with GGUF builds
Built on Qwen3.5-4B, with GGUF files from 1.43 GB for llama.cpp. The card reports 88.74% on 231 public JevBench text questions.
H2O.ai published h2o-lightning-4b on Hugging Face on 2 October 2026, by repository creation date. The GGUF repository is the llama.cpp packaging of it. The card describes a decision model that answers typed decisions with probabilities: named choice options, true/false questions and ordered score levels. It serves decisions through a /v1/systemone endpoint.
It has 4B parameters, is built on Qwen/Qwen3.5-4B and is licensed Apache-2.0. GGUF files range from 1.43 GB (IQ1_M) to 8.42 GB (BF16), and the card recommends Q8_0 at 4.48 GB. llama.cpp support needs a recent build.
The numbers are H2O's own. On 231 public JevBench text questions, BF16 gets 205 right (88.74%) and Q8_0 gets 204 (88.31%). Decision Index 0.3 on the public text suite is 51.38 with the launcher. Heavy quantisation costs accuracy: the card lists -3.5 points at IQ3_XXS and -21.1 at IQ1_M. Median Q8_0 latency is 161.5 ms on an RTX PRO 4500. Images were not part of the Index run, and the 30 of 30 image result comes from synthetic development questions.
A small model with a quantisation table, which is the most useful thing on the card.
Sources
Editor's first pass (AI-assisted)
- Real 4: Weights and quantised GGUF files are public; llama.cpp support needs a recent build.
- Useful 3.5: A 4B model that runs on modest hardware and serves /v1/systemone.
- Notable 3: Cited as a comparison point in Microsoft's launch material.
- Novel 2.5: Same format as others, in a small size.
Comments