State of the Art Survey (2016 → 2026)¶
Last reviewed (W12.T04): 2026-08-15 — Larq archived 2026-06-15; competitor table vs Brevitas / bitnet.cpp / torchao / this lab. No LCE FPS. No invented goldens. No GPU 32×.
Classic CNN BNNs (vision)¶
| Method | Year | Weights | Activations | Key idea | Practical note |
|---|---|---|---|---|---|
| BinaryConnect | 2015 | ±1 | FP | Binary weights only | Memory win, limited compute win |
| BinaryNet / BNN | 2016 | ±1 | ±1 | STE + binary GEMM kernel | Foundational; GPU kernel demo ~7× vs unoptimized |
| XNOR-Net | 2016 | ±1 | ±1 | Channel-wise scaling α | Better ImageNet than plain BNN |
| DoReFa-Net | 2016 | low-bit | low-bit | Multi-bit QAT, bit-conv | Flexible bitwidths |
| ABC-Net | 2017 | multi-binary bases | multi-binary | Approximate full-precision with K binary bases | Accuracy↑, speedup↓ |
| Bi-Real Net | 2018 | ±1 | ±1 | FP residual shortcuts every block | Huge accuracy recovery; still standard recipe |
| ReActNet | 2020 | ±1 | ±1 | RSign / RPReLU distribution reshape + distillation | SOTA-class 1-bit CNNs on ImageNet (~within 3% of FP MobileNet) |
| SURGE | 2026 (ICML) | ±1 | ±1 | Learnable surrogate gradients (DPGC) | Addresses STE mismatch |
Vision accuracy reality (ImageNet top-1, approximate)¶
| Model family | FP baseline | Best binary-ish | Gap |
|---|---|---|---|
| ResNet-18 class | ~69–70% | Bi-Real ~56%, later methods ~60%+ | Still material |
| Compact MobileNet-scale + ReActNet tricks | ~72% | ReActNet ~71% claimed in paper | Near-parity with architecture changes |
Lesson: Naive “sign everything” collapses accuracy. Bi-Real shortcuts + activation reshape + longer training close most of the gap for CNNs.
LLM / Transformer era (2023–2026) — where practice moved¶
| Method | Year | What is quantized | Status in practice |
|---|---|---|---|
| BitNet | 2023 | 1-bit weights, higher-bit activations | Proof that Transformers can train with BitLinear |
| BitNet b1.58 | 2024 | Ternary ({-1,0,1}) weights, ~8-bit acts | Matches FP16 LLaMA-class from ~3B+ at same tokens; Microsoft bitnet.cpp |
| BitNet a4.8 | 2024 | 4-bit activations for 1-bit LLMs | Further activation compression |
| bitnet.cpp | 2024–2026 | Inference kernels CPU (+ GPU path) | ARM 1.37–5.07×, x86 2.37–6.17× vs FP; energy −55–82% |
| BitNet-b1.58-2B-4T | 2025 | Official 2B HF model | Production-usable artifact |
| Litespark Inference | 2026 | SIMD ternary kernels for CPUs | Claims up to ~18× Apple Silicon, ~96× some x86 vs naive PyTorch |
| Sparse-BitNet | 2026 | 1.58-bit + N:M sparsity | Extra ~1.3× with sparse tensor cores |
| BitEmbed | 2026 | Ternary embedding encoders | Extends BitNet to retrieval |
BitNet b1.58 headline numbers (paper)¶
At 3B params (100B tokens pretrain):
- Memory: 3.55× less than FP16 LLaMA twin
- Latency: 2.71× faster
- PPL / zero-shot: matches FP16 twin
- At 70B: throughput 8.9× higher (batch capacity from memory)
Ternary (not pure ±1) matters: the 0 enables feature filtering and closes the quality gap.
Quantization-aware training & “almost binary”¶
| Approach | Bits | When to use |
|---|---|---|
| PTQ INT8 / INT4 (torchao, bitsandbytes, AWQ, GPTQ) | 4–8 | Default production path for LLMs on GPU today |
| QAT (Brevitas, torchao QAT) | 4–8 | Recover accuracy when PTQ fails at ≤4-bit |
| Binary / ternary from-scratch | 1 / 1.58 | Max efficiency on CPU/edge; needs special kernels |
| LUT methods (T-MAC, ternary LUT ASICs) | 1–2 | Replace mul with table / conditional add |
2026 industry truth: For NVIDIA datacenter GPUs, INT4/FP8 via torchao / vLLM / TensorRT usually beats “research BNN simulation.” For CPU / mobile / NPU / custom silicon, 1-bit / 1.58-bit is the frontier for extreme latency/energy.
Tooling landscape (2026)¶
Larq vacuum: larq/larq (TF/Keras BNN training) was archived 2026-06-15 and is read-only. This lab occupies packed PyTorch BNN optimisation — wrap / policy / QAT + uint64 XNOR–popcount + dual metrics — not TF/Keras and not LLM tok/s.
Do not quote Larq Compute Engine FPS as ours. Do not claim GPU 32× from sign().
| Tool | Stack | Occupies | Honest caveat |
|---|---|---|---|
| Larq + Larq Compute Engine | TF/Keras | Classic CNN BNN train + packed ARM deploy (while alive) | Archived 2026-06-15. Still readable; not a PyTorch default. No LCE FPS claimed here. |
| Brevitas | PyTorch | Flexible QAT including 1-bit | Training / export library — needs a separate runtime for real packed speed |
| torchao | PyTorch | FP8 / INT4 / INT8, QAT, sparsity on GPU | Not a 1-bit BNN stack; default for datacenter GPU quality |
| bitnet.cpp | C++ / CUDA | Ternary LLM inference (tok/s, energy) | Best open path for BitNet-style CPU LLM chat — we bridge, we do not compete |
This lab (bnn-lab) |
PyTorch + packed CPU kernels | PyTorch packed BNN optimiser after Larq’s archive: bnn.optimise, .bnnpack, XNOR–popcount, dual-metric REFUSE |
Lab/product for CPU/edge wrap — not llama.cpp, not ImageNet SOTA, not GPU 32× |
Related-work maintenance (W12.T04)¶
| Axis | This lab’s honest position | Cite / do not claim |
|---|---|---|
| Classic BNN accuracy | Canaries (MNIST / CIFAR proxy / synth audio) within tests/golden_floors.json |
Not ImageNet SOTA; not production ASR |
| Classic BNN DX | Larq (TF/Keras) was the default; archived 2026-06-15 | This lab occupies PyTorch packed BNN optimiser + honest routing. Not LCE FPS. |
| Packed CPU speedup | Wall-clock from results/benchmark.json; dual-metric vs theory 32× |
Never advertise theory pack ratio as latency |
| LLM serve | Bridge to bitnet.cpp / GGUF (bnn bridge cpu-llm) |
Not sign()+torch chat models |
| GPU datacenter | Bridge to torchao / AWQ / vLLM (bnn bridge gpu) |
Classic BNN XNOR is a non-goal on Tensor Cores |
Refresh this table when new SOTA papers land; keep claims whitelist in docs/PUBLICATION_PLAN.md.
What “works” in 2024–2026 practice (decision tree)¶
Need faster NN?
├─ Datacenter GPU serving LLM?
│ └─ Prefer FP8 / INT4 (torchao, vLLM, TensorRT). Binary rarely wins on CUDA TC.
├─ CPU / edge / mobile / NPU?
│ ├─ LLM → BitNet b1.58 + bitnet.cpp (or Litespark-class ternary kernels)
│ └─ CNN → Bi-Real / ReActNet recipe + this lab (`bnn-lab`) packed XNOR (Larq/LCE archived 2026-06-15)
└─ New silicon / max energy efficiency?
└─ Design for binary/ternary datapath from day one (LUT / CIM / BGEMM)
Citations (core)¶
- Courbariaux et al., Binarized Neural Networks, 2016
- Rastegari et al., XNOR-Net, ECCV 2016
- Liu et al., Bi-Real Net, ECCV 2018 / IJCV
- Liu et al., ReActNet, ECCV 2020
- Wang et al., BitNet, 2023; Ma et al., BitNet b1.58, 2024 (arXiv:2402.17764)
- Microsoft bitnet.cpp technical reports 2024–2025 (arXiv:2410.16144, 2502.11880)
- Bannink et al., Larq Compute Engine, MLSys
- Litespark Inference, arXiv:2605.06485 (2026)