ZZapdev-labs
GitHub
Machine learning lab

We make large language models faster on real hardware.

Zapdev-labs builds quantization, inference runtimes, and benchmarks — Rust and Python tooling that cuts memory, raises tokens-per-second, and ships OpenAI-compatible APIs you can run locally.

  • Weight & KV-cache quantization
  • Local inference (Rust + Python)
  • Cross-backend TPS benchmarks
  • Low-VRAM and edge deployment
github.com/Zapdev-labsSee our projectszapdev.link

12

Open-source projects

3-bit

KV cache compression

>

TPS vs stock llama.cpp

workbench — zapdev-labs

zapdev-labs / oxidize — bench session a91c

$ turboforge hardware probe

→ Ryzen 7 PRO 6850H · 28GB RAM · llama.cpp + vLLM candidates

$ turboquant bench --bits 3 --target kv-cache

→ FastVQ: 6x KV memory reduction · recall within baseline

$ oxidize serve --model qwen3.5-4b-q4

ok OpenAI /v1 ready · oxidize runtime (not llama-server)

$ python benchmark.py --backends llama_cpp,oxidize,miniforge

→ same GGUF · llama.cpp baseline · oxidize + miniforge ahead on TPS

ok fastest this run: oxidize (higher tok/s than llama.cpp)

$ ▊

Flagship work

Open source for faster LLMs

These are the repos we actively push on GitHub — inference engines, quantizers, runtimes, and the benchmarks that prove the wins.

InferenceRust

oxidize

Fast, local-first LLM inference in Rust — CLI, OpenAI-compatible server, Python bindings, and quantization. Beats stock llama.cpp on the same GGUF in our back-to-back TPS benchmarks.

oxidize
QuantizationPython

turboquant

FastVQ on PyPI — PolarQuant, QJL, and TurboQuant for ~3-bit compression, ~6× smaller KV caches, and optimized GPU kernels with near-zero accuracy loss.

turboquant
RuntimePython

turboforge

Hardware-aware enterprise runtime: planner hints, llama.cpp and vLLM backends, benchmark telemetry, and an OpenAI-compatible API with tokens-per-second in the response.

turboforge
ServingPython

miniforge

High-performance MiniMax M2.7 inference with GGUF quantization, TurboQuant KV cache, and hardware-tuned presets — higher TPS than stock llama-server on the same weights in our benchmarks.

miniforge

Install

One command to run models locally

The installer detects your OS and architecture, drops the oxidize binary on your PATH, and falls back to a source build when no prebuilt is published. macOS, Linux, and Windows — x86_64 and arm64.

$ curl -fsSL https://zapdev.link/install.sh | sh

Cargo

From source, any platform

cargo install --git https://github.com/Zapdev-labs/oxidize --bin oxidize --locked

wget

No curl? No problem

wget -qO- https://zapdev.link/install.sh | sh

Piping to a shell runs code from the network. Read it first: zapdev.link/install.sh

More from the org

Benchmarks, memory, and agents

How we ship performance

From probe to production API

Our repos chain together: measure the machine, compress weights and KV cache, benchmark backends, then serve through OpenAI-compatible endpoints.

  1. 01

    Probe hardware

    turboforge hardware probe · miniforge config doctor

  2. 02

    Compress

    turboquant / oxidize-quantize · 3-bit KV and weight paths

  3. 03

    Benchmark

    ollama-performance-benchmark · tokens/sec per backend

  4. 04

    Serve

    oxidize-server · turboforge runtime · miniforge streaming

  5. 05

    Ship

    OpenAI-compatible APIs and reproducible manifests

People

Who builds here

The team behind Zapdev-labs — inference, quantization, and the benchmarks that back our claims.

Proof, not promises

Benchmarks you can rerun

ollama-performance-benchmark runs each backend sequentially on the same GGUF and prompt. On our hardware, oxidize and miniforge beat stock llama.cpp tokens-per-second; results land in results.csv. turboforge persists benchmark history on its runtime path.

turboforge hardware probeoxidize servepython benchmark.py
View benchmark repo
Abstract hardware bench visual for Zapdev-labs

Reference

Common questions

We are a machine learning lab focused on LLM performance: quantization (turboquant), local inference stacks (oxidize, miniforge, turboforge), and honest cross-backend benchmarks. We also ship agent infrastructure like openagentmail and product tooling like zapdev.