A 150B MoE on a Developer Laptop: An Upper Bound on What Faster Storage Buys You

Frontier-scale models are being published as open weights faster than most teams have worked out what to do with them. The question I wanted answered was narrow and practical: can an engineer run one of these on the laptop they were issued, and if so, what is actually holding it back?

The use case is evaluation rather than production. Before anything gets deployed, somebody has to find out whether a model holds a house style across a long document, whether it calls tools without falling over, whether its output is worth the trouble at all. That work belongs on an engineer’s own machine, using material that has not been sent anywhere, and it does not need throughput. It needs the model to run.

Conventional wisdom says it will not, or that it will crawl, and the reason given is always storage. A model this size cannot sit in memory, so the parts you are not using have to stream off a disk, and the disk becomes the wall. The engine used here documents a reference machine where one token takes 24 seconds and states plainly that the bytes have to arrive, so faster silicon will not help.

That was my hypothesis going in. On a machine with a current NVMe, storage would still dominate, and the number worth measuring was how badly.

Across a three minute generation the drive spent under eleven seconds actually reading anything. A disk with infinite bandwidth would have moved the result from 1.11 tokens per second to roughly 1.18.

That figure is a bound, not a benchmark. Your machine will produce different throughput than mine, but the ceiling on what faster storage could win is the part that transfers, and on any drive of this class it is small.

The mechanism

A sparsely-activated mixture-of-experts model has a large parameter count and a small activation count. A router picks a handful of experts per token per layer and the rest of the network sits idle for that token.

Colibrì, an Apache-2.0 inference engine by Vincenzo Fornaro, treats that property as a placement problem (Fornaro, 2026a). Rather than requiring the model to fit in memory, it stages VRAM, RAM and storage as one hierarchy: the dense part of the network stays resident, the routed experts live on disk, and the engine reads the ones the router asks for. It keeps a per-layer cache, records which experts a workload routes to, and pins the hot ones. The engine is one C file per model family with no BLAS and no Python at runtime.

The project is explicit that this is research rather than a product, and that its scope is wider than storage. Its stated goal covers “model formats, memory hierarchy, storage I/O, placement, scheduling, kernels, speculation, and CPU/GPU overlap”, and it ships a fully-resident configuration in which the disk drops out of the decode path entirely. What follows measures one machine at one point on that hierarchy.

I ran it on my work laptop

An AMD Ryzen AI 9 365 with 20 logical CPUs and 61 GB of RAM, a single WD Blue SN5100 1 TB NVMe, an integrated Radeon 880M this engine cannot use, and Ubuntu 24.04.

Nothing about that is unusual for an engineering laptop in 2026, which is why I used it. Benchmarks published on eight H100s are interesting and I cannot act on them. I wanted to know what the machine on my desk would do, and I suspect most people reading this want the same. If you have a current NVMe, enough RAM to hold a dense set plus a cache, and 85 GB of free disk, everything below should reproduce. There is no accelerator anywhere in it, which conveniently removes the variable that usually decides these comparisons before they start.

Two things about the silicon are worth knowing before anyone reads too much into the CPU figures later. The part mixes performance and compact cores at different sustained clocks, and a laptop chassis holds a lower sustained power limit than a desktop would. Either could inflate CPU time per unit of work, and neither is something I measured.

The model was DeepSeek V4 Flash, in the REAP-pruned 150B variant published by puwaer: 150,128,549,111 parameters across 43 layers of 132 experts, routing to six per token, occupying 84.7 GB (puwaer, 2026). Routed experts are native fp4, which is why Colibrì reads the published checkpoint with no conversion step.

REAP is an expert-pruning method whose authors report near-lossless compression on code generation at fifty percent of experts removed, on models from 20B to 1T parameters (Lasby et al., 2025). I did not evaluate that claim. This post makes no quality argument at all, and the one answer reproduced below contains a factual error about its own architecture, which is worth holding on to.

Colibrì v1.11.0, built by its own setup.sh with gcc 13.3.0 at -O3 -march=native -fopenmp -flto. That build line matters more than it looks for a hand-rolled no-BLAS engine, since -march=native on Zen 5 is the difference between measuring the workload and measuring your compiler.

Measure the storage first

The engine refuses to plan without a storage measurement and marks it required. It ships its own benchmark, and the block size is not arbitrary: each expert’s matrices are stored adjacently and read in one call, so the read that matters is several megabytes rather than several kilobytes.

M=~/models/dsv4-reap-150b/model-00003.safetensors
./iobench $M 13 200 1 1
./iobench $M 13 200 4 1
./iobench $M 13 200 16 1
./iobench $M 13 200 4 0

Listing 1. Four runs varying one parameter at a time. Arguments are file, block size in MB, read count, thread count, and whether O_DIRECT is used.

With O_DIRECT: 5.67 GB/s at one thread, 6.17 at four, 5.72 at sixteen. Buffered at four threads: 7.43 GB/s, though that figure is inflated by the page cache on a file written minutes earlier and should not be used as a baseline for anything.

One run per configuration, no variance reported, so the 8% spread across thread counts is not a result. What the numbers do support is that a single thread already reaches within 10% of the best figure observed, which is unsurprising given that a 13 MB application read is split into many device-level requests before it reaches the drive.

For scale, the reference machine in Colibrì’s GLM-5.3-Flash documentation measures 72 MB/s at queue depth 1 and saturates near 200 MB/s, and that drive imposes a floor of 24 seconds per token on that model (Fornaro, 2026b). A 2026 consumer NVMe is roughly eighty times faster at the comparable setting. The floor that dominates that document is a property of one drive, which is worth knowing before anyone reads those figures as the cost of the technique.

The plan

RAM    39.6 GB budget · 8.8 GB dense · 5.9 GB runtime · 24.9 GB warm experts · cap 43/layer
disk   51.0 GB cold experts · 416.6 GB free
limit  disk expert misses
hit    33% projected expert residency

Listing 2. The planner splits a RAM budget between dense weights, runtime and expert cache, then projects how much of the expert set will be resident.

Two thirds of the expert set stays on disk and the planner names disk misses as the expected constraint. It disabled speculative decoding, on the reasoning that drafting widens the set of experts a token needs, and enabled read and compute overlap.

That overlap setting is load-bearing for everything that follows, and I will come back to it.

The self-tuning pass then swept eleven candidates: one OMP team size, three loader-thread counts, two smaller RAM caches, two CUDA paths that a CPU-only run still evaluates, and confirmation runs. None cleared its three percent improvement gate, so the defaults were kept. The profile recorded 0.92 tokens per second at a 66.75 percent hit rate.

The run

time ./coli run --model ~/models/dsv4-reap-150b --no-think --ngen 200 \
  "Write one paragraph explaining what a mixture-of-experts model is."

Listing 3. Thinking mode is a time control on a streaming engine rather than a matter of taste, and this checkpoint defaults to it being on.

Two hundred tokens in 180.1 seconds: 1.11 tokens per second, or 0.90 seconds per token. Time to first token 17.2 seconds on a seventeen-token prompt. Expert hit rate 87.6 percent, from 46,665 hits against 6,597 misses across 53,262 selections.

The answer was coherent and said that mixture-of-experts models activate “typically just two or three” experts. This checkpoint routes to six.

What the disk actually cost

v4_tokens prompt=17 generated=200 expert_requests=53262 hits=46665 misses=6597 bytes=88197562368
v4_direct reads=4816 flock_reads=0 fallbacks=1781 payload_bytes=60599304192
timing time_to_first_token=17.175s after_first=162.946s
real 3m2.409s / user 46m32.525s / sys 0m22.993s

Listing 4. Read counters and payload for the full generation.

Two counters report bytes and they disagree, so it is worth saying which one this analysis uses and why. v4_direct reads plus fallbacks comes to 6,597, which is exactly the miss count, so payload_bytes at 60.6 GB is the physical I/O attributable to misses: one read per miss, averaging 9.19 MB. The larger 88.2 GB figure in v4_tokens does not divide into any plausible per-read size and is some other accounting. If you reproduce this, check both.

So: 60.6 GB delivered by a drive measured at 5.67 GB/s is 10.7 seconds of device time across a 180-second generation. Remove the disk entirely and the same work finishes in about 169 seconds, which is 1.18 tokens per second. The ceiling on infinitely fast storage is about six percent.

The calculation needs no claim about where the other 169 seconds went. It is bytes divided by measured bandwidth, with no timing in it.

Halving the cache

A bound is not a diagnosis. If misses are so cheap, the obvious test is to cause a lot more of them and see what happens. So I pinned the cache budget explicitly, dropped the page cache before each run so nothing was read from memory, and halved it.

sudo sh -c 'sync; sysctl -w vm.drop_caches=3'
time ./coli run --model ~/models/dsv4-reap-150b --no-think --ngen 200 --ram 20 \
  "Write one paragraph explaining what a mixture-of-experts model is."

Listing 5. Arm B. The page cache is dropped first, otherwise the second run reads from RAM and the comparison measures nothing.

Halving the cache cost 15% of throughput, so misses are not free and the engine’s own advice to give it memory is right. My first draft implied memory had stopped paying at this configuration, and that was too strong.

The interesting part is the accounting. Decode got 34.7 seconds slower. Of that, 16.9 seconds is the extra device time from reading 96 GB more. The user CPU went up by 329 seconds, which spread across 17 threads is about 19 seconds of wall clock. The two together come to 36 seconds against an observed 35, which is close enough to say that a cache miss costs you roughly as much CPU as it costs you disk. Dequantising an fp4 expert, copying it into place, evicting something else and updating the bookkeeping is real work, and it scales with misses just as reads do.

That reframes what memory is buying. The README explains the benefit through read volume. On this machine about half the benefit arrives as CPU you do not have to spend.

And the drive still never becomes the constraint. Two and a third times the traffic raised decode time by 18%, and even at the worse setting device time was 29.5 seconds of a 226 second run.

Ruling out spin-wait

Before claiming any of that CPU time was arithmetic, there is an obvious objection. GNU OpenMP spins by default while waiting at a barrier, so a thread stuck behind a straggler or behind a loader sitting in pread burns CPU and computes nothing. High user time proves the threads were runnable and nothing more.

time OMP_WAIT_POLICY=passive ./coli run --model ~/models/dsv4-reap-150b --no-think --ngen 200 --ram 40 \
  "Write one paragraph explaining what a mixture-of-experts model is."

Listing 6. The same configuration as the 40 GB run, with OpenMP told to sleep at barriers instead of spinning.

Decode 191.703 seconds against 191.290, user CPU 49m 16s against 49m 10s. Hit rate, bytes and read counts match to three significant figures. Nothing moved.

So the threads were not spinning at OpenMP barriers. One caveat I will not paper over: this rules out that specific mechanism, and the engine runs its own loader threads with their own synchronisation that OMP_WAIT_POLICY does not touch. Spinning somewhere else remains possible and I have no instruction counts to exclude it.

Those two runs also give something I was missing. 191.290 and 191.703 seconds at identical settings is 0.2% apart, so run-to-run variance on this machine is small enough to ignore at the resolution anything here is argued at.

Three things that cut the other way

Prefill was slower per token than decode. 17.2 seconds for 17 prompt tokens is 1.01 seconds each, against 0.90 in decode. Prefill is batched and should be faster per token. The likely explanation is a cold sweep through experts, which would make time to first token nearly pure I/O and roughly ten percent of the wall clock genuinely storage-bound.

One short prompt touched three quarters of the model. 4,216 distinct experts out of 5,676 in 200 tokens. Whatever the cache is learning, it is not learning that this workload uses a small corner of the network.

Residency and hit rate are different quantities. The planner projected 33 percent residency and the run recorded 87.6 percent hit rate. Those are a capacity ratio and a routing outcome, and the gap between them may be a skewed router rather than anything learned. The project lists learned pinning in its own open-hypotheses table, with the note that history “can overfit a prompt” and that the validation it wants is held-out cross-session A/B testing. Running one prompt repeatedly is the overfitting case, not evidence against it.

Where this leaves the project’s advice

Colibrì’s README says of this model family, in bold, “Give it RAM”, and that the expert cache hit rate is what sets tokens per second, making --ram the single most valuable knob (Fornaro, 2026a).

The sweep says that is right, and says the stated mechanism is half the story. Memory does set throughput here. It sets it partly by avoiding reads, which is the documented reason, and about equally by avoiding the CPU cost of servicing a miss, which is not mentioned. If you are tuning a machine, that distinction matters, because it means the payoff from more memory does not disappear the moment your drive stops being busy.

There is a sharper version I got wrong in an earlier draft and want to correct in public. This model is 84.7 GB. A machine with 96 GB or more of usable memory holds all of it, at which point there is no streaming, no cache and no miss rate to improve, and the project publishes exactly that configuration. So the shape of the curve is: memory helps a lot while the hit rate is poor, helps less as it climbs, and then the moment the whole model fits, the streaming machinery and everything it costs in CPU goes away at once.

For comparison, the project’s published ladder for GLM-5.2 at 744B runs from 5.8 to 6.8 tokens per second on six RTX 5090s with full residency, 1.8 on a 128 GB CPU-only desktop, 1.07 on a laptop-class box with an RTX 5070 Ti, and 0.05 to 0.1 on a 25 GB machine reading cold. Its DeepSeek-specific figure is about 1.6 tokens per second at 3k context on an RTX 5080 with two NVMe drives. The 128 GB desktop datapoint is worth sitting with, since it is 62 percent faster than this laptop and the obvious difference is memory.

What this does not say

One machine, one model, one prompt. Four runs, with repeatability of 0.2% between the two at identical settings, which is enough to trust the comparisons and nowhere near a benchmark suite.

It is also one engine, and that is the gap I would most want closed. The CPU time is real work, but it could be a property of this engine’s hand-written kernels rather than of the workload itself. Running a comparable MoE through a second engine on the same box would separate those, and I have not done it. Until someone does, read the compute finding as being about this engine on this machine.

I also have no instruction counts, no device-level I/O trace, and no memory bandwidth figure, so where the CPU time actually goes is unmeasured. The share attributed to miss servicing comes from the difference between two runs, not from a profile.

And 1.11 tokens per second is slow. A 600-word answer is around ten minutes, extrapolated from a 200-token run rather than measured. That is workable for queued jobs and unusable for a conversation.

So can an engineer do this on their own machine

Yes, at 200 tokens in three minutes, which decides what it is good for.

Checking whether a model follows a style guide, holds context across a long document, or handles a tool-calling loop does not need throughput. It needs the model, a prompt and some patience. Queue the job, do something else, read the answer. Anything conversational is out.

The arithmetic that decides whether it will run at all is simple enough to do before installing anything. The dense set has to be resident, the expert cache wants whatever memory is left, and the free disk has to hold the whole checkpoint. This model needed 6.2 GB resident and 85 GB on disk, and 61 GB of system memory left 39.6 GB for the engine to divide. At 20 GB it still worked, 15% slower. On a 32 GB machine there would be very little cache left after the dense set, and the results above suggest that gets painful rather than impossible.

Storage is the part to worry about least. Any current NVMe is well past the point where the drive is what slows you down, and the two bounds are the argument: even after tripling the miss rate on purpose, device time was 13% of the run. Between a faster drive and more memory, the memory is what moves the number, and about half of what it buys is CPU not spent servicing misses.

The ceiling is worth naming too. Once the whole model fits in memory there is no streaming, no cache and no miss rate, and this checkpoint is 84.7 GB. Below that line you are trading against the cache. Above it the entire mechanism this post is about stops being relevant.

I should say what this does to our own position, since Sakura Sky runs managed environments for open-weight models and the measurement above cuts against part of the pitch.

Weights of this class run on an engineer’s laptop. Anyone who has been told that large open models are out of reach without renting hardware has been told something that stopped being true. That makes local evaluation free, and it removes the excuse for not doing it before committing to anything.

What the laptop does not give you is fifty people using it at once, isolation between them, an audit trail, a recovery path, or an answer in under ten minutes. None of that is affected by anything measured here. What has changed is that nobody has to rent hardware to find out whether the model is any good first. Which for me, was the real win from this.

Reproducing it

The engine is at github.com/JustVugg/colibri, with its quick start, tuning notes and benchmark protocol in the same repository. The checkpoint is puwaer/DeepSeek-V4-Flash-0731-reap-150b on Hugging Face, and the engine notes for that family cover the flags used here. Neither the engine nor the checkpoint needed a conversion step, so an afternoon is a build, an 85 GB download and a run.

If you run it, record what the project asks for: hardware, commit, model container, exact command, prompt, cache state, throughput, time to first token, expert hit rate and bytes read. Add the build line and the page cache state, since a warm cache will hand you a number that means nothing. What I still cannot give you is an instruction count or a device-level I/O trace, and both would say more about where the CPU time goes than anything I measured.

The comparison worth making is not tokens per second between machines. It is the ratio of device time to wall clock on your own, because that tells you whether your next purchase should be a drive, more memory, or neither.

Disclosure: Sakura Sky designs and operates managed data, security and AI platforms for customers, including managed environments for open-weight models, which is a commercial interest in this subject and one the argument above partly cuts against. Colibrì is not a Sakura Sky project and we have no relationship with its author. The REAP-pruned checkpoint was published by a third party and its output quality was not evaluated here. All measurements come from a single machine on 16 September 2026 and are reproducible from the commands shown, which is not the same as being representative.

References

Fornaro, V. (2026a) colibrì: run frontier MoE models on hardware you already own. Available at: https://github.com/JustVugg/colibri (Accessed: 16 September 2026).

Fornaro, V. (2026b) GLM-5.3-Flash engine (c/glm53.c). Available at: https://github.com/JustVugg/colibri/blob/main/docs/glm53-flash.md (Accessed: 16 September 2026).

Lasby, M., Lazarevich, I., Sinnadurai, N., Lie, S., Ioannou, Y. and Thangarasa, V. (2025) ‘REAP the Experts: Why Pruning Prevails for One-Shot MoE compression’, arXiv:2510.13999. Available at: https://arxiv.org/abs/2510.13999 (Accessed: 16 September 2026).

puwaer (2026) DeepSeek-V4-Flash-0731-reap-150b. Available at: https://huggingface.co/puwaer/DeepSeek-V4-Flash-0731-reap-150b (Accessed: 16 September 2026).