Qwen3.8-Flash-Next on the HP ZBook Ultra G1a

01 · Setup and memory

Why a 180B model runs on a notebook at all

A model of 176.9 billion parameters on a machine with 128 GiB of memory (125.1 GiB of it usable), whose dedicated graphics memory amounts to 0.5 GiB – that only works because three things come together: how the model is built, how the platform handles memory, and what the engine makes of both.

Why memory is the first question

With a model this size, memory decides before anything else does. It is the hard limit: what does not fit does not run. And what barely fits takes up so much of the machine for the duration of the computation that nothing else has room beside it – on a notebook that doubles as a work machine, that is the difference between usable and blocked.

Two figures therefore stand side by side here: how much memory was tied up during operation, and in what form it was held. For these two engines the answers diverge, and the second one determines how the first should be read.

What operation ties up

A sensor recorded the memory state every two seconds throughout both runs. Two terms in the table below need a word of explanation.

GTT stands for Graphics Translation Table and denotes the part of system memory that the graphics driver makes reachable for the GPU: pages of system memory are mapped into the GPU's address space so that it can read and compute there without copying anything into dedicated graphics memory. On this platform that is the rule rather than the exception – only 0.5 GiB are dedicated, and the ceiling for the GTT is set by the kernel parameter amdgpu.gttsize, here 110 GiB. What the table reports as GTT is therefore not additional memory, but the share of that same system memory handed to the GPU at the time of measurement.

MemAvailable is the kernel's estimate of how much memory a new application could still obtain – including whatever it could free up to satisfy the request. The row “left free” gives the low-water mark of that value across the run, that is, what remained for the rest of the system at the worst moment.

5,065 samples across the complete halogen run. For llama.cpp there are 5,854 samples, but they cover only the last three tasks: that run was carried out in two sessions and the sensor ran during the second. Those three are the context tasks – the most memory-hungry part of the set.

Two ways of holding the same weights

The bottom two rows invite being read as a statement of requirement. They would be that only if both engines claimed memory in the same way. They do not:

• llama.cpp places the weights as GTT allocations with -ngl 999 – device memory drawn from that same system RAM. Such pages cannot be reclaimed; they reduce MemAvailable by their full size.
• halogen maps the checkpoint instead (mapped 124.1 GB). Its pages are clean and file-backed. The kernel counts them as available, because it can drop them at any time and read them back from disk when needed.

The engine's own log shows this plainly: pinned 65.64 GiB is followed a few lines later by host memory left for everything else: 81.1 GiB. The pinned pages do not show up as used memory at all.

What the table reports, then, is how much memory the system holds as allocated. That is the figure that matters in practice – it decides whether the machine can still be worked on alongside the model. It is not a statement of requirement in the sense of “the engine needs this much”, and comparing the two columns measures the kernel's bookkeeping as much as the engines' appetite.

What the SSD is for

Of the 115.6 GiB that are mapped, 65.64 GiB are pinned; the remainder stays as file pages and can be evicted. That makes the drive part of the memory path: it carries the cold start and catches whatever is pushed out under memory pressure. Only this fallback allows a checkpoint of this size onto a machine with 128 GiB at all – and it turns the choice of a fast SSD into a decision about start-up time, not just about disk space.

The start-up itself shows how large that difference is. With the checkpoint still in the page cache, the 65.64 GiB are pinned in 3.6 seconds – 19.4 GB/s, which is to say no disk access at all. Without it, the engine really does read from the drive: 2.1 GB/s, and 3.6 seconds become 33.2. Both figures come from the same machine and the same checkpoint; all that differs is the path the data takes.

How much was paged back in during the runs is not something this measurement shows: the sensor recorded GPU power, clock, temperature, GTT and MemAvailable, but no I/O counters. A dependable answer would require a separate run with block-layer statistics.

The real limit is the generation budget

The measurement ran with a context window of 134,400 positions; the model supports 262,144. That full window is workable on this machine too. KV memory costs about 28 KiB per position – the positions used take 3.7 GiB, the full 262,144 accordingly 7.2 GiB. With a pool of that size and the same generation budget of 49,152 tokens, the engine on the otherwise idle machine is ready after 18.9 seconds and answers the longest prompt in the task set, 68,801 tokens, at 1,005 tokens/s of prompt processing, leaving 76.3 GiB of memory free.

The binding constraint is a different one: HALOGEN_MAX_TOK not only caps how long an answer may become, it also sizes the prefill arena.

The bottleneck is not the amount of memory but its shape. The engine needs large contiguous blocks; when only small ones remain, it spends the start-up compacting. At a budget of 65,536 tokens, 184 contiguous 2 MiB blocks are left after start-up, after 637 compaction passes for the pool reservation alone, and a prefill of 60 tokens then takes 127 seconds instead of 2.3. The large KV pool pushes in the same direction, though less hard: with 262,144 positions, 275 blocks remain after 285 compaction passes.

For the measurement the choice therefore fell on 134,400 positions: that window covers the longest prompt in the set together with the full budget, and leaves the arena the most contiguous memory. Anyone who needs the full context can have it – and pays for it with a tighter start.

02 · And in comparison

Same machine, same tasks, the other engine

Success rate

17 : 18 of 20

Exact McNemar p = 1.000 – no measurable difference in capability.

Prompt processing

7.1 ×

126 against 886 tokens/s on a 69,000-token prompt.

Length of the run

486 → 157 min

The same twenty tasks, one after another, with nothing else running.

Reproducibility

0/18 : 9/9

Byte-identical answer pairs to the same question at temperature 0.

What follows below is this measurement in nine chapters: what was asked, what each of them solved, where the time goes, how they approach the tasks – and what the work does not show.

03 · What was measured

Twenty tasks, three kinds, two stacks

Twelve tasks from the AIME competitions of 2024 and 2025 – integer answers, checkable by machine. Four code tasks of our own, whose answers are executed against test cases in a rootless sandbox with no network. And four context tasks of our own over a measurement log of some 69,000 tokens that the prompt carries in full.

The self-made tasks are the contamination-free control on the public AIME sets: they were nowhere on the web when the model was trained. Both engines received exactly the same prompts in the same order, at temperature 0, with the prompt cache off, on the 140 W power supply in the performance profile. Nothing else ran on the machine during a run; the runner aborts if it finds a foreign process or a different power profile.

Click a column header to sort

One difference in the setup is worth naming, because it would otherwise look like an advantage: llama.cpp ran with a token budget of 65,536, halogen with 49,152 – for halogen that value is also the prefill arena and would have claimed some 33 GiB at 65,536. Scoring is therefore done on the shared budget of 49,152. That conversion costs llama.cpp one solved task and hands neither engine anything.

04 · Capability

In capability, nothing separates them

17 against 18 tasks solved out of twenty. Evaluated as paired data: exact McNemar, p = 1.000. With twenty tasks and 3 answered differently, a real difference of this size cannot be told apart from chance.

Choosing either engine because it gives better answers means choosing on grounds this measurement does not provide. That is not a weakness of the comparison but its most important result: it settles the question of capability and clears the way for the question that does show differences.

05 · Prompt processing

The lead grows with the length of the prompt

On short questions llama.cpp reads the prompt at 99 tokens/s and halogen at 129 – noticeable, but not dramatic. On the four context tasks of nearly 69,000 tokens it is 126 against 886 tokens/s: a factor of 7.1 in halogen's favour.

Why this matters is not in the number but in the way of working: a long prompt is the normal case as soon as a model is asked to read a file, a log or a conversation. Right there, llama.cpp takes a good seven minutes for what costs halogen one – before the first character of the answer appears.

06 · Output speed

One collapses on long context, the other does not

On short questions llama.cpp produces 21.1 tokens/s and halogen 39.3 – a good 1.9 × faster. With 69,000 tokens ahead of it, llama.cpp drops to 9.7 tokens/s, roughly half its own speed. halogen stays at 39.9.

The two together – slower to read in, then slower to write out – are why the run times differ in the end by a factor rather than by percentages.

07 · Why this counts

The difference is throughput, not verbosity

A faster run could simply be a shorter one. It is not: over the twenty tasks llama.cpp generated 362,635 tokens, halogen 339,134 – that is 6.5 % fewer for halogen.

Both therefore think for a comparable length about the same tasks. The difference in time does not come from one of them being more concise, but from how fast the same amount of text moves through the machine.

08 · Taken together

Seconds per solved task

Success rate and speed brought into one figure: 1,714 against 524 seconds per solved task. A factor of 3.3 – at the same success rate. Across the whole run: 486 against 157 minutes.

09 · The blind control

They do not just solve as often, they solve alike

A judge who did not know which engine had produced which text was given both complete solutions for each of the twenty tasks. The result: the same approach in 19 of 20 cases – the same substitution, the same theorem, the same algorithm.

Checking behaviour does not differ either: who recomputes, who verifies, who walks into a dead end and how they find their way out is distributed alike across both stacks. That supports the finding of chapter 04 from another side – not through the success rate but through the road to it.

10 · The unplanned finding

Same question, same answer – or else not

At temperature 0 the same request ought to yield the same answer. llama.cpp does so in not a single case: 0 of 18 answer pairs across 18 runs in two configurations were byte-identical. halogen in all of them: 9 of 9, across restarts of the engine.

On one task the answers to the same question ranged between 1,165 and 36,455 tokens. The obvious suspect – llama.cpp carrying the slot state over between requests – was ruled out by a second pass with cache_prompt: false. Only the first request after a server start matched, and that one byte for byte across two independent starts: the variance sits in the state carried between requests, not in sampling.

In practice this means two things. First: anyone who has to document an answer or bisect a regression needs reproducibility – and gets it here from only one of the two. Second, and less comfortable: the llama.cpp times in this measurement are a sample, not a constant. They should be read with a wide spread that is not quantified here.

11 · What this work does not show

Four limits to read along with it

• Contamination. AIME 2024 and 2025 are public and are probably in the training data. For comparing two quantisations of the same model that is tolerable – both share the same memory of them – but it is no basis for an absolute claim about capability. The self-made code and context tasks are the control.
• A notebook with a 70 W GPU budget is not a desktop. The absolute figures hold for this machine on this power supply.
• One pass per engine – and for llama.cpp that is demonstrably a sample, not a constant (chapter 10).
• Today's state is not the state that was measured. The rocmfpx fork has moved thirteen commits on since the day of the measurement, with an optimisation aimed at exactly the weak spot measured here; halogen stands at 0.5.0. Neither was re-measured – this page holds for the versions it names.