holistechlabshalogen · 32k → 990k
DEEN

Measured · 21 to 23 September 2026 · 54 runs

Long context on the HP ZBook Ultra G1a

halogen-flash-server 0.12.1 and 0.13.1 from 32,000 to 990,000 tokens

Authors

Dipl.-Ing. Sören GebbertInstitut für holistische Technologieforschung GmbH Claude Code Opus 5.5Anthropic

Machine

HP ZBook Ultra G1a

SoC
Ryzen AI Max+ PRO 395
GPU
Radeon 8060S, gfx1151
Memory
128 GiB LPDDR5X-8000, soldered – shared by CPU and GPU as unified memory
SSD
Samsung 990 PRO, 4 TB, NVMe on PCIe 4.0 x4
OS
Ubuntu 24.04, 140 W power supply, platform_profile=performance

Model

Qwen3.8-Flash-Next

Size
about 180B – measured at 176.9 billion parameters
Build
“512×56B”, 48 blocks, 512 experts, 10 of them active per token
Context
native up to 262,144 tokens, beyond that YaRN with factor 4

Engine

halogen-flash-server

Version
image 0.12.1 and 0.13.1, Podman – byte-identical tokens
Weights
w4b.hgn + quality overlay, 5.53 bpw
Flags
HALOGEN_CTX per length up to 1,048,576, HALOGEN_ROPE_YARN=4 beyond 262k
Budget
3,072 output tokens per task, temperature 0, a fresh container per run

Tasks

Eight questions about a weather log

Text
651 lines of weather-station readings, the same at every length
Length
padded with an unrelated maintenance journal
Kind
look up, compare, count, sum
Scored
six; the two arithmetic tasks test sums, not context
32k Context32,000 tokens in the prompt

1,194tokens/s

Prompt processing on the first question about a text

27seconds

Reading the whole text once, on the first question

1.8seconds

Reading a follow-up the text itself is already processed

47.2tokens/s

Output when answering follow-up questions

100.1GiB

Memory held of 125.1 GiB; left afterwards: 13.2 GiB

990k Context990,000 tokens in the prompt

768tokens/s

Prompt processing on the first question about a text

21.5minutes

Reading the whole text once, on the first question

2.3seconds

Reading a follow-up the text itself is already processed

40.9tokens/s

Output when answering follow-up questions

109.3GiB

Memory held of 125.1 GiB; left afterwards: 4.1 GiB

Medians over all runs at the given length. On the first question about a text the machine reads all of it. On every follow-up question it is already processed, and only the new question is added. Chapter 02 shows the lengths in between.

01 · Setup

The same task at every length

To find out whether a model understands a long context, you have to change the length and hold the task still. So the questions always concern the same weather log: 651 lines of weather-station readings, about 32,000 tokens, with eight questions that each have one unambiguous answer. An unrelated maintenance journal before and after it brings the text to each length, with 957,999 tokens at 990k. If the result changes, it can only be the length.

Up to 245k three versions of the weather log with different random values ran per step, one beyond: 21 runs for understanding. For speed, all 54 scored runs of those days count, including those with other documents of the same length; how fast the machine reads and answers depends on the number of tokens, not on their content. Discarded are warm-up runs, repeats of the same document and pilot runs with other settings. Which run is missing and why is in the evaluation in the repository.

02 · Speed

Read once, then it answers in seconds

On the first question about a text the machine reads 1,194 tokens per second at 32k and 768 at 990k. Every follow-up on the same text then takes at most 2.3 seconds, at every length.

By 990k, reading loses just over a third of its speed Tokens read per second on the first question about a text, one dot per run. HP ZBook Ultra G1a · Ryzen AI Max+ PRO 395 · Qwen3.8-Flash-Next · halogen 0.12.1 and 0.13.1 temperature 0 · eight questions with one clear answer · native up to 262,144 tokens, YaRN 4 beyond · measured 21 to 23 Sept 2026 32k 64k 128k 256k 512k 1M beyond the native limit: YaRN 4 native limit Length of the prompt in tokens (logarithmic) 0 250 500 750 1,000 1,250 Tokens per second 1,194 1,120 1,084 1,109 1,076 1,069 1,037 1,026 996 1,044 1,000 1,037 891 837 768
Reading loses 13 percent of its speed up to the native limit and 36 by 990k. Every dot is one run, the line the median per step.
Wait once, then it answers in seconds Reading time on the first question and on every follow-up, logarithmic. HP ZBook Ultra G1a · Ryzen AI Max+ PRO 395 · Qwen3.8-Flash-Next · halogen 0.12.1 and 0.13.1 temperature 0 · eight questions with one clear answer · native up to 262,144 tokens, YaRN 4 beyond · measured 21 to 23 Sept 2026 follow-up questions first question 32k 64k 128k 256k 512k 1M beyond the native limit: YaRN 4 native limit Length of the prompt in tokens (logarithmic) 1 s 10 s 1 min 5 min 20 min Reading time 27.0 s 57.3 s 1.2 min 1.4 min 1.7 min 2.0 min 2.3 min 2.6 min 3.0 min 3.1 min 3.8 min 3.9 min 7.4 min 11.8 min 21.5 min 1.8 s 1.7 s 1.8 s 2.0 s 1.5 s 1.9 s 1.9 s 2.0 s 1.9 s 2.1 s 1.9 s 1.8 s 1.8 s 2.1 s 2.3 s
The wait comes with the first question. At 990k reading takes 21.5 minutes; after that the text is processed, and every further question costs only its own tokens.
Generation stays almost as fast Tokens generated per second when answering follow-up questions, median per run. HP ZBook Ultra G1a · Ryzen AI Max+ PRO 395 · Qwen3.8-Flash-Next · halogen 0.12.1 and 0.13.1 temperature 0 · eight questions with one clear answer · native up to 262,144 tokens, YaRN 4 beyond · measured 21 to 23 Sept 2026 32k 64k 128k 256k 512k 1M beyond the native limit: YaRN 4 native limit Length of the prompt in tokens (logarithmic) 0 10 20 30 40 50 60 Tokens per second 47.2 45.6 40.9 43.6 45.5 44.4 36.5 37.0 43.3 42.3 44.1 41.9 44.1 42.5 40.9
Output barely depends on the length. From 47.2 to 40.9 tokens per second at thirty times the context. Single answers stalled by the kernel's memory compaction are left out; they change no token.

For work on a long document this means: you wait once, 3.9 minutes at 245k, and then keep asking in conversation. An agent that extends its history pays for reading only what is new.

03 · Memory

The pool sets the memory, not the prompt

At startup halogen reserves a pool for a fixed number of positions and holds all of it, however long the text turns out to be. Up to 262k every run used the same pool: 100.1 GiB, whether the prompt had 32,000 or 245,000 tokens. For 990k it is 109.3 GiB of 125.1 GiB.

The pool sets the memory, not the prompt What halogen holds after starting, per configuration; the engine reports these figures itself at startup. HP ZBook Ultra G1a · Ryzen AI Max+ PRO 395 · Qwen3.8-Flash-Next · halogen 0.12.1 and 0.13.1 temperature 0 · eight questions with one clear answer · native up to 262,144 tokens, YaRN 4 beyond · measured 21 to 23 Sept 2026 working memory KV pool weights 0 20 40 60 80 100 120 gigabytes (GiB) machine: 125.1 GiB 32k to 245k pool 393,216 positions 68.0 10.8 21.4 100.1 GiB left afterwards: 13.2–13.7 GiB 396k pool 425,984 positions 68.0 11.7 12.5 92.1 GiB left afterwards: 21.1–21.7 GiB 594k pool 622,592 positions 68.0 17.1 12.5 97.6 GiB left afterwards: 15.3–16.3 GiB 990k pool 1,048,576 positions 68.0 28.8 12.5 109.3 GiB left afterwards: 4.1–5.6 GiB
The weights never change, the KV pool grows with the positions. 10.8 GiB for 393,216 positions, 28.8 GiB for 1,048,576. The engine reports these figures itself at startup, here from 15 starts of dedicated measurement containers.

Beyond 262k halogen ran with a generation budget of 16,384 instead of 32,768 tokens. That lowers the working memory from 21.4 to 12.5 GiB, which is why the pool for 396k holds less than the native one.

At 990k it gets tight. After startup the machine had 4.1 to 5.6 GiB left for everything else, and while reserving the pool the kernel already had to compact memory 534 times; with the smaller pools not once. Next to a pool this size nothing large should run on the notebook.

04 · Understanding

Understanding holds up to 262k and slips beyond

Up to 245k the model solves 106 of 108 scorable tasks.

Understanding holds up to 262k and slips beyond Share of scorable tasks solved correctly; three versions of the weather log up to 245k, one beyond. HP ZBook Ultra G1a · Ryzen AI Max+ PRO 395 · Qwen3.8-Flash-Next · halogen 0.12.1 and 0.13.1 temperature 0 · eight questions with one clear answer · native up to 262,144 tokens, YaRN 4 beyond · measured 21 to 23 Sept 2026 32k 64k 128k 256k 512k 1M beyond the native limit: YaRN 4 native limit Length of the prompt in tokens (logarithmic) 0 % 25 % 50 % 75 % 100 % Share correct 100 % 94 % 100 % 100 % 100 % 94 % 83 % 67 % 67 %
Up to the native limit the line stays up. Only beyond 262k, with YaRN, does it drop.

Within the native limit, length is no obstacle for this model. The first version of the weather log solves all eight tasks at every step up to 245k, the two arithmetic tasks included; across all 18 runs 2 answers are missing, spread over 64k and 245k, not piled up at the top end.

05 · Beyond 262k

The first thing to fail is the look-up

Beyond the native limit it is the length. The same weather log gives 6 of 8 correct answers at 396k, 4 of 8 at 594k and 5 of 8 at 990k.

The first thing to fail is the look-up The same weather log at every length: filled dot correct, open one wrong. HP ZBook Ultra G1a · Ryzen AI Max+ PRO 395 · Qwen3.8-Flash-Next · halogen 0.12.1 and 0.13.1 temperature 0 · eight questions with one clear answer · native up to 262,144 tokens, YaRN 4 beyond · measured 21 to 23 Sept 2026 32k 64k 96k 128k 192k 245k 396k 594k 990k Read a value from a named row (wind) Read a value from a named row (sensor) Distance between two far-apart entries Tell a near-duplicate apart Find a unique combination Find the maximum of a subset Count a subset Sum a subset
The harder tasks hold out longer. Finding a combination, determining a maximum, spotting a near-duplicate: correct at every length. Reading the value of a named row fails from 396k.

The pattern is not an even decline. The model names a real sensor ID from the same log, only from the wrong row; at 245k it still picks that row reliably. So it does not lose the text but the exact place in it.

Reading also gets more expensive beyond the limit: at 990k every token costs 35 percent more time than at 245k. Output is almost untouched by it.

06 · YaRN

At equal length, YaRN does not change the result

Beyond 262,144 tokens the model only gets there with YaRN, a stretch of the position encoding. The suspicion that it causes the drop is obvious. The measurement refutes it.

At equal length, YaRN does not change the result Each pair the same 245,000-token document, once native and once with YaRN 4. HP ZBook Ultra G1a · Ryzen AI Max+ PRO 395 · Qwen3.8-Flash-Next · halogen 0.12.1 and 0.13.1 temperature 0 · eight questions with one clear answer · native up to 262,144 tokens, YaRN 4 beyond · measured 21 to 23 Sept 2026 YaRN 4 native 0 50 100 150 200 250 300 Reading on the first question, in seconds long weather log, run 1 238 s · 4 / 8 correct 246 s · 4 / 8 correct long weather log, run 2 252 s · 4 / 8 correct 252 s · 4 / 8 correct short log in the maintenance journal 230 s · 8 / 8 correct 236 s · 8 / 8 correct
The same tasks correct in all three pairs. Reading takes at most 3 percent longer with YaRN.

Two independent cross-checks stand behind this: a block of four runs in alternating order on a long weather log of 4,993 lines, and one run with the short weather log inside the maintenance journal that joins the native series exactly. The wording of the answers changes with YaRN, the solution does not.

YaRN is what makes lengths beyond 262k possible, but it does not cause the model to look up worse there. That is the length itself.

07 · Every figure

Results per length step

Understanding: the weather log inside the maintenance journal, the six scorable tasks across all runs of the step. Speed: medians across all scored runs of the step. Every chart in this report is computed from these values; the values per run are in docs/kontext-werte.json.

Understanding
Length Scaling Runs correct
32knative318 / 18
64knative317 / 18
96knative318 / 18
128knative318 / 18
192knative318 / 18
245knative317 / 18
396kYaRN 415 / 6
594kYaRN 414 / 6
990kYaRN 414 / 6
Speed
Length Runs Reading (s) Reading (t/s) Output (t/s) Follow-up (s)
32k427.01,19447.21.8
64k657.31,12045.61.7
80k373.91,08440.91.8
96k686.71,10943.62.0
112k3104.21,07645.51.5
128k6119.91,06944.41.9
144k3138.91,03736.51.9
160k3156.01,02637.02.0
176k3177.599643.31.8
192k6184.11,04442.32.1
229k1229.11,00044.11.9
245k4236.41,03741.91.8
396k2444.989144.11.8
594k2709.683742.52.1
990k21,289.076840.92.3
Memory
Pool Budget Starts Weights KV-Pool Working total left
393,21632,768968.010.821.4100.113.2–13.7
425,98416,384268.011.712.592.121.1–21.7
622,59216,384268.017.112.597.615.3–16.3
1,048,57616,384268.028.812.5109.34.1–5.6

08 · Limits

What this work does not show

One version beyond 262k. Above the native limit one version of the weather log ran per length. At temperature 0 the same document repeats the same answer; the spread lies between documents and is not measured there. That 990k solves one task more than 594k belongs to this. What holds is the gap: the same version is error-free up to 245k and no longer from 396k.

One pool for every length up to 262k. Up to 262k every run used the everyday pool of 393,216 positions. How much memory a smaller pool would save for short texts is not measured.

The smaller arena. From 396k halogen works with a generation budget of 16,384 instead of 32,768 tokens. That changes together with the length and is not measured separately.

One kind of document. The tasks ask about a tabular measurement log. For source code, prose or conversations the limit may lie elsewhere.

One engine, one machine. What is measured is halogen 0.12.1 and 0.13.1 on the HP ZBook Ultra G1a. The speed holds for this stack; whether llama.cpp understands the same at the same lengths is not measured here.