Long context on the HP ZBook Ultra G1a

To find out whether a model understands a long context, you have to change the length and hold the task still. So the questions always concern the same weather log: 651 lines of weather-station readings, about 32,000 tokens, with eight questions that each have one unambiguous answer. An unrelated maintenance journal before and after it brings the text to each length, with 957,999 tokens at 990k. If the result changes, it can only be the length.

Up to 245k three versions of the weather log with different random values ran per step, one beyond: 21 runs for understanding. For speed, all 54 scored runs of those days count, including those with other documents of the same length; how fast the machine reads and answers depends on the number of tokens, not on their content. Discarded are the warm-up run, the repeat of the same document and the comparison runs with YaRN below 262k that chapter 06 shows. Which run is missing and why is in the evaluation in the repository.

For work on a long document this means: you wait once, 3.4 minutes at 245k, and then keep asking in conversation. An agent that extends its history pays for reading only what is new.

Beyond 262k halogen ran with a generation budget of 16,384 instead of 32,768 tokens. That lowers the working memory from 12.6 to 8.1 GiB, which is why the pool for 396k holds less than the native one.

At 990k it gets tight. After startup the machine had 18.3 to 18.6 GiB left for everything else, and while reserving the pool the kernel already had to compact memory 342 to 3,499 times depending on the start; with the smaller pools not once. Next to a pool this size nothing large should run on the notebook.

Up to 192k, length is no obstacle for this model: not a single wrong scorable answer across all runs up to there. The 3 missing answers all lie at 245k, at the top of the native series, spread over 2 of the 3 versions of the weather log.

The pattern is not an even decline. The model names a real value from the same log, only from the wrong row: at 245k and at 990k a peak wind of 23 km/h instead of 81, at 594k the sensor ID of the neighbouring row. At 396k it reads both values correctly. So it does not lose the text but the exact place in it.

Reading also gets more expensive beyond the limit: at 990k every token costs 28 percent more time than at 245k. Output is almost untouched by it.

Behind this are a block of four runs in alternating order on a long weather log of 4,993 lines, whose repeats give the same answers, and one run with the short weather log inside the maintenance journal. In the long log YaRN finds the unique combination that the model reports as absent natively. In the short one YaRN reads the peak wind correctly, which natively comes from the wrong row, and in turn miscounts a subset that is right natively.

Two documents are not enough to credit YaRN with an advantage. They are enough to conclude that YaRN does not cause the model to look up worse beyond 262k: at equal length it looks up at least as well with YaRN as without.