Long context on the HP ZBook Ultra G1a

To find out whether a model understands a long context, you have to change the length and hold the task still. So the questions always concern the same weather log: 651 lines of weather-station readings, about 32,000 tokens, with eight questions that each have one unambiguous answer. An unrelated maintenance journal before and after it brings the text to each length, with 957,999 tokens at 990k. If the result changes, it can only be the length.

Up to 245k three versions of the weather log with different random values ran per step, one beyond: 21 runs for understanding. For speed, all 54 scored runs of those days count, including those with other documents of the same length; how fast the machine reads and answers depends on the number of tokens, not on their content. Discarded are the warm-up run, the repeat of the same document and the comparison runs with YaRN below 262k that chapter 06 shows. Which run is missing and why is in the evaluation in the repository.

For work on a long document this means: you wait once, 3.3 minutes at 245k, and then keep asking in conversation. An agent that extends its history pays for reading only what is new.

Beyond 262k halogen ran with a generation budget of 16,384 instead of 32,768 tokens. That lowers the working memory from 12.5 to 8.0 GiB, which is why the pool for 396k holds less than the native one.

At 990k it gets tight. After startup the machine had 19.0 to 19.2 GiB left for everything else, and while reserving the pool the kernel already had to compact memory 378 to 6,192 times depending on the start; with the smaller pools not once. Next to a pool this size nothing large should run on the notebook.

Within the native limit, length is hardly an obstacle for this model. The two missing answers lie at 128k and 245k, not piled up at the top end: at 128k the model overlooks the location condition of the maximum in one version, at 245k it reads the peak wind from a wrong row in one version.

The pattern is not an even decline. The model names real values from the same log, only from the wrong row: at 396k and 594k the sensor ID of the neighbouring row, at 990k a peak wind of 23 km/h instead of 81. For the maximum it picks the reading point with the highest peak wind but overlooks that it lies at the wrong location. So it does not lose the text but the exact place in it.

Reading also gets more expensive beyond the limit: at 990k every token costs 26 percent more time than at 245k. Output is almost untouched by it.

Behind this are a block of four runs in alternating order on a long weather log of 4,993 lines, whose repeats give the same answers, and one run with the short weather log inside the maintenance journal. In the long log YaRN finds the unique combination that the model reports as absent natively, and measures the distance between two entries correctly. In the short one the same scorable tasks are right; only on the two arithmetic tasks do native and YaRN swap the correct answer.

Two documents are not enough to credit YaRN with an advantage. They are enough to conclude that YaRN does not cause the model to look up worse beyond 262k: at equal length it looks up at least as well with YaRN as without.