Qwen3.8-27B as an agent on the Mac Studio M1 Ultra

At 4 bits Qwen3.8-27B fits into the 64 GiB of the Mac Studio and leaves room for a long history. On the Mac six servers with an OpenAI-compatible interface run it: mlx-lm as Apple's reference, its offshoots Rapid-MLX, oMLX, MTPLX and mlx-dspark, and llama.cpp with Metal. Every engine ran with the weights it is built for and with its cache switched on the way one would use it day to day.

Four things are measured. First, a session: an agent history of 52,000 tokens growing turn by turn, interrupted by a side question, a second session and a server restart. Second, eight single agent steps from 2,300 to 98,000 tokens, each cold, for plain reading and output speed. Third, follow-up questions about a document of 32,000 to 128,000 tokens with checked answers. Fourth, whether the speculation that four of the engines advertise changes the output.

The agent histories are cut from the trace of a real coding agent (mini-swe-agent); the same set already ran on the HP ZBook Ultra G1a. All times are taken on the measuring script's clock, not from what the servers report. Before every start a script checked that no other server, download or build was running. All six handle tool calls: every engine solved the five tool tasks in three rounds without a formal error.

The reason lies in the model. Its DeltaNet layers carry a state that sums up the whole history and cannot be cut back to an earlier point. An engine can therefore reuse a stored history only if the new prompt continues it exactly, or if it kept snapshots of the state along the way. Agent steps continue the history; every engine manages that.

A side question or a second session puts another history in between. mlx-lm holds only one entry of this size and then reads for 4.7 min again. Only the engines that write the history to the SSD carry it across a restart: Rapid-MLX (1.2 s), oMLX (6.6 s) and MTPLX (5.9 s). llama.cpp keeps the history in RAM; mlx-dspark does write a disk cache but does not find it again after the restart.

Reading sets an agent's wait as soon as the cache loses the history. Output sets the time after that: a step with 1,000 tokens of thinking and a tool call takes about 27 seconds at 37.3 tokens per second and about 50 at 20.2. During the steps the chip drew 95 to 102 watts at 71 to 74 °C; the GPU clock stayed at 1,296 MHz on every engine.

At 32k all six engines solve all 24 tasks from three versions of the document. At 128k llama.cpp stays error-free, oMLX solves 22 of 24 and then writes only 10.5 tokens per second. MTPLX reads 128k in 19 minutes and then fails on memory errors: 3 of 24. mlx-dspark ran up to 64k; at 128k its memory guard emptied the cache and one follow-up took 16 minutes.

Re-checked with full text, MTPLX departs in the prose prompt from character 712 of the thinking, in agent task A03 from character 856; there it then writes a different answer and calls the shell with different commands. The result is not wrong, but it is not the model's own without a draft. mlx-dspark writes the same tokens with speculation and only appends accepted draft tokens at the limit of 512 or 1,024 tokens. Rapid-MLX, oMLX, llama.cpp and mlx-lm give byte-identical output in every run.

oMLX switches speculation only in its model settings, not at start; there it ran in both arms. llama.cpp and mlx-lm have none for this model.

For OpenCode with histories up to 52k, as far as measured: Rapid-MLX. It keeps the history across side questions, a second session in between and restarts, answers within 1.3 seconds, gives the same tokens with and without speculation and writes 31.5 tokens per second at 13k. The client's time-out still has to survive the first turn: a new 52k history takes 3.6 min here too.

For questions about documents beyond 64k: llama.cpp. It is the slowest writer, but the only one that answers at 128k at once and without error. Its cache does not survive a restart.

MTPLX writes fastest and keeps the history too, but changes the output with speculation, sends tool calls in a few large chunks and runs into its memory guard from 52k. oMLX keeps the history and reaches 98k, but aborts a tool call cut off at the token limit with an error. mlx-lm is measured as the reference; for an agent it loses the history too easily.