Qwen3.8-Flash-Next as a coding agent on the HP ZBook Ultra G1a

The agent gets a repository at the state before the fix and the issue verbatim, plus one sentence: resolve the issue, you may run commands and the existing tests, stop when you are done. It does not get the tests used for scoring. Dependencies are installed; only the model server is reachable.

The tasks come from SWE-bench Verified, from its easiest level: issues that the dataset rates at under 15 minutes for an experienced developer. Eligible are the 87 tasks of this level from Django, sympy, Sphinx and pytest that run under Python 3.9 to 3.11. 40 of them were drawn at random, stratified by repository, with a fixed seed. Nothing was selected by solvability. One Sphinx task could not be checked in the scoring environment and was replaced by the next one drawn.

Scoring follows SWE-bench: the patch the agent leaves in the working tree is applied to a fresh checkout together with the task's tests. A task is solved when the tests for the bug pass (FAIL_TO_PASS) and no previously passing test fails (PASS_TO_PASS). Scoring runs without Docker in a Python environment following the SWE-bench harness, version 4.1.0. Each of the 40 tasks passed there with the reference fix beforehand.

Settings: context up to 262,144 tokens, output up to 32,768 tokens per request, temperature 1.0, top_p 0.95, reasoning effort xhigh, automatic compaction on, prompt cache in RAM. First all 40 tasks run once, then all of them a second time, in the same order, with one server start for all passes. Nothing else runs.

The agent solves 72 of 80 passes, that is 90.0 %. The 95 % range runs from 80.0 to 97.5 %; it is drawn over tasks, not passes, because two passes of the same task are not independent. The rate holds for the 87 tasks the draw came from, not for SWE-bench Verified as a whole.

The two passes of a task always end the same way: the agent solves 36 tasks both times, 4 neither time, none just once.

G01 (django__django-15127): the agent ends both passes after 1.8 and 2.1 minutes with a patch under which all existing tests pass, but not the test for the bug.

G18 (sympy__sympy-15875): the same picture after 27.8 and 39.8 minutes and 68 and 90 steps.

M08 (sphinx-doc__sphinx-7462): of the two tests for the bug, none passes in the first pass and one in the second.

M04 (sphinx-doc__sphinx-8595): in both passes the agent creates a test file, tests/roots/test-ext-autodoc/target/empty_all.py, which the task's test patch also creates. The test patch therefore cannot be applied, and the pass counts as not solved.

Half of the passes finish within 4.6 minutes. Django and sympy take a median of 4.2, Sphinx and pytest 5.9 minutes. Some take much longer: the three longest needed 120.0, 59.7 and 39.8 minutes. All 80 passes ran in one go without intervention, in 13.0 hours including scoring.

Every agent step is one request to the server: it reads the new part of the history, usually the output of the last tool, and writes thinking and the next tool call. The passes themselves took 12.8 hours together. Of these, 49 minutes go to reading, 403 to output and 313 to tools, mainly the tests the agent runs itself, and to OpenCode. Server speed hardly changes as the context grows.

A pass needs a median of 29 steps. The deepest session reached 140,268 tokens, just over half of the 262,144-token context window. Of 2,992 tool calls, 19 failed.

One pass ended at the time limit: the first pass of G20, after 120 minutes and only 49 steps. The patch the agent had written by then passes all tests and therefore counts as solved, as in SWE-bench.

No pass established a connection other than to the model server. The operating system's sandbox blocks every other destination; a network monitor checks this for every pass.