Long-context performance is one of the most frequently claimed advantages of current models and one of the hardest to evaluate honestly. A single impressive demo with a carefully placed fact does not tell you how the system behaves when the relevant detail sits in the middle of a fifty-page packet, when instructions compete with one another, or when the same workflow is repeated across several documents. I therefore built a repeatable long-context workflow and ran it under identical conditions on Muse Spark, GPT, and Claude.
The workflow mimics work I actually do for clients: ingest a multi-document packet, answer a set of decision-oriented questions, produce a structured brief, and flag anything that cannot be supported by the supplied material. The results below reflect that specific test, not a universal ranking.
Designing a Long-Context Test That Matches Real Work
Selecting the document packet
I assembled a packet of roughly 40–50 pages drawn from public filings, product documentation, and anonymized internal-style notes. The packet contained pricing tables, policy language, timelines, and a small number of deliberately placed conflicting statements. The material was long enough to stress context handling yet still realistic for a focused professional review.
Defining the workflow steps
The workflow had four stages: (1) extract answers to five pre-written questions with source anchors, (2) produce a structured one-page brief, (3) list any contradictions or silences, and (4) answer a set of follow-up probes that required combining information from non-adjacent sections. Each model received the identical packet and the identical instruction sequence.
Scoring criteria that matter in practice
I scored four dimensions: retrieval accuracy (did the answer match the source), instruction following (did the output respect format and constraint requests), summarization fidelity (did the brief preserve important qualifiers), and hallucination control (did the model invent details or resolve silence into false certainty). Speed and cost were noted but not scored.
Keeping conditions identical
All runs used the default web interfaces available on the test day, fresh conversations, and no custom system prompts. The packet was supplied in the same order and format. Follow-up probes were asked only after the main brief was complete. Results are a snapshot; model updates can change behavior.
Retrieval and Source-Anchor Performance
Locating facts placed in different packet regions
Facts placed near the beginning were recovered reliably by all three systems. Facts placed in the middle and final third produced more variation. Muse Spark and Claude both returned higher rates of correct source-anchored answers for mid-packet details in this particular packet. GPT occasionally paraphrased a nearby but incorrect passage.
Handling deliberately conflicting statements
When two sections contained conflicting figures or policy language, the stronger performances explicitly noted the conflict and quoted both sides. Weaker performances either chose one version without comment or produced a blended statement that appeared in neither source. Explicit conflict flagging is a practical advantage for decision support.
Respecting “silent or ambiguous” instructions
The prompt required the model to mark questions the packet did not answer. Models that followed this instruction cleanly avoided a common failure mode: filling gaps with plausible invention. In this test Claude and Muse Spark marked silence more consistently than the third system.
Quantitative snapshot of retrieval
Model | Correct Anchored Answers (out of 15) | Conflicts Explicitly Flagged | Silences Correctly Marked |
|---|---|---|---|
Muse Spark | 13 | 3 of 3 | 4 of 5 |
GPT | 11 | 2 of 3 | 3 of 5 |
Claude | 13 | 3 of 3 | 5 of 5 |
Instruction Following and Brief Quality
Adherence to format and length constraints
All three models produced the requested one-page brief structure. Muse Spark and Claude more consistently respected the length target and the required source-marker format. GPT’s briefs were occasionally longer and required more trimming to meet the same constraint.
Preservation of qualifiers and uncertainty
Summarization often erases hedges, conditions, and time bounds. The stronger performances in this test retained critical qualifiers (“subject to annual review,” “applies only to contracts signed after...”). Loss of qualifiers was the most common fidelity failure across all systems, but the rate differed.
Consistency across the multi-step workflow
After the initial brief, each model received the same follow-up probes. Models that had maintained cleaner source tracking in the first stage answered the probes with less drift. Models that had already smoothed over conflicts or silences tended to compound the earlier error.
Practical readability for a decision-maker
A brief that is technically accurate but difficult to scan still fails in real use. In this round the outputs from Muse Spark and Claude required fewer formatting edits before they could be handed to a non-technical reader.
Hallucination Control and Failure Modes
Invention of numbers and dates
Numerical hallucination remains a material risk. In this packet the models that performed dedicated number-extraction style reasoning inside the workflow introduced fewer invented figures. When an invented number did appear, it was usually a plausible variation of a real number located nearby in the text.
False resolution of ambiguity
The most dangerous failure mode is not an obvious fabrication but a confident statement that quietly resolves ambiguity the source left open. Prompts that explicitly reward “silent or ambiguous” answers reduced this behavior. Models that ignored the instruction produced smoother but less trustworthy briefs.
Drift across conversation turns
As the workflow moved from extraction to brief to probes, some models gradually lost earlier constraints. Restating the most important rules at each stage improved results for all three systems, confirming that long-context reliability is still partly a prompting discipline.
Summary of observed failure patterns
Failure Mode | Muse Spark | GPT | Claude |
|---|---|---|---|
Invented numerical detail | Low | Medium | Low |
Smoothed-over conflict | Low | Medium | Low |
Lost qualifier in summary | Medium | Medium | Low–Medium |
Drift on follow-up probes | Low–Medium | Medium | Low |
What Holds Up for Real Long-Context Work
Strengths that appeared consistently
All three models can usefully accelerate the first pass over a long packet when the questions are clear and source anchors are required. The combination of explicit silence marking and conflict flagging produced the most decision-ready output in this test.
Weaknesses that remain material
No model was immune to qualifier loss or occasional mid-packet misses. Cross-document reasoning that required tracking an entity across several non-adjacent sections remained a point of fragility. Human verification of every material number and obligation is still required.
Practical recommendations by use case
For workflows that emphasize explicit uncertainty and conflict surface-area, Claude and Muse Spark held up more cleanly in this packet. For users already invested in a particular interface or ecosystem, the differences may be smaller than the cost of switching. The only durable method is to re-run a personal reference packet after any major model update.
The continuing necessity of clean-run discipline
Long-context claims are easy to publish and hard to trust without controlled repetition. Keep a small set of representative packets and questions. Re-test when the model or the interface changes. Anything that cannot survive that discipline does not belong in a high-stakes workflow.
Long-context performance is real and useful within limits. In this particular multi-document workflow Muse Spark and Claude delivered stronger source anchoring, conflict flagging, and silence handling, while all three systems still required human verification of material details. The clean-run standard remains the only reliable filter between a promising demo and a process you can stake work on.
