Jump Trading’s quant research with ChatGPT: What actually scales

A convincing research report can be an answer—or a new assignment for the person who has to check it. That is the practical tension behind Jump Trading’s quant research with ChatGPT: delegating a larger investigation is useful only if the result remains economical to evaluate.

In an October 6, 2026 customer story, OpenAI describes Jump researchers using GPT‑6 Astra for multi-day investigations spanning several data sources, with regular human check-ins and final human acceptance. It provides no quantified productivity gain, research-cost comparison, error rate or attributable trading-return improvement. This is customer testimony published by the vendor, not an independent measurement of the economics. (openai.com)

The useful question for other organisations is therefore how much expert effort these AI research workflows require to resolve a worthwhile question. That includes preparing the assignment, checking the evidence and repairing mistakes—not merely the work performed while the assistant appears to be busy.

What quant research with ChatGPT actually delegates

Quantitative research develops and tests data-driven hypotheses; at Jump, those concern markets. Lucas Baker, identified in the story as Jump’s Head of LLM R&D, describes researchers defining a problem, working environment and evaluation criteria, then steering agents through the investigation. The reported shift is from assistance with individual coding tasks towards responsibility for a connected sequence of analytical work. (openai.com)

Consider a hypothetical assignment: investigate whether a proposed relationship survives plausible alternative explanations. An assistant might inspect data definitions, implement a baseline, test competing explanations and document why an initially promising result disappeared. The deliverable would be an account of what the investigation established, rather than simply a functioning script.

In this kind of AI research workflow, the important delegation includes decisions between steps. Which discrepancy deserves investigation? Does a failed test undermine the hypothesis or expose a faulty implementation? When should exploration stop?

For quantitative research, that changes what the human must specify. The brief needs to distinguish choices the assistant may make from changes that would alter the question itself. Otherwise, a resourceful system could solve a more convenient problem and present the substitution as progress.

It also changes what good performance looks like. A well-supported negative finding can complete the assignment. A promising result with unresolved methodological problems may leave most of the consequential work untouched.

AI research workflows need more than access

The recommendations below are an editorial framework for testing delegated research, not a reconstruction of Jump’s internal controls.

OpenAI’s separate account of its Codex engineering experiment offers a useful starting point. Its engineers describe shifting effort towards working environments, accessible organisational knowledge and feedback loops. When agents struggled, the team looked for missing tools or structure rather than simply requesting another attempt. That experience concerns software engineering, but it suggests a practical lesson about the infrastructure surrounding a model. (openai.com)

For quantitative research, we would translate that lesson into an environment where authoritative datasets, current calculation methods and execution rules are discoverable. An assistant should not have to infer which definition of revenue the organisation means, or discover through repeated failure that a particular job cannot be restarted.

Source access is only the beginning. OpenAI’s general deep research documentation describes working with uploaded files, web sources and enabled ChatGPT apps. Those capabilities establish ways to retrieve information; they do not establish that a particular analysis has combined it correctly. (help.openai.com)

Imagine an investigation joining prices, company disclosures and an external dataset. Each source could be accurate in isolation while the combined analysis is misleading. Two systems might identify the same company differently. A document’s reporting period might be mistaken for its publication date. A revised figure might be attached to a date before the revision existed.

For AI research workflows, this makes the route from source to conclusion as important as the source list. The reviewer needs to see consequential joins, exclusions and timing assumptions. A bibliography is not a substitute for knowing what happened to the data.

Permissions require similar care. OpenAI’s API guidance warns that malicious instructions embedded in retrieved material can induce unintended actions, including sending private information outside the organisation. It recommends trusted connections and controls over outgoing tool calls. Research access should therefore be scoped separately from permission to transmit information or act on a finding. (developers.openai.com)

Cheaper experiments can produce more persuasive mistakes

Making investigation faster introduces a less comfortable possibility: the assistant becomes better at finding impressive historical results without becoming better at finding durable explanations.

In The Probability of Backtest Overfitting, David H. Bailey and co-authors examine how selecting investment strategies through historical simulations can produce misleading conclusions. Their analysis makes the search process important: the winning result cannot be assessed sensibly without considering the trials behind its selection. (davidhbailey.com)

The implication for AI-assisted quantitative research is that cheaper experimentation could increase the opportunity to select a flattering accident. That is an inference from the methodology, not a reported outcome at Jump.

Suppose an assistant explores numerous plausible variations and returns only the strongest-looking result. The reviewer receives one chart, although the process that produced it involved a much larger search. Without that history, the report hides information needed to assess its apparent strength.

AI research workflows should therefore preserve the alternatives explored, the selection procedure and the evidence against the preferred explanation. This is not an argument for discouraging exploration. It is a reason to stop treating the end of exploration as the beginning of a clean, previously specified test.

The distinction also matters outside trading. A marketing assistant asked to discover “what worked” across many customer segments should report the search behind its favourite pattern. The appropriate next step might be a new experiment, rather than an immediate budget change.

A successful rerun is not independent confirmation

Reproducibility is necessary for inspecting computational work, but it does not establish that the conclusion is valid. The National Academies distinguishes reproducing results with the same data and computational procedures from replication through studies obtaining their own data. The difference is substantive, not merely terminological. (nationalacademies.org)

Return to the hypothetical market study. Suppose it uses financial figures revised after the historical dates being analysed. Running the same code over the same file could reproduce the chart perfectly. It would also reproduce the use of information unavailable at the time.

A clean rerun establishes that the recorded procedure produces the recorded result. It does not establish that the procedure answers the intended question, or that the apparent relationship survives beyond the data used to discover it. Reproducible quantitative research can still be wrong. (nationalacademies.org)

For findings intended to support decisions, our recommendation is to specify validation appropriate to the claim. A separate implementation can help expose coding errors. Testing a fixed hypothesis against suitably independent evidence—such as appropriately time-separated data excluded from selection, or a prospectively designed study—addresses whether the finding survives beyond its discovery conditions. These checks serve different purposes; neither is a universal guarantee. (davidhbailey.com)

Independence also needs protecting during the investigation. Cynthia Dwork and co-authors’ work on adaptive data analysis shows how repeatedly using feedback from an ordinary holdout dataset can lead to overfitting that dataset itself. If an assistant keeps changing its approach after seeing test results, the nominally untouched test is becoming part of development. (arxiv.org)

AI research workflows should consequently distinguish an investigation completed to its agreed scope from a claim sufficiently validated for a specified decision. An exploratory finding may satisfy the first standard while awaiting the second. Accepting a research report should not silently promote its contents into operational authority.

Define the handover before the agent starts

A useful brief for quant research with ChatGPT should begin with the question, intended use and conditions under which the team would consider it answered. Specify whether the assignment is exploratory or decision-supporting, what evidence would count against the hypothesis, and the time and resource limits. Naming a stopping condition helps prevent persistence from becoming an expensive substitute for progress.

In an AI research workflow, the brief should also identify decisions requiring intervention: introducing a new dataset, changing a central assumption, expanding access or exceeding the budget. Give the assistant a legitimate way to report an obstruction and stop. Otherwise, the instruction to finish can conflict with the requirement to remain within scope.

The handover should contain source references, data versions, transformations, executable calculations, alternatives tested and unresolved limitations. A named reviewer should know which checks are required and where to find the evidence supporting each consequential claim. The objective is an inspectable investigation, not an obligation to read every intermediate message.

OpenAI’s agent-evaluation guidance makes a useful distinction between inspecting workflow traces—records of calls, tools and handoffs—and running repeatable evaluations against datasets. That supports examining both how the assistant behaved and how its outputs performed, although neither check alone validates a research conclusion. (developers.openai.com)

For quantitative research, this means a tidy execution record cannot rescue a poor study design. Equally, an interesting hypothesis does not excuse an implementation error. The handover needs to make both kinds of problem discoverable.

Measure the complete cost of an accepted investigation

A pilot of quant research with ChatGPT should compare bounded, similarly difficult assignments under a consistent review standard. Record differences in data availability and task complexity; where practical, have reviewers assess outputs without knowing which workflow produced them. Comparing unrelated tasks and calling the difference productivity would give the pilot its own methodological problem.

The following are proposed operating measures for AI research workflows, not reported Jump metrics.

Accepted-output rate should count investigations meeting the agreed criteria within the pilot’s limits, divided by all assigned investigations. Keep stopped and abandoned assignments in the denominator, and distinguish first-submission acceptance from acceptance after correction. A defensible negative finding can qualify; a promising result with missing evidence cannot. Exploratory assignments and decision-support assignments should be reported separately rather than treated as interchangeable successes.

Expert time should include preparation, active analysis, check-ins, review and correction across all assignments—not just successful ones. Report those stages separately, then divide their total by accepted investigations. Include relevant setup and maintenance effort rather than treating the working environment as a complimentary accessory.

Rework should record correction cycles, their causes and their human and machine costs. Keep repairs distinct from deliberate extensions of a valid investigation. Otherwise, AI research workflows can make error correction look like additional research output.

Escaped errors are material errors discovered after acceptance within a stated follow-up period. Record severity and consequences, and revise the accounting where an error invalidates an acceptance. No errors detected during a short pilot is a limited observation, not proof of error-free work.

A hypothetical pilot, with the arithmetic exposed

Consider a hypothetical quantitative research pilot, not Jump’s results. Each workflow receives ten comparable assignments; eight in each meet the same investigation-level acceptance standard. The hours below include effort spent on unsuccessful assignments.

The assisted workflow reduces hands-on analysis and execution time by 80%. Once preparation, review and rework are included, the reduction in expert time per accepted investigation is 40%: 120 ÷ 8 versus 72 ÷ 8.

That is still a meaningful saving in this illustration, but it is a different claim. If only four assisted investigations qualified, the same 72 hours would mean 18 hours each—worse than the conventional baseline despite much faster execution.

For AI research workflows, this calculation reveals where effort moves instead of allowing the most impressive stage-level improvement to represent the entire process. It also prevents an increase in reports produced from masquerading as an increase in questions resolved.

The example measures labour, not financial return. A real comparison should also record model, tool and compute costs, elapsed time, and the difference between initial setup and recurring operation. If translating hours into money, account for the expertise involved rather than assuming every hour has the same cost.

Finally, group quantitative research assignments by intended use and difficulty. Eight completed exploratory screens are not equivalent to eight independently validated findings ready to support a decision. A useful denominator still needs a useful definition of the work inside it.

Make more questions affordable

The strongest case for AI research workflows may be a question that previously cost too much to investigate properly. Perhaps the assistant establishes that an attractive idea fails under a straightforward alternative explanation, and records the failure well enough that the next researcher need not repeat the exercise.

That is valuable quantitative research even though no promising strategy survives. A pilot designed to reward positive findings would miss the benefit—and encourage precisely the behaviour its reviewers ought to distrust.

The ambition behind quant research with ChatGPT should be to make more worthwhile questions affordable while preserving the organisation’s ability to reject an attractive answer. More scope is justified when the resulting work can be checked without disproportionate effort and its conclusions withstand the tests their intended use requires.

The assistant can widen the search. It should not quietly move the finish line.