Feedback Loops for Agentic Search

Hyacehila
  2026-06-06 12:00 2026-06-06 12:00 Created      

Shunyu Yao is here The Second Half One of the judges I always remember is that the second half of AI will move from problem solving to definition, and that the education and RL's generalization will be more important than training algorithms. I put this judgment on Agent, and I get a more specific systemic problem: define the problem and give the system an environment where it can always return a signal. Thus, Agent had the opportunity to solve new problems without training in parameters.

The questions in this article can also be addressedBehaviour Audit and Decoded Behaviour: From Reward to Agent Observation、From Black Box Forecast to Retroactive MedicineHow the concept of a relatively close read together is developed in different contexts.

The model is already very good at saying, writing, thinking and quickly offering candidates on many issues. But in the absence of feedback, these candidates can easily be stopped in “a reasonable explanation”. The problem is placed in a system that can be validated, compared and rolled back, and the generation slowly becomes a search.

The article was originally entered from AI for Science because the scientific scene was particularly sensitive to validation and feedback loops were more visible. And then I found out that the same set of questions also appeared in the coded task. The cosmological string is based on a scientific mission. tree search + verifier, TraeAgent uses coded tasks ensemble search + pruning + selectorI'm sorry. Two examples seem far away, and they're asking the same problem: how does the system manage candidates when they're growing up in space?

The role of the human being did not disappear, but changed location: defining problems, designing a certificationer, controlling the search space, cutting down the broken branch. Finally, one has to judge whether the result is meaningful or not.

The cosmos string is a place to see.

Solving an Open Problem in Theoretical Physics using AI-Assisted Discovery On the surface, a ball size fraction in the gravitational radiation spectrum of the cosmological string. I'm more concerned with its methodological component: the author has organized a theoretical physics issue into a searchable, verifiable, feedbackable Agent system.

They first set up a neuro-symbolic system, then put the model into the system. Gemini is responsible for generating mathematical assumptions, spreading paths and Python authentication codes; Trey search is responsible for organizational solvency space; high-precision numerical calculations are responsible for checking candidate formulae; errors, numerical errors and instability are then re-referenced to the context of the model. The process explored about 600 candidate nodes, and more than 80% of the branches were cut off because of algebra errors, numeric dissipation or catastrophic offsets.

This is probably the process:

LLM 生成候选推导
→ tree search 展开解法空间
→ Python verifier 执行和打分
→ traceback / error / instability 回注
→ 剪枝、修正、换路线
→ 继续搜索

This is a big difference from the normal chat-based model. In the ordinary chat, models produce an answer, read it, and then judge it. The system places the model output at the candidate node: the candidate is able to run, compare and be rebutted before handing it over to the verifier.

And it's interesting to have a new version of the story. When the system finds an effective route, the author expressly prohibits the model from continuing to use it and leads it to find another way. The final model gives six types of resolution: single-sizing, Gaussia lifening, Legendre spectrometry, Volterra push, Gegenbauer roll-out, etc. What is more useful to me is the search governance: the slow-down model is constricting to a single route while driving it away from the expression that has been used repeatedly.

The same way back in the encoded task

Trae Agent We moved similar problems into software engineering. It discusses the question of whether a code library and a natural language description are given, and the system generates a set of candidate patches from which to pick a final patch. The task is formally like a search of the formulae of scientific discovery, except for the conversion of the candidate objects from formula and extrapolation paths to a patch, and the conversion of the certifier from numerical score to test, to environmental and code understanding.

I'll write Trae's structure down in three steps. The two gates in the second step are horizontal, and it's easy to write the wrong thing:

Patch Generation：多样化生成候选 patch
→ Patch Pruning：deduplication 与 regression testing 平级剪枝
→ Patch Selection：repo-level understanding + majority voting

Diversification is at the forefront. Trae started with a coder anent to generate multiple pies in parallel instead of a bet on an Agent at a time. Diversity comes from high temperature sampling, multiple independent operations, and from multiple models of round-robin mixture. The judgment behind this design is simple: the same issue runs over and over, the overall pass rate may be similar, but each time it is solved in different subsets. There is indeed complementary information in the candidate space and what the follow-up system needs to do is to extract it.

The number of candidates also costs. When you increase your size, oracle rises, meaning that the theoretical potential is higher if you can pick the right patch. At the same time, the lower boundary of the election will decline because more errors, redundancy and interferences are mixed in the candidate. The more candidates, the longer the single-wheeled self-portrait sees, the more serious the selection itself will become.

Diversity generates a back-to-back governance of candidates, which will truly improve the chances of finding effective programmes.

Next is the catch running. It carries the filtering of the way, and the final choice is two separate layers of governance. The pruning in Trae consists of two lateral strategies:

patch deduplication：移除 redundant candidates
regression testing：移除 faulty / low-quality candidates

Dedupplication solves redundancy. Multiple patch can be different on the surface, actually modifying the equivalent; if it is thrown to self, it will only waste context windows and will also contaminate voting and comparison by repeating programs. Trae, first. unidiff Converts the candidate to a structured object, then compares at the structured object level, so that LLM does not judge the proximity of the two patches by the natural language only.

The solution is a non-feasible or low-quality candidate. Trae will run tests from the original code library, keep the testes that could have passed, and then choose a subset more like True Regrementations. Every candidate Patch then runs alone on these returns; failed patch is cut off and passes before entering the list. The verifiable feedback here is a filter that search midway.

And there's a very careful design: if all the candidates fail to pass the regreasing test, Trae will keep the whole set of options for the follow-up. Automatically selected results may not be allowed, and the cost of killing the correct Patch is too high. This retreat path recognizes that there may also be a mistake in the way that the cut leaves room for self-correction.

Watch selection relies on a repository-level program. Trae's self-elaborator ant will read the code clips, look at the files mentioned in the issue description, the files modified by the Patch and rely on the relevant codes; and generate and execute tests, collect descriptions, and spell out repository-level unzzy with static understanding and dynamic validation. The majority vote finally put a single pressure on instability in the country.

In complex tasks, cutting and final selection are introduced in two rounds. Pruning starts by keeping the candidate space low so that the system can search in depth. And then, selfctor, make more expensive and more detailed judgments and give the final answer.

What's the role of Feedback here?

feed-forward and feedback It is the two horizontal, complementary mechanisms of the Agent outer layer of naturals. The article focuses on the latter. Thoughtworks is in Harness engineering and agent feedback: Exploring AI coding sensors It divides the outer layer of the law into two categories. Skills, rules and reference documents provide knowledge and restraint before generation; test, static analysis and implementation results observe actual output. The two are not needed: the former tells the model what to do, and the latter tells it how it did.

I used to work on Prompt, RAG, Skills and context engineering, and I spent a lot of time studying how to get into models. I'm much less able to write about how the signal after execution returns, whether it's fast enough, whether it's wrong. This imbalance has a direct impact on the system: it may have incorporated much knowledge but it does not have the quality signals of the first step. The progress of coding anent and scientific discovery Agent reminds me that generating capacity and knowledge is only half of the system. After testing, implementing the environment and the verifier return the results to the loop, Agent can fix the course and continue to go down.

And when you combine the cosmological string with Trae Agent, feedback starts at the generation stage. The model is responsible for the submission of candidatures, the environment continues to operate, filters and returns signals, and finds that this is happening in the cycle.

There is implementation feedback when generated. In the cosmic string case, the Python authentication code that the model wrote exposed traceback, value dissipation, catastrophic offsets; Trae ' s coder anent also explored the cod library, re-emerged bug, generated a patch, rerun re-runs. Such feedback directly helps the model to modify the current course.

Feedback that supports the screening should be at least quick and partial. Every change is a few months of wet experiments, and Agent is hard to do with high frequency iterative; if there's only one "mistake" in feedback, it's hard to really get better in the next round. The tree search of the cosmological string cuts out the branches of algebra errors, values instability and error. Trae's watch pruning cuts through structural cuts, cuts through false candidates. The search system needs to know which candidate is better and which branch is worth continuing.

And then at the selection stage, the feedback became evidence. In the scientific scene, this may be numerical benchmarks, form validation, experimental results, documentary evidence and expert review; in the coding scene, it is static code understanding, dependency, test execution results and access tracks. They help self-identify which candidate is more worthy to stay.

Controlled search makes single-wheel capability a continuous improvement

Cosmic string papers, Trae Agent and several AI for Science systems that have emerged over the past few years are actually using similar skeletons.

FunSearch Rephrase the mathematical problem into a program search question: LLM Generation Candidate, evaluator Auto-execution and Rating, High Score Program into the database, and then be prompted to the next generation. Google DeepMind's official presentation sums it up as a combination of LLM creativity and automatted evaluator, filtering hallucinations and misperceptions.

AlphaEvolve It is similar, but more engineering. It is an evolutionary working agent generated and modified by Gemini, authenticating the answer by an automatic evaluator, and retaining a promising program with an evolutionary framework. It can handle algorithm detection and optimization, but users have to define the impact operation.

Co-Scientist Put this mechanism in the life sciences hypothesis generation. It creates multiple Agent systems by creating assumptions: General anent creates assumptions, Reflecting anticipants plays virtual people reviewer, Ranking ant organization ideaa explanation, Evolutation ant reorder and rewriting high-point assumption, Proximity ant controls diversity, Meta-review integrated feedback. In addition to numerical errors, the feedback here includes documentary evidence, hypothetical novelty, testability, virtual debate and sequencing.

The AI Scientist Approaching end-to-end automation, trying to cover the full process of idea, code, evaluation, paper writing and automatized review. It shows what an automatic research process can look like, and it pushes another question forward: when the system begins to write its own paper, the credibility of automatized reviewer becomes a bottleneck. If evaluator is not reliable enough, automatic science can easily produce something that is in the form of a paper but that is not sufficiently validated.

Robin The multiAgent system is then brought into the experimental biological process: literature search, hypothetical generation, experimental advice, data analysis, and regenerated updated assumptions. It pushes feedback from pure computing to lab-in-the-loop. The costs are straightforward: the feedback from the experiment is more expensive and every chance of experiment is saved.

The field is different, but the skeleton is similar:

候选生成
→ 搜索或排序
→ 外部工具执行
→ 验证/剪枝/集成
→ 反馈进入下一轮

LLM is responsible for presenting possibilities and the system is responsible for exerting selective pressure. Once this division of labour was established, Agent went from a one-off proponent to a system that continued to push candidates in a controlled environment.

tree search Reliance on several premises: relatively limited action space, state can be expressed, next action can be counted and candidate results can be compared with some feedback. These conditions are usually met for chess, program search and aggregation optimization, while natural language issues often lack clear boundaries. A problem can go in countless directions, each node continues to be a big piece, and it is difficult to compare between nodes, and failure is often not clearly located.

LLM brings a more subtle change: the natural language space is still large, but it is good at creating a workable candidate structure on a temporary basis. One question can be broken down into sub-issues, vague ideas can be written into hypothetical lists, reasoning lines can be organized into branches, experimental designs can be taken, mathematical extrapolations can be converted into enforceable codes, software modifications can be turned into patch, and the quality of answers can be broken down into a few evaluation dimensions. There is a continuous, loose and inexcusable language space, where, at least in part, there is a shape that can be expanded, selected and reversible. The search is where it is done.

Feedback-driven Agent's common structure

If a moving structure were drawn from these systems, I would now write as follows:

Problem Interface
→ Candidate Generator
→ Diversity / Sampling Controller
→ Pruning Gates
→ Domain Runtime / Tools
→ Verifier / Evaluator
→ Selector / Aggregator
→ Feedback Memory
→ Human Owner

Problem Interface Responsible for the issue as input, output, discipline, resource budgeting and success criteria. Encoding tasks also include description of the issue, the status of the code repository, the test environment, the permissible scope of modifications and the acceptance criteria. FunSearch needs question descriptions, food program and evaluator; AlphaEvolve needs initial program and evaluation function; cosmic string papers crush physical problems I(N, α) ; Trae writes the software issue resolution as a check that can be selected from the pool of candidates to pass gold testes and meet the issue requirements.

Candidate Generator LLM is the best position: to propose formulas, programs, experiments, proof routes, mechanism assumptions, and to generate multiple patch, test and remodel paths. It is located as a high-throughput candidate and is ultimately awarded to the later certification and selection layers.

Diversity / Sampling Controller (c) Regulatory coverage. The negative prompting in the cosmological string paper will prevent models from repeating existing routes; Trae expands the pixture diversity with high temperature sampling, independent operation and multimodel mix; and Proximity angent control assumptions are also contained in Co-Scientist. There is enough diversity to get the search out of the repeated rewrite of the same answer.

Pruning Gates Put the candidate space back on the optional range. Trae gave a clear look here: Dedication and regrementing are all proliferating, but they're horizontal gates. One to handle redundancy, one to handle impracticable or low scores. In other areas, dedup could be molecular structure equivalents, proof of route equivalents, retrieving evidence; testing could be a simulation of failure, breach of restraint, lack of statistical visibility, and quotations that do not support the conclusion.

Domain Runtime / Tools It's the interface that Agent has to contact the outside world. Math is the numerical fraction, symbol computation, theorem library; biobiology is the literature database, sequence analysis, experimental data processing; materials are DFT, structural stability prediction, synthetic feasibility check; software is the test set, benchmark, profiling, type systems and real code implementation environment.

Verifier / Evaluator The candidate is pulled back from the narrative to external constraints and ideally independent of the model self-assessment. Numerical error, cross-references, experimental readings, simulation results, statistical visibility, type checks, unit tests, Lean/Coq certification, expert review can all provide feedback. There are boundaries, too. Trae’s conservative strategy for regreating reminds us that the system must keep the regression path as long as there is a risk of accidental killing. What every discipline and problem needs Verifier It's different.

Selector / Aggregator Stay to the end, and pruning each with a division of labour. Cutting is the responsibility of reducing the pool of candidates, and the final answer is chosen. Simple tasks can be given -- and -- rerank, and complex tasks require more expensive self-portor. Trae's self-elaborator anent to create repository-level underwritering and reduce single-judgement instability by majority vote; Co-Scientist uses raning history and meta-review to make similar aggregates.

Feedback Memory To remember failure. What is unstable, what is useful, what is assumed to be killed in reverse, which patch types often undermine compatibility, and which tests are easily miscalculated should be part of the system memory. Otherwise, many failures will recur in different rounds.

Human Owner Responsible for value judgement and ultimate responsibility. Humans need to judge whether the value of the problem is not worth doing, whether the certification machine is measuring the real target, and whether the result is simply drilling the space of the benchmark. Finally, it is important that the system be transformed into something that the community or engineering team can review, repeat and deliver.

Which questions fit in this way?

Such systems are more appropriate for “unsolvable, easily verifiable” issues and do not cover all scientific or engineering issues. In the paper on Nature by FunSearch, it was also emphasized that many mathematics and computer science issues, although difficult to solve, were easily assessed in quality. So is the cosmological string fraction: it's hard to find, but it's given. N and α , the reference may be based on high-precision numerical fractions. The coding tasks are similar: real job fixes are difficult to write at a time, but the check is possible to compile, test, destroy or not, at least in part, destroy existing behavior.

The question of fit for Agent usually has several common features. The task can be broken down into intermediate steps, and every step can be checked, avoiding dragging the whole thing to the end. Candidates are also clear, whether it is a procedure, formula, molecule, material structure, experimental programme, causal chart, draft certification, parameterized model or a watch. Early screening is best served by cheap local feedback, leaving expensive experiments and manual review to fewer candidates. The signal of failure is also clear, such as dispersal, counter-censorship, test failure, physical restraint violation, ineffectiveness of the experiment, statistical invisibility, failure. These signals are much more useful than "not looking good."

Such questions must also be allowed to be searched. The candidate space is large enough to be relevant for modeling; it also needs boundaries to avoid a random attempt at degradation. Trae's experience reminds me in particular that with the expansion of candidate space, the selection difficulties will rise simultaneously. The search system is based on the generation of the upper world and the cost of the selection.

Conversely, if a problem is not reliable, the feedback is slow and the criteria for success are vague, it is not appropriate to call it automatic discovery or automatic resolution prematurely, but rather a lengthy explanation. In this scenario, AI can still document, hypothetically inspire, draft codes and write aids, but it cannot yet be considered a closed-ring search system.

Do not mix all AI systems into Agent

Now there's a lot of research on AI for Science and on the subject, and these systems are actually different types.

AlphaFold 3 and GNoME All truly useful scientific AI systems. The former predicts the structure of biomolecule compounds, while the latter discover space using map network extension materials. They show that AI can significantly reduce candidate space and change the flow of scientific work. But they are more strictly like powerful predictive models or discovery tools than the type of Agent system that is discussed here as “generation of candidates, execution of searches, absorption of feedback, iterative action”.

The same thing in software engineering. The generation of codes is only the starting point. A stable coding agent usually includes the search for a code library, the call of tools, the testing of execution, the recovery of errors, the recording of tracks and the final selection. Trae is not only worth Pass@1 numbers, but rather it organizes repositiry-level issue resolution into three separate and interlocking phases.

The determination of whether a system is a feedback-driving anticipatory search can be seen in five things: whether candidates are diversified, whether they are structured, whether there are verifiable cutlets, whether there are independent self-selectors, and whether there are people who can review and take over. The absence of such a feedback loop system may still be useful, but it is of another type.

The way to take it away.

I'll ask a few questions and then model them.

Candidate objects are defined first: formulae, procedures, mechanism assumptions, experimental schemes, molecular structure, material formulation, a browser operation, a piece of search results, or a watch. The clearer the candidate, the more heavier, the more authentic and the more selective the candidate is.

Let the model generate multiple routes at a time. High temperature sampling, multiple independent operations, multi-modeling, negativity propling, proximity control can be used to widen candidate differences.

Cuts can have multiple gates, not necessarily a relationship, but feedback is reliable. The description and review test in Trae is a lateral strategy: one removes the radicalities, one removes the falseities. Science missions can also be accompanied by multiple cuttings such as numerical errors, physical constraints, simulation results, statistical tests, documentary evidence, etc.

After cutting, you still need a sector. This self-portor can be a ranger, majority vending, repo-level expert ant, expert review, or combinations of different ways. Through local verifeer, only one gate was crossed, and more evidence was needed for the end.

And control the restemble size. The more candidates there are, the higher the upper world, but the costs of context, execution, weighting, testing and selection will rise. Test-time scaling requires a synchronized extension of sampling, cutting and selection, and failure to keep up with any level will be a new bottleneck.

I'll still go back to the beginning. Agent's ability to keep getting stronger depends to a large extent on whether we can turn the problem into a searchable, verifiable, cutable, optional feedback environment. Scientific discoveries and coding missions are good examples.

LLM is better suited to generate candidates. Test and verifeer first removes obvious errors, the search organizational candidate space, running the size of the candidate, and then handing the rest of the judgement over to self and human owner. The faster the model is generated, the easier it is to verify and select it as a bottleneck.

At the end of the game, AI could think of something that didn't bother me that much. I wonder more if we can give it a good feedback world, where new things are screened and refuted and eventually become credible knowledge and deliverable software.

References

  • Title: Feedback Loops for Agentic Search
  • Author: Hyacehila
  • Created at : 2026-06-06 12:00:00
  • Link: https://hyacehila.github.io//blog/2026/06/06/feedback-driven-agentic-scientific-discovery/
  • License: This work is licensed under CC BY-NC-SA 4.0.
Comments
On this page
Feedback Loops for Agentic Search
  1. The cosmos string is a place to see.
  2. The same way back in the encoded task
  3. What's the role of Feedback here?
  4. Controlled search makes single-wheel capability a continuous improvement
  5. Feedback-driven Agent's common structure
  6. Which questions fit in this way?
  7. Do not mix all AI systems into Agent
  8. The way to take it away.
  9. References