The Evolution of LLM Tool Use: From Toolformer to ToolLLM

This English version was machine-translated from the Chinese original. Technical terms may require verification.

Why do you care about the tools?

The questions in this article can also be addressedMCP (Model Context Protocol)、Make it work, big model structured output and restricted decoding techniquesHow the concept of a relatively close read together is developed in different contexts.

The Big Language model is very strong in a number of classic NLP fields, but still unstable in the context of arithmetical questions and factual answers -- - The model is not able to update internal parameters in a timely manner and there are hallucinating problems. LLM is allowed to use external tools that allow it to access real-time, accurate knowledge banks and perform computing tasks.

About Tool Use,Perhaps the MCP protocol is of greatest interest to the engineering communityI'm sorry. MCP provides a standardized tool definition and description communication protocol that allows models to identify and access tools in a uniform manner. For a detailed description of MCP, reference can be made to previous articles of this blog. The paper on Tool Use is more in order to understand the development context and the evolution of the technical approach.

Toolformer: Self-supervised learning tool call

Schick et al., Toolformer: Language Models Can Teach Themselves to Use Tools, NeurIPS 2023.

Toolformer considers fine-tuning the original model to enhance its ability on Tool Use while learning the tools given. Its micro-modification dataset is generated using self-monitoring and is used once in a single decode.

• On the fine-tuning of self-supervised data sets: Use context learning to make models volunteer for tools. When the model uses the tool, the value of the tool is measured by comparing the tool/decode loss without the tool, and the value results are then organized into micro-alteration data Set
• About tools calling in decoding: Execute a general decode, request to call API when the decode has been requested to use the token symbol, return the result to the decoding sequence and continue the decoder process

Toolformer is an early attempt to introduce external tools, but reliance on decoding modifications is limited because of the lack of the ability to combine the reasoning of the current model. Also, because of the one-way decoded strategy, Toolformer cannot use tools in a chained manner to meet the needs of multi-smart body tools. The most worthwhile part of this is probably a self-monitoring learning strategy.

Gorilla: The combination of fine-tuning and retrieval

Patil et al., Gorilla: Large Language Model Connected with Massive APIs, 2023.

Most of the previous work on integrating tools into LLM takes into account a small, well-documented set of APIs that can easily be injected into the hint. However, support for a super-large, overlapping API warehouse requires new technologies to address it.

This paper considers ways to circumvent the introduction of the phrase by using Self-Instract Fine-Tuning (a fine-tuned parameter for the original model) and Retrieval (a search to obtain contextal tips, similar to MCP, but not all of which are injected into Prompt) to enhance the correct callability of the model to the large API library.

Combining directly Fine-Tune and Retrieva is the main improvement of this paper, namely, Retrieval-Aware TrainingI'm sorry. Experimental certificate:

• When there is a good searcher, Retriev-Aware Training is better than a simple fine-tuning.
• Retrieval-Aware Training adapts to fast changes in API documents
• When the call constraints (which require balancing multiple needs to determine which API to choose) are involved, the performance of all models has decreased significantly

Consider using fine-tuning and simple retrieval to achieve a tool call. Since multi-wheel reasoning and call is not involved, the relevant API that is retrieved from the command is referred to the model as its context.

Tulip Agent: Remittance task decompose and semantic search

Ruis et al., Tulip Agent: Enabling LLM-Based Agents to Solve Tasks Using Large Tool Libraries, 2024.

Tulip Agent does not encode all available tools into the system alert (which will take advantage of the context window of the model) nor does it embed the entire tip to retrieve the tool. It will then translate tasks into multiple subtasks, and then allow each subtask to perform a semantic-level vector database search, match the appropriate tool and allow dynamic management tools.

Compared to the technology before:

1. Dropping all tool descriptions into LLM as a hintTo avoid the problem of the long context
2. Discard the one-time embedded tool to find the hint instead of the first yesThe mission plans, embeds, retrieves and continues to reason.
3. UseVector Database, Embeding and RAG-like TechnologiesConducting tool retrieval
4. Dynamic management of allowed tools

In this technique, the searcher is activated to retrieve the API after each of the steps planned for the reasoning, and then the form below provides the model with reference to which API should be called.

ToolLLM: A framework for the use of common tools

Qin et al., ToolLLM: Facilitating Large Language Models to Master 16000+ Real-World APIs, ICLR 2024.

ToolLLM introduced a framework for the use of generic tools covering data construction, model training and evaluation, and the engineering design was more complete. As an open source project, it received considerable attention in the Tool Use direction and was one of the representative projects in the direction of the LLM Acts tool.

The closed-source model already has a strong tool mobilization capacity, but existing research in open-source communities is inadequate:

1. API limited in number, possibly too small in coverage and insufficient in diversity
2. Limit to single-tool calls, often assuming that the user gives the desired API set manually
3. Insufficient planning and reasoning, including CTT reasoning or REACH reasoning and action

Core Component

API Collection: 16,464 RET APIs were collected from the RapidAPI platform, covering 49 different categories and containing detailed documents for LLM learning.

Command Generation: Sample from the whole API collection, prompting ChatGPT to generate diversified commands, involving single and multi-tool scenarios (Self-Instract).

Path to Solutions: Each solution path may contain multi-wheel model reasoning and real-time API calls. To this end, the Decision Tree (DFSDT), based on the Depth Priority Search, was developed to enhance the planning and reasoning capabilities of LLM.

Evaluation (ToolEval): The AutoEval was developed to assess the use of LLM tools.

ToolLAMA: command-generated model by fine-tuning LLAMA on ToolBech.

Key findings

• ToolLaMA demonstrates the ability to process single tools and multi-tool commands
• ToolLAMA has demonstrated a strong panorama capability for an API that has not been seen, and only an API document can be used to adapt effectively to a new API
• DFSDT expanded the search space by considering multiple reasoning tracks, achieving better performance than React --It's a study of the reasoning strategy.

In this study, searches are conducted only once for user commands, and the search is linked to API, which gives models in multiple rounds of reasoning as context references to facilitate their thinking on reasoning strategies.

The DFSDTT given here is a decision-making reasoning strategy that is closely linked to the React. This paper is concerned not only with the utility caller performance but also with the process of extrapolating questions with the tool caller.

Concluding remarks

Recalling the research context of Tool Use, from the self-supervised fine-tuning of Toolformer, the Retrieving Enhancement Training of Gorilla, the Retrievation Retrieval of Tulip Agent, to the large-scale system engineering of ToolLLM, the study line has been structured around a question:How to get the model to find the right one in the big tool and call it right.

But,With the upgrading of basic modelling capacity and the emergence of agreements such as MCP (Model Context Protocol), the marginal benefits of Tool Calling as an independent training orientation are declining. The current front-line model already has a strong functional call capability and many scenes no longer require additional fine-tuning or complex search tubes for the “church” model tool. MCP deals with another layer of questions: how the tool is described, discovered, called and how to create a stable interface between the client and Server.

Many of the problems previously covered by the Tool Use study, such as the expansion of context, tool retrieval, multiple rotations, are now being more oriented towards protocols, running time and Agent projects.The focus of follow-up attention may be on how to move beyond training a model that is more tool-friendly and how to use tools, competencies, context and implementation processes as a stabilization system. This document is not updated.