TL;DR
Toolformer teaches language models to autonomously decide when and how to call external APIs using self-supervised learning, where only API calls that reduce next-token perplexity are retained for fine-tuning.
Summary
Language models lack numerical, temporal, and factual precision that external tools can provide, but learning when to use tools typically requires human annotation. Toolformer eliminates this by generating candidate API calls — calculator, calendar, Wikipedia search, translation, QA model — inserting them into text, and retaining only those that reduce perplexity on subsequent tokens. Fine-tuning on this self-curated data produces a 6.7B GPT-J model that outperforms GPT-3 (175B) on several downstream benchmarks while preserving general language modeling quality.
Key contributions
- Proposes a self-supervised data curation method where API calls are selected by perplexity reduction on subsequent tokens — no human annotation required.
- Demonstrates that a 6.7B model fine-tuned with tool use surpasses GPT-3 175B on multiple downstream benchmarks.
- Shows that fine-tuning on tool-augmented data preserves base language modeling capability without degradation.
- Introduces perplexity reduction as a principled, task-agnostic signal for deciding when tool invocation is beneficial.
When to cite
- When designing a self-supervised approach to teaching LLMs tool use without human-labeled API call data.
- When arguing that small models with tool access can outperform much larger models without tools.
- When justifying perplexity reduction as a signal for selecting useful tool invocations.
- When surveying LLM tool-use methods and needing the foundational self-supervised approach.