CoHyDE: Iterative co‑training of an LLM rewriter and dense encoder aims to fix tool retrieval

What the paper proposes

Researchers have posted a new preprint on arXiv (arXiv:2605.29271) introducing CoHyDE, an iterative co‑training scheme that pairs a large language model (LLM) used as a query rewriter with a dense encoder trained for retrieval. The problem is simple to state and hard to solve: user queries are colloquial and underspecified, while large API catalogs use terse, technical vocabulary — how do you bridge the gap? CoHyDE reportedly alternates between using the LLM to generate task‑oriented rewrites and using those rewrites to fine‑tune a dense encoder, with the encoder’s retrieval outputs then used to further refine the rewriter in a loop.

How it differs from prior work

Existing approaches have fallen into two camps: contrastive fine‑tuning of encoders to align queries with API metadata, and HyDE‑style query expansion where a frozen LLM generates context to improve retrieval. CoHyDE’s key angle is co‑training: the rewriter and encoder improve one another iteratively instead of treating one component as fixed. The authors report improvements on standard tool‑retrieval benchmarks versus single‑stage baselines, though these results come from a fresh arXiv submission and should be treated as preliminary.

Why practitioners — and platforms — should care

Tool retrieval is a core bottleneck for LLM agents that orchestrate APIs and automation. Better retrieval reduces latency, error-prone tool selection, and brittle prompt engineering. That matters to cloud and AI platform operators worldwide, including major Chinese firms such as Baidu (百度) and Alibaba (阿里巴巴), which are building agentic LLM services. Geopolitics is relevant too: as export controls and chip supply issues shape where heavy models run, algorithmic gains that improve retrieval efficiency can influence deployment choices and competitive advantage.

Caveats and next steps

This is an arXiv preprint and not yet peer‑reviewed; it has been reported that code and detailed evaluations may follow. Open questions remain about robustness on messy, real‑world API catalogs, the cost of iterative training, and privacy implications when synthetic rewrites touch user data. Still, CoHyDE is a clear attempt to move beyond frozen‑LLM heuristics toward tighter co‑design of rewriting and retrieval.