Research note. This study evaluates retrieval models on public benchmarks. It does not describe an available product or its clinical performance.
Clinical mention
A short query in EN, ES, CA, IT, PT or FR
Stage 1 · Bi-encoder (multilingual)
Encodes the query and every CIE-10 description independently and ranks them by cosine similarity. Trained on 19,502 curated synthetic examples across 17 ICD-10 chapters.
Stage 2 · Cross-encoder reranker (Spanish)
Reads each query–code pair together and reorders the shortlist. Trained on 10,628 listwise groups with clinically plausible hard negatives.
Ranked CIE-10-ES codes
Laterality, severity, etiology and episode type resolved in the final order
01 - The question
Can compact task-specific retrievers outperform broad public embeddings?
Sentence embeddings work well for broad semantic search, but ICD coding is not broad semantic search. The model must map short clinical mentions to a controlled vocabulary where neighboring codes often differ by a qualifier: anatomical site, laterality, severity, encounter type, or etiology.
The paper asks whether high-quality synthetic supervision can close that gap for Spanish clinical coding. We use a frontier LLM to generate multilingual examples grounded in the ICD-10 hierarchy, fine-tune a Spanish biomedical bi-encoder, then train a cross-encoder reranker on listwise groups with clinically plausible hard negatives.
The headline finding: task-aligned supervision and clinically meaningful negatives matter more than embedding size or generic English biomedical pretraining for Spanish ICD-10/CIE-10-ES retrieval.
02 - Method
A bi-encoder for recall, a cross-encoder for exact-code precision.
The retriever uses a standard but carefully aligned two-stage architecture. Stage 1 encodes each query and CIE-10 description independently and ranks by cosine similarity. Stage 2 jointly reads each query-code pair and reorders the top candidates with a listwise objective.
The training data was not treated as raw synthetic output. We curated the generated examples through rule-based validation, semantic deduplication, language consistency checks, and manual spot inspection before using them to build the bi-encoder and reranking classifiers. That curation step is central to the paper: the goal is not simply to make more data, but to create examples and hard negatives that reflect the clinical distinctions encoded in ICD-10/CIE-10.
- Synthetic bi-encoder examples across 17 ICD-10 chapters.
- 19,502
- Languages used for synthetic supervision: EN, ES, CA, IT, PT, and FR.
- 6
- Spanish listwise cross-encoder groups with one positive and hard negatives.
- 10,628
- Evaluation resolutions: full code, 3-character category, and chapter.
- 3
Retrieval strategies compared
| Strategy | Role in the study |
|---|---|
| Tietai Cross-Encoder | Reranks the bi-encoder shortlist and optimizes exact-code ordering. |
| Tietai Bi-Encoder | Spanish biomedical dense retrieval backbone fine-tuned on synthetic supervision. |
| BM25 (Postgres FTS) | Lexical full-text baseline over CIE-10-ES descriptions. |
| ST MiniLM-L6-v2 | General-purpose public sentence-transformer baseline. |
| ST BioBERT | English biomedical sentence-transformer baseline. |
| ST MPNet-v2 | Larger 768-dimensional general sentence-transformer baseline. |
03 - Results
F1 and MAP@10 put the ranking quality in focus.
On CodiESP v4, the Tietai cross-encoder reaches F1 = 0.709 and MAP@10 = 0.747 at exact-code resolution. On DISTEMIST, it reaches F1 = 0.776 and MAP@10 = 0.812. The updated paper treats MAP@10 as the ranking metric and F1 as the top-1 decision metric.
- CodiESP exact-code F1 for the Tietai Cross-Encoder.
- 0.709
- CodiESP exact-code MAP@10 for the Tietai Cross-Encoder.
- 0.747
- DISTEMIST exact-code F1 for the Tietai Cross-Encoder.
- 0.776
- DISTEMIST exact-code MAP@10 for the Tietai Cross-Encoder.
- 0.812
CodiESP v4 F1 and MAP@10
| Model | F1 exact | F1 category | MAP@10 exact | MAP@10 category |
|---|---|---|---|---|
| Tietai Cross-Encoder | 0.709 | 0.823 | 0.747 | 0.851 |
| Tietai Bi-Encoder | 0.359 | 0.617 | 0.461 | 0.694 |
| BM25 (Postgres FTS) | 0.239 | 0.376 | 0.322 | 0.471 |
| ST MiniLM-L6-v2 | 0.225 | 0.371 | 0.287 | 0.426 |
| ST BioBERT | 0.193 | 0.314 | 0.252 | 0.373 |
| ST MPNet-v2 | 0.161 | 0.317 | 0.226 | 0.376 |
CodiESP exact-code top-1 precision, recall and F1
| Model | Precision | Recall | F1 |
|---|---|---|---|
| Tietai Cross-Encoder | 0.709 | 0.709 | 0.709 |
| Tietai Bi-Encoder | 0.359 | 0.359 | 0.359 |
| BM25 (Postgres FTS) | 0.254 | 0.227 | 0.239 |
| ST MiniLM-L6-v2 | 0.225 | 0.225 | 0.225 |
| ST BioBERT | 0.193 | 0.193 | 0.193 |
| ST MPNet-v2 | 0.161 | 0.161 | 0.161 |
CodiESP top-1 accuracy by hierarchy
DISTEMIST F1 and MAP@10
| Model | F1 exact | F1 category | MAP@10 exact | MAP@10 category |
|---|---|---|---|---|
| Tietai Cross-Encoder | 0.776 | 0.818 | 0.812 | 0.846 |
| Tietai Bi-Encoder | 0.603 | 0.690 | 0.682 | 0.747 |
| BM25 (Postgres FTS) | 0.431 | 0.470 | 0.504 | 0.537 |
04 - Interpretation
The bottleneck is task alignment, not embedding scale.
The public-baseline ordering is the important negative result. BioBERT-ST has biomedical pretraining, but it is English-centric. MPNet-v2 has larger embeddings, but it still trails MiniLM-L6-v2. Neither scale nor generic domain pretraining solves Spanish clinical-code retrieval by itself.
ICD retrieval requires models to separate sibling concepts that are semantically close but operationally different. The bi-encoder captures chapter-level topic information well, but exact-code ranking needs a reranker trained on hard negatives drawn from neighboring CIE-10 codes.
- Task alignment matters. Synthetic examples must reflect the ICD hierarchy, not just broad biomedical relatedness.
- Hard negatives matter. Neighboring codes teach the model to distinguish etiology, anatomy, severity, and episode type.
- Reranking matters. The cross-encoder adds the qualifier-level discrimination required by CIE-10-ES guidelines.
- Small data can be enough. A compact synthetic corpus can train a competitive retriever when examples are aligned with the target vocabulary structure.
- Biomedical pretraining is not automatic transfer. English biomedical encoders still struggle when both query and target vocabulary are Spanish.
For production clinical coding search, the practical lesson is to train for the target vocabulary, target language, and target ranking problem. The useful unit is not a bigger embedding model; it is a retriever trained on the distinctions coders actually need to make.
