TrecTools: an open-source Python library for Information Retrieval practitioners involved in TREC-like campaigns

2019 | Proceedings of the 42nd annual international ACM SIGIR conference on Research and development in Information Retrieval
Share via twitter Share via email Download PDF



This paper introduces TrecTools, a Python library for assisting Information Retrieval (IR) practitioners with TREC-like campaigns. IR practitioners tasked with activities like building test collections, evaluating systems, or analysing results from empirical experiments commonly have to resort to use a number of different software tools and scripts that each perform an individual functionality – and at times they even have to implement ad-hoc scripts of their own. TrecTools aims to provide a unified environment for performing these common activities.

Written in the most popular programming language for Data Science, Python, TrecTools offers an object-oriented, easily exten- sible library. Existing systems, e.g., trec_eval, have considerable barrier to entry when it comes to modify or extend them. Fur- thermore, many existing IR measures and tools are implemented independently of each other, in different programming languages. TrecTools seeks to lower the barrier to entry and to unify exist- ing tools, frameworks and activities into one common umbrella. Widespread adoption of a centralised solution for developing, eval- uating, and analysing TREC-like campaigns will ease the burden on organisers and provide participants and users with a standard environment for common IR experimental activities. TrecTools is distributed as an open source library under the MIT license at