Publications

On the Orthogonality of Bias and Effectiveness in Ad hoc Retrieval

Amin Bigdeli and Negar Arabzadeh and Shirin Seyedsalehi and Morteza Zihayat and Ebrahim Bagheri
Reference:
Amin Bigdeli; Negar Arabzadeh; Shirin Seyedsalehi; Morteza Zihayat and Ebrahim Bagheri On the Orthogonality of Bias and Effectiveness in Ad hoc Retrieval. In The 44th International ACM SIGIR Conference. on Research and Development in Information Retrieval (SIGIR 2021), 2021.
Links to Publication:
Abstract:
Various researchers have recently explored the impact of different types of biases on information retrieval tasks such as ad hoc retrieval and question answering. While the impact of bias needs to be controlled in order to avoid increased prejudices, the literature has often viewed the relationship between increased retrieval utility (effectiveness) and reduced bias as a tradeoff where one can suffer from the other. In this paper, we empirically study this tradeoff and explore whether it would be possible to reduce bias while maintaining similar retrieval utility. We show this would be possible by revising the input query through a bias-aware pseudo-relevance feedback framework. We report our findings based on four widely used TREC corpora namely Robust04, Gov2, ClueWeb09 and ClueWeb12 and using two classes of bias metrics. The findings of this paper are significant as they are among the first to show that decrease in bias does not necessarily need to come at the cost of reduced utility.
Bibtex Entry:
@inproceedings{sigir2021a, author = {Amin Bigdeli and Negar Arabzadeh and Shirin Seyedsalehi and Morteza Zihayat and Ebrahim Bagheri}, title = {On the Orthogonality of Bias and Effectiveness in Ad hoc Retrieval}, booktitle = {The 44th International ACM SIGIR Conference. on Research and Development in Information Retrieval (SIGIR 2021)}, year = {2021}, abstract = {Various researchers have recently explored the impact of different types of biases on information retrieval tasks such as ad hoc retrieval and question answering. While the impact of bias needs to be controlled in order to avoid increased prejudices, the literature has often viewed the relationship between increased retrieval utility (effectiveness) and reduced bias as a tradeoff where one can suffer from the other. In this paper, we empirically study this tradeoff and explore whether it would be possible to reduce bias while maintaining similar retrieval utility. We show this would be possible by revising the input query through a bias-aware pseudo-relevance feedback framework. We report our findings based on four widely used TREC corpora namely Robust04, Gov2, ClueWeb09 and ClueWeb12 and using two classes of bias metrics. The findings of this paper are significant as they are among the first to show that decrease in bias does not necessarily need to come at the cost of reduced utility.} }




Powered by WordPress