LETOR: Learning to Rank for Information Retrieval

Established: January 1, 2009

  • Because of the fast development of this area, it is difficult to keep the list up-to-date and comprehensive. For a comprehensive list and more recent papers, please refer to

    If your paper is not listed, please let us know taoqin@microsoft.com (opens in new tab).

    1. James Petterson, Tiberio Caetano, Julian McAuley and Jin Yu. Exponential Family Graph Matching and Ranking. In NIPS 2009.
    2. C. Rudin. The P-Norm Push: A Simple Convex Ranking Algorithm that Concentrates at the Top of the List. Journal of Machine Learning Research, 10 (2009) 2233-2271.
    3. C. Rudin, and R. Schapire. Margin-Based Ranking and an Equivalence Between AdaBoost and RankBoost. Journal of Machine Learning Research, 10 (2009) 2193-2232.
    4. C. Rudin, R. Passonneau, A. Radeva, H. Dutta, S. Ierome, and D. Isaac. A Process for Predicting Manhole Events in Manhattan. To appear, Machine Learning, 2010.
    5. S. Agarwal, T. Graepel, T. Herbrich, S. Har-Peled, and D. Roth. Generalization bounds for the area under the roc curve. Journal of Machine Learning, 6:393-425, 2005.
    6. S. Agarwal and P. Niyogi. Stability and generalization of bipartite ranking algorithms. In COLT 2005, pages 32-47, 2005.
    7. E. Agichtein, E. Brill, S. T. Dumais, and R. Ragno. Learning user interaction models for predicting web search result preferences. In SIGIR 2006, pages 3-10, 2006.
    8. N. Ailon and MehryarMohri. An efficient reduction from ranking to classification. In COLT 2008, 2008.
    9. H. Almeida, M. Goncalves, M. Cristo, and P. Calado. A combined component approach for finding collection-adapted ranking functions based on genetic programming. In SIGIR 2007, pages 399-406, 2007.
    10. M.-R. Amini, T.-V. Truong, and C. Goutte. A boosting algorithm for learning bipartite ranking functions with partially labeled data. In SIGIR 2008, pages 99-106, 2008.
    11. M.-F. Balcan, N. Bansal, A. Beygelzimer, D. Coppersmith, J. Langford, and G. B. Sorkin. Robust reductions from ranking to classification. In COLT 2007, 2007.
    12. B. Bartell, G. W. Cottrell, and R. Belew. Learning to retrieve information. In SCC 1995, 1995.
    13. C. J. Burges, R. Ragno, and Q. V. Le. Learning to rank with nonsmooth cost functions. In NIPS 2006, pages 395-402, 2006.
    14. C. J. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender. Learning to rank using gradient descent. In ICML 2005, pages 89-96, 2005.
    15. G. Cao, J. Nie, L. Si, J. Bai, Learning to Rank Documents for Ad-Hoc Retrieval with Regularized Models, SIGIR 2007 workshop: Learning to Rank for Information Retrieval, 2007
    16. Y. Cao, J. Xu, T.-Y. Liu, H. Li, Y. Huang, and H.-W. Hon. Adapting ranking svm to document retrieval. In SIGIR 2006, pages 186-193, 2006.
    17. Z. Cao, T. Qin, T.-Y. Liu, M.-F. Tsai, and H. Li. Learning to rank: from pairwise approach to listwise approach. In ICML 2007, pages 129-136, 2007.
    18. V. R. Carvalho, J. L. Elsas, W. W. Cohen, and J. G. Carbonell. A metalearningapproach for robust rank learning. In SIGIR 2008 workshop on Learning to Rank for Information Retrieval, 2008.
    19. S. Chakrabarti, R. Khanna, U. Sawant, and C. Bhattacharyya. Structured learning for non-smooth ranking losses. In SIGKDD 2008, pages 88-96, 2008.
    20. O. Chapelle, Q. Le, and A. Smola. Large margin optimization of ranking measures. In NIPS workshop on Machine Learning for Web Search 2007, 2007.
    21. W. Chu and Z. Ghahramani. Gaussian processes for ordinal regression. Journal of Machine Learning Research, 6:1019-1041, 2005.
    22. W. Chu and Z. Ghahramani. Preference learning with Gaussian processes. In ICML 2005, pages 137-144, 2005.
    23. W. Chu and S. S. Keerthi. New approaches to support vector ordinal regression. In ICML 2005, pages 145-152, 2005.
    24. S. Clemenson, G. Lugosi, and N. Vayatis. Ranking and scoring using empirical risk minimization. In COLT 2005, 2005.
    25. W. W. Cohen, R. E. Schapire, and Y. Singer. Learning to order things. In NIPS 1998, volume 10, pages 243-270, 1998.
    26. C. Cortes, M. Mohri, and etc. Magnitude-preserving ranking algorithms. In ICML 2007, pages 169-176, 2007.
    27. D. Cossock and T. Zhang. Subset ranking using regression. In COLT 2006, pages 605-619, 2006.
    28. K. Crammer and Y. Singer. Pranking with ranking. In NIPS 2002, pages 641-647, 2002.
    29. K. Duh and K. Kirchhoff. Learning to rank with partially-labeled data. In SIGIR 2008, pages 251-258, 2008.
    30. W. Fan, E. A. Fox, P. Pathak, and H. Wu. The effects of fitness functions on genetic programming based ranking discovery for web search. Journal of American Society for Information Science and Technology, 55(7):628-636, 2004.
    31. W. Fan, M. Gordon, and P. Pathak. Discovery of context-specific ranking functions for effective information retrieval using genetic programming. IEEE Transactions on Knowledge and Data Engineering, 16(4):523-527, 2004.
    32. W. Fan, M. Gordon, and P. Pathak. A generic ranking function discovery framework by genetic programming for information retrieval. Information Processing and Management, 40(4):587-602, 2004.
    33. W. Fan, M. Gordon, and P. Pathak. Genetic programming-based discovery of ranking functions for effective web search. Journal of Management of Information Systems, 21(4):37-56, 2005.
    34. W. Fan, M. Gordon, and P. Pathak. On linear mixture of expert approaches to information retrieval. Decision Support System, 42(2):975-987, 2006.
    35. W. Fan, M. D. Gordon, W. Xi, and E. A. Fox. Ranking function optimization for effective web search by genetic programming: an empirical study. In HICSS 2004, page 40105, 2004.
    36. Y. Freund, R. Iyer, R. E. Schapire, and Y. Singer. An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research, 4:933-969, 2003.
    37. N. Fuhr. Optimum polynomial retrieval functions based on the probability ranking principle. ACM Transactions on Information Systems, 7(3):183-204, 1989.
    38. G. Fung, R. Rosales, and B. Krishnapuram, Learning Rankings via Convex Hull Separation, NIPS 2005 workshop on Learning to Rank, 2005
    39. J. Gao, H. Qi, X. Xia, and J. Nie. Linear discriminant model for information retrieval. In SIGIR 2005, pages 290-297, 2005.
    40. X.-B. Geng, T.-Y. Liu, and T. Qin. Feature selection for ranking. In SIGIR 2007, pages 407-414, 2007.
    41. X.-B. Geng, T.-Y. Liu, T. Qin, H. Li, and H.-Y. Shum. Query-dependent ranking using k-nearest neighbor. In SIGIR 2008, pages 115-122, 2008.
    42. J. Guiver and E. Snelson. Learning to rank with softrank and gaussian processes. In SIGIR 2008, pages 259-266, 2008.
    43. E. F. Harrington. Online ranking/collaborative filtering using the perceptron algorithm. In ICML 2003, pages 250-257, 2003.
    44. R. Herbrich, K. Obermayer, and T. Graepel. Large margin rank boundaries for ordinal regression. In Advances in Large Margin Classifiers, pages 115-132, 2000.
    45. R. Jin, H. Valizadegan, and H. Li. Ranking refinement and its application to information retrieval. In WWW 2008, pages 397-406, 2008.
    46. T. Joachims. Optimizing search engines using clickthrough data. In KDD 2002, pages 133-142, 2002.
    47. T. Joachims. A support vector method for multivariate performance measures. In ICML 2005, pages 377-384, 2005.
    48. S. Kramer, G. Widmer, B. Pfahringer, and M. D. Groeve. Prediction of ordinal classes using regression trees. Funfamenta Informaticae, 34:1-15, 2000.
    49. J. Lafferty and C. Zhai. Document language models, query models and risk minimization for information retrieval. In SIGIR 2001, pages 111-119, 2001.
    50. Y. Lan, T.-Y. Liu, T. Qin, Z. Ma, and H. Li. Query-level stability and generalization in learning to rank. In ICML 2008, pages 512-519, 2008.
    51. G. Lebanon and J. Lafferty. Cranking: Combining rankings using conditional probability models on permutations. In ICML 2002, pages 363-370, 2002.
    52. P. Li, C. Burges, and Q. Wu. Mcrank: Learning to rank using multiple classification and gradient boosting. In NIPS 2007, 2007.
    53. T.-Y. Liu, J. Xu, T. Qin, W.-Y. Xiong, and H. Li. LETOR: Benchmark dataset for research on learning to rank for information retrieval. In SIGIR ’07 Workshop on learning to rank for information retrieval, 2007.
    54. Y. Liu, T.-Y. Liu, T. Qin, Z. Ma, and H. Li. Supervised rank aggregation. In WWW 2007, pages 481-490, 2007.
    55. I. Matveeva, C. Burges, T. Burkard, A. Laucius, and L. Wong. High accuracy retrieval with multiple nested ranker. In SIGIR 2006, pages 437-444, 2006.
    56. D. A. Metzler and W. B. Croft. A Markov random field model for term dependencies. In SIGIR 2005, pages 472-479, 2005.
    57. D. A. Metzler, W. B. Croft, and A. McCallum. Direct maximization of rank based metrics for information retrieval. In CIIR Technical Report, 2005.
    58. D. A. Metzler and T. Kanungo. Machine learned sentence selection strategies for query-biased summarization. In SIGIR 2008 workshop on Learning to Rank for Information Retrieval, 2008.
    59. T. Minka and S. Robertson. Selection bias in the LETOR datasets. In SIGIR 2008 workshop on Learning to Rank for Information Retrieval, 2008.
    60. R. Nallapati. Discriminative models for information retrieval. In SIGIR 2004, pages 64-71, 2004.
    61. T. Pahikkala, E. Tsivtsivadze, A. Airola, J. Boberg, T. Salakoski, Learning to Rank with Pairwise Regularized Least-Squares, SIGIR 2007 workshop: Learning to Rank for Information Retrieval, 2007
    62. L. Rigutini, T. Papini, M. Maggini, and F. Scarselli. Learning to rank by a neural-based sorting algorithm. In SIGIR 2008 workshop on Learning to Rank for Information Retrieval, 2008.
    63. T. Qin, T.-Y. Liu, W. Lai, X.-D. Zhang, D.-S.Wang, and H. Li. Ranking with multiple hyperplanes. In SIGIR 2007, pages 279-286, 2007.
    64. T. Qin, T.-Y. Liu, M.-F. Tsai, X.-D. Zhang, and H. Li. Learning to search web pages with query-level loss functions. Technical Report, MSR-TR-2006-156, 2006.
    65. T. Qin, T.-Y. Liu, X.-D. Zhang, D. Wang, and H. Li. Learning to rank relational objects and its application to web search. In WWW 2008, pages407-416, 2008.
    66. T. Qin, T.-Y. Liu, X.-D. Zhang, D.-S. Wang, and H. Li. Global ranking using continuous conditional random fields. In NIPS 2008, 2008.
    67. T. Qin, T.-Y. L. X.-D. Zhang, M.-F. Tsai, D.-S. Wang, and H. Li. Query-level loss functions for information retrieval. Information Processing & Management, 44(2):838-855, 2007.
    68. T. Qin, T.-Y. Liu, J. Xu, and H. Li. How to make LETOR more useful and reliable. In SIGIR 2008 workshop on Learning to Rank for Information Retrieval, 2008.
    69. F. Radlinski and T. Joachims. Query chain: Learning to rank from implicit feedback. In KDD 2005, pages 239-248, 2005.
    70. F. Radlinski and T. Joachims. Active exploration for learning rankings from clickthrough data. In KDD 2007, 2007.
    71. F. Radlinski, R. Kleinberg, and T. Joachims. Learning diverse rankings with multi-armed bandits. In ICML 2008, pages 784-791, 2008.
    72. S. Rajaram and S. Agarwal. Generalization bounds for k-partite ranking. In NIPS 2005 WorkShop on Learning to Rank, 2005.
    73. S. Robertson and H. Zaragoza. On rank-based effectiveness measures and optimization. Information Retrieval, 10(3):321-339, 2007.
    74. C. Rudin, C. Cortes, M. Mohri, and R. E. Schapire, Margin-Based Ranking Meets Boosting in the Middle, COLT 2005.
    75. A. Shashua and A. Levin. Ranking with large margin principles: Two approaches. In NIPS 2002, pages 937-944, 2002.
    76. M. Talyor, J. Guiver, and etc. Softrank: Optimising non-smooth rank metrics. In WSDM 2008, pages 77-86, 2008.
    77. M. Taylor, H. Zaragoza, N. Craswell, S. Robertson, and C. J. Burges. Optimisation methods for ranking functions with multiple parameters. In CIKM 2006, pages 585-593, 2006.
    78. A. Trotman. Learning to rank. Information Retrieval, 8(3):359-381, 2005.
    79. M.-F. Tsai, T.-Y. Liu, T. Qin, H.-H. Chen, and W.-Y. Ma. Frank: a ranking method with fidelity loss. In SIGIR 2007, pages 383-390, 2007.
    80. A. Veloso, H. M. de Almeida, M. A. Gon?alves, and W. M. Jr. Learning to rank at query-time using association rules. In SIGIR 2008, pages 267-274, 2008.
    81. N. Usunier, V. Truong, M. R. Amini, and P. Gallinari, Ranking with Unlabeled Data: A First Study, NIPS 2005 workshop:Learning to Rank, 2005.
    82. W. Xi, J. Lind, and E. Brill, Learning effective ranking functions for newsgroup search, SIGIR 2004.
    83. F. Xia, T.-Y. Liu, J. Wang, W. Zhang, and H. Li. Listwise approach to learning to rank – theorem and algorithm. In ICML 2008, pages 1192-1199, 2008.
    84. J. Xu, Y. Cao, H. Li, and Y. Huang. Cost-sensitive learning of SVM for ranking. In ECML 2006, pages 833-840, 2006.
    85. J. Xu and H. Li. Adarank: a boosting algorithm for information retrieval. In SIGIR 2007, pages 391-398, 2007.
    86. J. Xu, T.-Y. Liu, M. Lu, H. Li, and W.-Y. Ma. Directly optimizing IR evaluation measures in learning to rank. In SIGIR 2008, pages 107-114, 2008.
    87. J.-Y. Yeh, J.-Y. Lin, and etc. Learning to rank for information retrieval using genetic programming. In LR4IR 2007, 2007.
    88. H. Yu. SVM selective sampling for ranking with application to data retrieval. In KDD 2005, pages 354-363, 2005.
    89. Y. Yue, T. Finley, F. Radlinski, and T. Joachims. A support vector method for optimizing average precision. In SIGIR 2007, pages 271-278, 2007.
    90. Y. Yue and T. Joachims. Predicting diverse subsets using structural SVM. In ICML 2008, pages 1224-1231, 2008.
    91. C. Zhai and J. Lafferty. A risk minimization framework for information retrieval. Information Processing and Management, 42(1):31-55, 2006.
    92. Z. Zheng, K. Chen, G. Sun, and H. Zha. A regression framework for learning ranking functions using relative relevance judgments. In SIGIR 2007, pages 287-294, 2007.
    93. Z. Zheng, H. Zha, and G. Sun. Query-level learning to rank using isotonic regression. In SIGIR 2008 workshop on Learning to Rank for Information Retrieval, 2008.
    94. Z. Zheng, H. Zha, and etc. A general boosting method and its application to learning ranking functions for web search. In NIPS 2007, 2007.
    95. K. Zhou, G.-R. Xue, H. Zha, and Y. Yu. Learning to rank with ties. In SIGIR 2008, pages 275-282, 2008.
    96. O. Zoeter, M. Taylor, E. Snelson, J. Guiver, N. Craswell, and M. Szummer. A decision theoretic framework for ranking using implicit feedback. In SIGIR 2008 workshop on Learning to Rank for Information Retrieval, 2008.
    97. Jonathan L. Elsas, Vitor R. Carvalho, Jaime G. Carbonell. “Fast Learning of Document Ranking Functions with the Committee Perceptron,” Proceedings of the First ACM International Conference on Web Search and Data Mining (WSDM 2008), 2008.
    98. Ronan Cummins and Colm O’Riordan. An axiomatic comparison of learned term-weighting schemes in information retrieval: clarifications and extensions. Artificial Intelligence Review Journal.
    99. Ronan Cummins and Colm O’Riordan. Evolving local and global weighting schemes in information retrieval. Journal of Information Retrieval.

  • Tutorial talks

    Tutorial papers/books

 

 

People

Portrait of Tao Qin

Tao Qin

Partner Research Manager

Portrait of Tie-Yan Liu

Tie-Yan Liu

Distinguished Scientist, Microsoft Research AI for Science