Publications of Lihong Li


My Google Scholar Entry

My DBLP Entry

My arXiv Entry


  • Journal Papers
  1. Jiang Bian, Bo Long, Lihong Li, Taesup Moon, Anlei Dong, and Yi Chang: Exploiting user preference for online learning in Web content optimization systems.   In ACM Transactions on Intelligent Systems and Technology, 5(2):33, 2014.
  2. Taesup Moon, Wei Chu, Lihong Li, Zhaohui Zheng, and Yi Chang: An online learning framework for refining recency search results with user click feedback.   In ACM Transactions on Information Systems, 30(4):20, 2012.
  3. John Langford, Lihong Li, Preston McAfee, and Kishore Papineni: Cloud control: Voluntary admission control for Intranet traffic management .  In Information Systems and e-Business Management, 10(3):295-308, 2012.
  4. Lihong Li, Michael L. Littman, Thomas J. Walsh, and Alexander L. Strehl: Knows what it knows: A framework for self-aware learning.  In Machine Learning, 82(3):399-443, 2011.
  5. Lihong Li and Michael L. Littman: Reducing reinforcement learning to KWIK online regression.  In Annals of Mathematics and Artificial Intelligence, 58(3-4):217-237, 2010.
  6. John Langford, Lihong Li, Yevgeniy Vorobeychik, and Jennifer Wortman: Maintaining equilibria during exploration in sponsored search auctions.  In Algorithmica, 58(4):990-1021, 2010.
  7. Alexander L. Strehl, Lihong Li, and Michael L. Littman: Reinforcement learning in finite MDPs: PAC analysis.  In the Journal of Machine Learning Research, 10:2413-2444, 2009.
  8. Emma Brunskill, Bethany R. Leffler, Lihong Li, Michael L. Littman, and Nichlos Roy: Provably efficient learning with typed parametric models . In the  Journal of Machine Learning Research, 10:1955-1988, 2009.
  9. John Langford, Lihong Li, and Tong Zhang: Sparse online learning via truncated gradient.  In the Journal of Machine Learning Research, 10:777-801, 2009.
  10. Thomas J. Walsh, Ali Nouri, Lihong Li, and Michael L. Littman: Planning and learning in environments with delayed feedback.  In the Journal of Autonomous Agents and Multi-Agent Systems, 18(1):83-105, 2009.  A preliminary version appeared in ECML-07.
  11. Lihong Li, Vadim Bulitko, and Russell Greiner: Focus of attention in reinforcement learning.  In the Journal of Universal Computer Science, 13(9):1246-1269, November, 2007.
  12. Lihong Li, Min Shao, Zhenkun Zheng, Chuan He, and Zhi-Hui Du: Typical XML document transformation methods and an application system. In Computer Science, 30(2): 40-44, China, 2003.

 

  • Conference
  1. Lihong Li, He He, and Jason D. Williams: Temporal supervised learning for inferring a dialog policy from example conversations.  To appear in SLT-14, South Lake Tahoe, NV, USA, December, 2014.
  2. Alekh Agarwal, Daniel Hsu, Satyen Kale, John Langford, Lihong Li, and Robert E. Schapire: Taming the monster: A fast and simple algorithm for contextual bandits.  In ICML-14, Beijing, China, June, 2014.
  3. Emma Brunskill and Lihong Li: PAC-inspired option discovery in lifelong reinforcement learning.  In ICML-14, Beijing, China, June, 2014.
  4. Emma Brunskill and Lihong Li: Sample complexity of multi-task reinforcement learning.  In UAI-13, Bellevue, WA, USA, July, 2013.
  5. Miroslav Dudik, Dumitru Erhan, John Langford, and Lihong Li: Sample-efficient nonstationary-policy evaluation for contextual bandits.  In UAI-12, Catalina Island, CA, USA, August, 2012.
  6. Lihong Li, Wei Chu, John Langford, Taesup Moon, and Xuanhui Wang: An unbiased offline evaluation of contextual bandit algorithms with generalized linear models.  In Journal of Machine Learning Research - Workshop and Conference Proceedings 26: On-line Trading of Exploration and Exploitation 2, 26:19-36, 2012.
  7. Vidhya Navalpakkam, Ravi Kumar, Lihong Li, and D Sivakumar: Attention and selection in online choice tasks.   In UMAP-12, Montreal, Canada, July, 2012.
  8. Hongning Wang, Anlei Dong, Lihong Li, Yi Chang and Evgeniy Gabrilovich: Joint relevance and freshness learning From clickthroughs for news search.   In WWW-12, Lyon, France, April, 2012.
  9. Olivier Chapelle and Lihong Li: An empirical evaluation of Thompson sampling.  In NIPS-11, Granada, Spain, December, 2011.
  10. Wei Chu, Martin Zinkevich, Lihong Li, Achint Thomas, and Belle Tseng: Unbiased online active learning in data streams.   In KDD-11, San Diego, CA, August, 2011.
  11. Miroslav Dudik, John Langford, and Lihong Li: Doubly robust policy evaluation and learning.   In ICML-11, Bellevue, WA, June, 2011. [arXiv version]
  12. Deepak Agarwal, Lihong Li, and Alexander J. Smola: Linear-time estimators for propensity scores.   In AISTATS-11, Fort Lauderdale, FL, April, 2011.
  13. Alina Beygelzimer, John Langford, Lihong Li, Lev Reyzin, and Robert E. Schapire: Contextual bandit algorithms with supervised learning guarantees.   In AISTATS-11, Fort Lauderdale, FL, April, 2011. Co-winner of the Notable Paper Award.
  14. Wei Chu, Lihong Li, Lev Reyzin, and Robert E. Schapire: Contextual bandits with linear payoff functions.   In AISTATS-11, Fort Lauderdale, FL, April, 2011.
  15. Lihong Li, Wei Chu, John Langford, and Xuanhui Wang: Unbiased offline evaluation of contextual-bandit-based news article recommendation algorithms .  In WSDM-11, Hong Kong, China, February, 2011. Winner of the Best Paper Award.
  16. Alexander L. Strehl, John Langford, Lihong Li, and Sham M. Kakade: Learning from logged implicit exploration data.  In NIPS-10, Vancouver, BC, Canada, 2010.
  17. Martin Zinkevich, Alexander J. Smola, Markus Weimer, and Lihong Li: Parallelized stochastic gradient descent.  In NIPS-10, Vancouver, BC, Canada, 2010.
  18. Taesup Moon, Lihong Li, Wei Chu, Ciya Liao, Zhaohui Zheng, and Yi Chang: Online learning for recency search ranking using real-time user feedback.  In CIKM-10, Toronto, Canada, October, 2010.
  19. Lihong Li, Wei Chu, John Langford, and Robert E. Schapire: A contextual-bandit approach to personalized news article recommendation.  In WWW-10, Raleigh, NC, April, 2010.
  20. Lihong Li, Jason D. Williams, and Suhrid Balakrishnan: Reinforcement learning for dialog management using least-squares policy iteration and fast feature selection.  In INTERSPEECH-09, Brighton, UK, September, 2009.
  21. Carlos Diuk, Lihong Li, and Bethany R. Leffler: The adaptive k-meteorologists problem and its application to structure learning and feature selection in reinforcement learning.  In ICML-09, Montreal, Canada, June, 2009.
  22. John Asmuth, Lihong Li, Michael L. Littman, Ali Nouri, and David Wingate: A Bayesian sampling approach to exploration in reinforcement learning.  In UAI-09, Montreal, Canada, June, 2009.
  23. Lihong Li, Michael L. Littman, and Christopher R. Mansley: Online exploration in least-squares policy iteration.  In AAMAS-09, Budapest, Hungary, May, 2009.
  24. John Langford, Lihong Li, and Tong Zhang: Sparse online learning via truncated gradient.  In NIPS-08, Vancouver, BC, Canada, 2008. [poster][spotlight]
  25. Lihong Li: A worst-case comparison between temporal difference and residual gradient with linear function approximation.  In ICML-08, Helsinki, Finland, July, 2008. [slides][poster]
  26. Lihong Li, Michael L. Littman, and Thomas J. Walsh: Knows what it knows: A framework for self-aware learning.  In ICML-08, Helsinki, Finland, July, 2008. [slides][poster][video] Co-winner of the Best Student Paper Award.
  27. Ronald Parr, Lihong Li, Gavin Taylor, Christopher Painter-Wakefield, and Michael L. Littman: An analysis of linear models, linear value-function approximation, and feature selection for reinforcement learning.  In ICML-08, Helsinki, Finland, July, 2008. [Rons slides][poster][addendum]
  28. Emma Brunskill, Bethany R. Leffler, Lihong Li, Michael L. Littman, and Nicholas Roy: CORL: A continuous-state offset-dynamics reinforcement learner.  In UAI-08, Helsinki, Finland, July, 2008. [Emmas slides]
  29. Lihong Li and Michael L. Littman: Efficient value-function approximation via online linear regression.  In AI&Math-08, Fort Lauderdale, FL, January, 2008.  [slides]
  30. Jennifer Wortman, Yevgeniy Vorobeychik, Lihong Li, and John Langford: Maintaining equilibria during exploration in sponsored search auctions.  In WINE-07, San Diego, CA, December, 2007.  Also appears in LNCS 4858.  A longer version with proofs is here.  [Jenns slides].
  31. Thomas J. Walsh, Ali Nouri, Lihong Li, and Michael L. Littman: Planning and learning in environments with delayed feedback.  In ECML-07, Warsaw, Poland, September, 2007.  Also appears in LNCS 4701. [Toms slides]
  32. Ronald Parr, Christopher Painter-Wakefield, Lihong Li, and Michael L. Littman: Analyzing feature generation for value-function approximation.  In ICML-07, Corvallis, OR, June, 2007. [poster]
  33. Alexander L. Strehl, Lihong Li, Eric Wiewiora, John Langford, and Michael L. Littman: PAC model-free reinforcement learning.  In ICML-06, Pittsburgh, PA, June, 2006. [slides]
  34. Alexander L. Strehl, Lihong Li, and Michael L. Littman: Incremental model-based learners with formal learning-time guarantees.  In UAI-06, Cambridge, MA, July, 2006.
  35. Lihong Li, Thomas J. Walsh, and Michael L. Littman: Towards a unified theory of state abstraction for MDPs.  In AI&Math-06, Fort Lauderdale, FL, January, 2006. [Toms slides]
  36. Lihong Li and Michael L. Littman: Lazy approximation for solving continuous finite-horizon MDPs.  In AAAI-05, pages 1175-1180, Pittsburgh, PA, 2005. [slides]
  37. Lihong Li, Vadim Bulitko, and Russell Greiner: Batch reinforcement learning with state importance. In ECML-04, pages 566-568, Pisa, Italy, 2004.  Also appears in LNCS 3201. [poster]
  38. Vadim Bulitko, Lihong Li, Russell Greiner, and Ilya Levner: Lookahead pathologies for single agent search. In IJCAI-03, pages 1531-1533, Acapulco, Mexico, August, 2003. [poster]
  39. Lihong Li, Vadim Bulitko, Russell Greiner, and Ilya Levner: Improving an adaptive image interpretation system by leveraging. In the Eighth Australian and New Zealand Conference on Intelligent Information Systems, Sydney, Australia, December 2003.
  40. Ilya Levner, Vadim Bulitko, Lihong Li, Greg Lee, and Russell Greiner: Learning robust object recognition strategies. In the Eighth Australian and New Zealand Conference on Intelligent Information Systems, Sydney, Australia, December 2003.
  41. Ilya Levner, Vadim Bulitko, Lihong Li, Greg Lee, and Russell Greiner: Automated feature extraction for object recognition. In Image and Vision Computing'03 New Zealand, Palmerston North, New Zealand, November 2003.
  42. Ilya Levner, Vadim Bulitko, Lihong Li, Greg Lee, and Russell Greiner: Towards automated creation of image interpretation systems. In the Sixteenth Australian Joint Conference on Artificial Intelligence, pages 653-665, Perth, Australia, December 2003.  Also appears in LNCS 2903.
  43. Min Shao, Lihong Li, Zhenkun Zheng, Chuan He, Peng Liu, Yu Chen, Zhi-Hui Du, and Sanli Li: XML and its application in clusters THNPSC-2. In Proc. Chinese Symposium on High Performance Computing and Application, Shanghai, China, 2001.

 

  • Theses
  1. Lihong Li: A Unifying Framework for Computational Reinforcement Learning Theory.  PhD dissertation, Department of Computer Science, Rutgers University, New Brunswick, NJ, USA, October, 2009. [slides]
  2. Lihong Li: Focus of Attention in Reinforcement Learning. Master's thesis, Department of Computing Science, University of Alberta, Edmonton, Alberta, Canada, July, 2004.
  3. Lihong Li: Design and Implementation of an Agent Communication Module based on KQML. Bachelor¡¯s thesis (in Chinese), Department of Computer Science and Technology, Tsinghua University, Beijing, China, July, 2002.

 

  • Book Chapters
  1. Lihong Li: Sample Complexity Bounds of Exploration. In Reinforcement Learning: State of the Art, Marco Wiering and Martijn van Otterlo (editors), Springer Verlag, March 31, 2012.
  2. Min Shao, Lihong Li, Zhenkun Zheng, and Chuan He: Practical Programming in XML. Tsinghua University Press, Beijing, China, Dec. 2002. ISBN 7-900643-85-0.
  • Workshop Papers
  1. Zhen Qin, Vaclav Petricek, Nikos Karampatziakis, Lihong Li, and John Langford: Efficiont online bootstrapping for large scale learning.  In NIPS-13 BigData Workshop, Lake Tahoe, USA, December, 2013.
  2. Lihong Li, Thomas J. Walsh, and Michael L. Littman: Knows what it knows: A framework for self-aware learning.  In Multidisciplinary Symposium on Reinforcement Learning, Montreal, Canada, June, 2009.
  3. John Asmuth, Lihong Li, Michael L. Littman, Ali Nouri, and David Wingate: PAC-MDP reinforcement learning with Bayesian priors.  In Multidisciplinary Symposium on Reinforcement Learning, Montreal, Canada, June, 2009.
  4. Ali Nouri, Michael L. Littman, Lihong Li, Ronald Parr, Christopher Painter-Wakefield, and Gavin Taylor: A novel benchmark methodology and data repository for real-life reinforcement learning.  In Multidisciplinary Symposium on Reinforcement Learning, Montreal, Canada, June, 2009.
  5. Ronald Parr, Gavin Taylor, Christopher Painter-Wakefield, Lihong Li, Michael L. Littman: Linear value function approximation and linear models.  In Multidisciplinary Symposium on Reinforcement Learning, Montreal, Canada, June, 2009.
  6. Ali Nouri, Michael L. Littman, and Lihong Li: PAC-MDP reinforcement learning with Bayesian priors.  In NIPS-08 Workshop on Model Uncertainty and Risk in Reinforcement Learning, December, 2008.
  7. Ali Nouri and Lihong Li: PAC-MDP reinforcement learning with Bayesian priors.  In New York Academy of Sciences Symposium on Machine Learning, October, 2008.  [poster]
  8. Lihong Li, Jason D. Williams, and Suhrid Balakrishnan: Fast feature selection for reinforcement-learning-based spoken dialog management: A case study.  In New York Academy of Sciences Symposium on Machine Learning, October, 2008.  [poster]
  9. Lihong Li and Thomas J. Walsh: Knows what it knows: A framework for self-aware learning.  In New York Academy of Sciences Symposium on Machine Learning, October, 2008.  [poster]  Co-winner of the Google Student Award.
  10. Lihong Li, Michael L. Littman, and Thomas J. Walsh: Knows what it knows: A framework for self-aware learning.  In European Workshop on Reinforcement Learning, France, July, 2008.  Also appeared in ICML-08.
  11. Lihong Li: Reinforcement learning via online linear regression.  In New York Academy of Sciences Symposium on Machine Learning, October, 2007.  [poster]
  12. Thomas J. Walsh, Ali Nouri, and Lihong Li: Planning and learning in environments with delays.  In New York Academy of Sciences Symposium on Machine Learning, October, 2007.  [poster]
  13. Ronald Parr, Christopher Painter-Wakefield, Lihong Li, and Michael L. Littman: Analyzing feature generation for value function approximation.  In Snowbird Learning Workshop, San Juan, Puerto Rico, March, 2007.
  14. Alexander L. Strehl, Lihong Li, Eric Wiewiora, John Langford, and Michael L. Littman: PAC model-free reinforcement learning.  In New York Academy of Sciences Symposium on Machine Learning, October, 2006.  Winner of the Best Student Paper Award.  A conference version appeared in ICML-06.
  15. Alexander L. Strehl, Lihong Li, and Michael L. Littman: PAC reinforcement learning bounds for RTDP and Rand-RTDP. In AAAI-06 Workshop on Learning for Search, Boston, MA, July, 2006. Also appeared as an AAAI technical report.
  16. Thomas J. Walsh, Lihong Li, and Michael L. Littman: Transferring state abstractions between MDPs. In ICML-06 Workshop on Structural Knowledge Transfer in Machine Learning, Pittsburgh, PA, June, 2006. [Tom¡¯s slides]
  17. Lihong Li and Jin Zhu: Algorithm description. In NIPS-05 Workshop on Reinforcement Learning Benchmarks and Bake-offs II, Whistler, BC, Canada, December, 2005.
  18. Lihong Li, Vadim Bulitko, and Russell Greiner: Focus of attention in sequential decision making. In AAAI-04 Workshop on Learning and Planning in Markov Processes --- Advances and Challenges, CA, July, 2004. Also published as an AAAI technical report.
  19. Lihong Li, Vadim Bulitko, Russell Greiner, and Ilya Levner: Automated learning distance metrics for the kNN. In NIPS-03 Workshop on Approximate Nearest Neighbors Methods for Learning and Vision, Whistler, BC, Canada, December 2003.
  20. Vadim Bulitko, Greg Lee, Ilya Levner, and Lihong Li: Open challenges in learning vision systems. In NIPS-03 Workshop on the Open Challenges in Cognitive Vision, Whistler, BC, Canada, December 2003.
  21. Vadim Bulitko, Lihong Li, Greg Lee, Russell Greiner, and Ilya Levner: Adaptive image interpretation: A spectrum of machine learning problems. In ICML-03 Workshop on the Continuum from Labeled to Unlabeled Data in Machine Learning and Data Mining, Washington D.C., US, August, 2003.

 

  • Technical/Unpublished Reports
  1. Lihong Li, Michael L. Littman, and Christopher R. Mansley: Exploration in least-squares policy iteration.   Technical report DCS-TR-641, Department of Computer Science, Rutgers University, September 2008.
  2. John Langford, Lihong Li, and Tong Zhang: Sparse online learning via truncated gradient.  In arXiv:0806.4686, June, 2008.
  3. Lihong Li and Michael L. Littman: Prioritized sweeping converges to the optimal value function.  Technical report DCS-TR-631, Department of Computer Science, Rutgers University, May 2008.
  4. Alexander L. Strehl, Lihong Li, and Michael L. Littman: PAC reinforcement learning bounds for RTDP and Rand-RTDP. In AAAI technical report WS-06-11, pages 50-56, July 2006. Also presented at an AAAI-06 workshop.
  5. Lihong Li, Michael L. Littman, and Alexander L. Strehl: A model-free reinforcement learning algorithm with low computational and sample complexity.  Technical report DCS-TR-591, Department of Computer Science, Rutgers University, December, 2005.
  6. Lihong Li and Michael L. Littman: Lazy approximation: A new approach for solving continuous finite-horizon MDPs.  Technical report DCS-TR-577, Department of Computer Science, Rutgers University, May 2005.
  7. Lihong Li, Vadim Bulitko, and Russell Greiner: Focus of attention in sequential decision making. AAAI technical report WS-04-08, pages 43-48, July 2004. Also presented at an AAAI-04 workshop.