Publications of Lihong Li

My DBLP Entry

  • Journal Papers
  1. Alexander L. Strehl, Lihong Li, and Michael L. Littman: Reinforcement learning in finite MDPs: PAC analysis.  In the Journal of Machine Learning Research, 10:2413-2444, 2009.
  2. Emma Brunskill, Bethany R. Leffler, Lihong Li, Michael L. Littman, and Nichlos Roy: Provably efficient learning with typed parametric models . In the  Journal of Machine Learning Research, 10:1955-1988, 2009.
  3. John Langford, Lihong Li, Yevgeniy Vorobeychik, and Jennifer Wortman: Maintaining equilibria during exploration in sponsored search auctions.  To appear in Algorithmica (Special Issue on WINE-07), in press.
  4. John Langford, Lihong Li, and Tong Zhang: Sparse online learning via truncated gradient.  In the Journal of Machine Learning Research, 10:777-801, 2009.
  5. Thomas J. Walsh, Ali Nouri, Lihong Li, and Michael L. Littman: Planning and learning in environments with delayed feedback.  In the Journal of Autonomous Agents and Multi-Agent Systems, 18(1):83-105, 2009.  A preliminary version appeared in ECML-07.
  6. Lihong Li, Vadim Bulitko, and Russell Greiner: Focus of attention in reinforcement learning.  In the Journal of Universal Computer Science, 13(9):1246-1269, November, 2007.
  7. Lihong Li, Min Shao, Zhenkun Zheng, Chuan He, and Zhi-Hui Du: Typical XML document transformation methods and an application system. In Computer Science, 30(2): 40-44, China, 2003.

 

  • Conference
  1. Lihong Li, Jason D. Williams, and Suhrid Balakrishnan: Reinforcement learning for dialog management using least-squares policy iteration and fast feature selection.  In INTERSPEECH-09, Brighton, UK, September, 2009.
  2. Carlos Diuk, Lihong Li, and Bethany R. Leffler: The adaptive k-meteorologists problem and its application to structure learning and feature selection in reinforcement learning.  In ICML-09, Montreal, Canada, June, 2009.
  3. John Asmuth, Lihong Li, Michael L. Littman, Ali Nouri, and David Wingate: A Bayesian sampling approach to exploration in reinforcement learning.  In UAI-09, Montreal, Canada, June, 2009.
  4. Lihong Li, Michael L. Littman, and Christopher R. Mansley: Online exploration in least-squares policy iteration.  In AAMAS-09, Budapest, Hungary, May, 2009.
  5. John Langford, Lihong Li, and Tong Zhang: Sparse online learning via truncated gradient.  In Advances in Neural Information Processing Systems 21 (NIPS-08), 905-912, 2009. [poster][spotlight]
  6. Lihong Li: A worst-case comparison between temporal difference and residual gradient with linear function approximation.  In ICML-08, Helsinki, Finland, July, 2008. [slides][poster]
  7. Lihong Li, Michael L. Littman, and Thomas J. Walsh: Knows what it knows: A framework for self-aware learning.  In ICML-08, Helsinki, Finland, July, 2008. [slides][poster][video] Co-winner of the Best Student Paper Award.
  8. Ronald Parr, Lihong Li, Gavin Taylor, Christopher Painter-Wakefield, and Michael L. Littman: An analysis of linear models, linear value-function approximation, and feature selection for reinforcement learning.  In ICML-08, Helsinki, Finland, July, 2008. [Ron’s slides][poster]
  9. Emma Brunskill, Bethany R. Leffler, Lihong Li, Michael L. Littman, and Nicholas Roy: CORL: A continuous-state offset-dynamics reinforcement learner.  In UAI-08, Helsinki, Finland, July, 2008. [Emma’s slides]
  10. Lihong Li and Michael L. Littman: Efficient value-function approximation via online linear regression.  In AI&Math-08, Fort Lauderdale, FL, January, 2008.  [slides]
  11. Jennifer Wortman, Yevgeniy Vorobeychik, Lihong Li, and John Langford: Maintaining equilibria during exploration in sponsored search auctions.  In WINE-07, San Diego, CA, December, 2007.  Also appears in LNCS 4858.  A longer version with proofs is here.  [Jenn’s slides].
  12. Thomas J. Walsh, Ali Nouri, Lihong Li, and Michael L. Littman: Planning and learning in environments with delayed feedback.  In ECML-07, Warsaw, Poland, September, 2007.  Also appears in LNCS 4701. [Tom’s slides]
  13. Ronald Parr, Christopher Painter-Wakefield, Lihong Li, and Michael L. Littman: Analyzing feature generation for value-function approximation.  In ICML-07, Corvallis, OR, June, 2007. [poster]
  14. Alexander L. Strehl, Lihong Li, Eric Wiewiora, John Langford, and Michael L. Littman: PAC model-free reinforcement learning.  In ICML-06, Pittsburgh, PA, June, 2006. [slides]
  15. Alexander L. Strehl, Lihong Li, and Michael L. Littman: Incremental model-based learners with formal learning-time guarantees.  In UAI-06, Cambridge, MA, July, 2006.
  16. Lihong Li, Thomas J. Walsh, and Michael L. Littman: Towards a unified theory of state abstraction for MDPs.  In AI&Math-06, Fort Lauderdale, FL, January, 2006. [Tom’s slides]
  17. Lihong Li and Michael L. Littman: Lazy approximation for solving continuous finite-horizon MDPs.  In AAAI-05, pages 1175-1180, Pittsburgh, PA, 2005. [slides]
  18. Lihong Li, Vadim Bulitko, and Russell Greiner: Batch reinforcement learning with state importance. In ECML-04, pages 566-568, Pisa, Italy, 2004.  Also appears in LNCS 3201. [poster]
  19. Vadim Bulitko, Lihong Li, Russell Greiner, and Ilya Levner: Lookahead pathologies for single agent search. In IJCAI-03, pages 1531-1533, Acapulco, Mexico, August, 2003. [poster]
  20. Lihong Li, Vadim Bulitko, Russell Greiner, and Ilya Levner: Improving an adaptive image interpretation system by leveraging. In the Eighth Australian and New Zealand Conference on Intelligent Information Systems, Sydney, Australia, December 2003.
  21. Ilya Levner, Vadim Bulitko, Lihong Li, Greg Lee, and Russell Greiner: Learning robust object recognition strategies. In the Eighth Australian and New Zealand Conference on Intelligent Information Systems, Sydney, Australia, December 2003.
  22. Ilya Levner, Vadim Bulitko, Lihong Li, Greg Lee, and Russell Greiner: Automated feature extraction for object recognition. In Image and Vision Computing'03 New Zealand, Palmerston North, New Zealand, November 2003.
  23. Ilya Levner, Vadim Bulitko, Lihong Li, Greg Lee, and Russell Greiner: Towards automated creation of image interpretation systems. In the Sixteenth Australian Joint Conference on Artificial Intelligence, pages 653-665, Perth, Australia, December 2003.  Also appears in LNCS 2903.
  24. Min Shao, Lihong Li, Zhenkun Zheng, Chuan He, Peng Liu, Yu Chen, Zhi-Hui Du, and Sanli Li: XML and its application in clusters THNPSC-2. In Proc. Chinese Symposium on High Performance Computing and Application, Shanghai, China, 2001.

 

  • Theses
  1. Lihong Li: A Unifying Framework for Computational Reinforcement Learning Theory.  PhD dissertation, Department of Computer Science, Rutgers University, New Brunswick, NJ, USA, October, 2009. [slides]
  2. Lihong Li: Focus of Attention in Reinforcement Learning. Master's thesis, Department of Computing Science, University of Alberta, Edmonton, Alberta, Canada, July, 2004.
  3. Lihong Li: Design and Implementation of an Agent Communication Module based on KQML. Bachelor隆炉s thesis (in Chinese), Department of Computer Science and Technology, Tsinghua University, Beijing, China, July, 2002.

 

  • Book Chapters
  1. Min Shao, Lihong Li, Zhenkun Zheng, and Chuan He: Practical Programming in XML. Tsinghua University Press, Beijing, China, Dec. 2002. ISBN 7-900643-85-0.

 

  • Workshop Papers
  1. Lihong Li, Thomas J. Walsh, and Michael L. Littman: Knows what it knows: A framework for self-aware learning.  In Multidisciplinary Symposium on Reinforcement Learning, Montreal, Canada, June, 2009.
  2. John Asmuth, Lihong Li, Michael L. Littman, Ali Nouri, and David Wingate: PAC-MDP reinforcement learning with Bayesian priors.  In Multidisciplinary Symposium on Reinforcement Learning, Montreal, Canada, June, 2009.
  3. Ali Nouri, Michael L. Littman, Lihong Li, Ronald Parr, Christopher Painter-Wakefield, and Gavin Taylor: A novel benchmark methodology and data repository for real-life reinforcement learning.  In Multidisciplinary Symposium on Reinforcement Learning, Montreal, Canada, June, 2009.
  4. Ronald Parr, Gavin Taylor, Christopher Painter-Wakefield, Lihong Li, Michael L. Littman: Linear value function approximation and linear models.  In Multidisciplinary Symposium on Reinforcement Learning, Montreal, Canada, June, 2009.
  5. Ali Nouri, Michael L. Littman, and Lihong Li: PAC-MDP reinforcement learning with Bayesian priors.  In NIPS-08 Workshop on Model Uncertainty and Risk in Reinforcement Learning, December, 2008.
  6. Ali Nouri and Lihong Li: PAC-MDP reinforcement learning with Bayesian priors.  In New York Academy of Sciences Symposium on Machine Learning, October, 2008.  [poster]
  7. Lihong Li, Jason D. Williams, and Suhrid Balakrishnan: Fast feature selection for reinforcement-learning-based spoken dialog management: A case study.  In New York Academy of Sciences Symposium on Machine Learning, October, 2008.  [poster]
  8. Lihong Li and Thomas J. Walsh: Knows what it knows: A framework for self-aware learning.  In New York Academy of Sciences Symposium on Machine Learning, October, 2008.  [poster]  Co-winner of the Google Student Award.
  9. Lihong Li, Michael L. Littman, and Thomas J. Walsh: Knows what it knows: A framework for self-aware learning.  In European Workshop on Reinforcement Learning, France, July, 2008.  Also appeared in ICML-08.
  10. Lihong Li: Reinforcement learning via online linear regression.  In New York Academy of Sciences Symposium on Machine Learning, October, 2007.  [poster]
  11. Thomas J. Walsh, Ali Nouri, and Lihong Li: Planning and learning in environments with delays.  In New York Academy of Sciences Symposium on Machine Learning, October, 2007.  [poster]
  12. Ronald Parr, Christopher Painter-Wakefield, Lihong Li, and Michael L. Littman: Analyzing feature generation for value function approximation.  In Snowbird Learning Workshop, San Juan, Puerto Rico, March, 2007.
  13. Alexander L. Strehl, Lihong Li, Eric Wiewiora, John Langford, and Michael L. Littman: PAC model-free reinforcement learning.  In New York Academy of Sciences Symposium on Machine Learning, October, 2006.  Winner of the 隆掳Best Student Paper隆卤 Award.  A conference version appeared in ICML-06.
  14. Alexander L. Strehl, Lihong Li, and Michael L. Littman: PAC reinforcement learning bounds for RTDP and Rand-RTDP. In AAAI-06 Workshop on Learning for Search, Boston, MA, July, 2006. Also appeared as an AAAI technical report.
  15. Thomas J. Walsh, Lihong Li, and Michael L. Littman: Transferring state abstractions between MDPs. In ICML-06 Workshop on Structural Knowledge Transfer in Machine Learning, Pittsburgh, PA, June, 2006. [Tom隆炉s slides]
  16. Lihong Li and Jin Zhu: Algorithm description. In NIPS-05 Workshop on Reinforcement Learning Benchmarks and Bake-offs II, Whistler, BC, Canada, December, 2005.
  17. Lihong Li, Vadim Bulitko, and Russell Greiner: Focus of attention in sequential decision making. In AAAI-04 Workshop on Learning and Planning in Markov Processes --- Advances and Challenges, CA, July, 2004. Also published as an AAAI technical report.
  18. Lihong Li, Vadim Bulitko, Russell Greiner, and Ilya Levner: Automated learning distance metrics for the kNN. In NIPS-03 Workshop on Approximate Nearest Neighbors Methods for Learning and Vision, Whistler, BC, Canada, December 2003.
  19. Vadim Bulitko, Greg Lee, Ilya Levner, and Lihong Li: Open challenges in learning vision systems. In NIPS-03 Workshop on the Open Challenges in Cognitive Vision, Whistler, BC, Canada, December 2003.
  20. Vadim Bulitko, Lihong Li, Greg Lee, Russell Greiner, and Ilya Levner: Adaptive image interpretation: A spectrum of machine learning problems. In ICML-03 Workshop on the Continuum from Labeled to Unlabeled Data in Machine Learning and Data Mining, Washington D.C., US, August, 2003.

 

  • Technical/Unpublished Reports
  1. Lihong Li, Michael L. Littman, and Christopher R. Mansley: Exploration in least-squares policy iteration.   Technical report DCS-TR-641, Department of Computer Science, Rutgers University, September 2008.
  2. John Langford, Lihong Li, and Tong Zhang: Sparse online learning via truncated gradient.  In arXiv:0806.4686, June, 2008.
  3. Lihong Li and Michael L. Littman: Prioritized sweeping converges to the optimal value function.  Technical report DCS-TR-631, Department of Computer Science, Rutgers University, May 2008.
  4. Alexander L. Strehl, Lihong Li, and Michael L. Littman: PAC reinforcement learning bounds for RTDP and Rand-RTDP. In AAAI technical report WS-06-11, pages 50-56, July 2006. Also presented at an AAAI-06 workshop.
  5. Lihong Li, Michael L. Littman, and Alexander L. Strehl: A model-free reinforcement learning algorithm with low computational and sample complexity.  Technical report DCS-TR-591, Department of Computer Science, Rutgers University, December, 2005.
  6. Lihong Li and Michael L. Littman: Lazy approximation: A new approach for solving continuous finite-horizon MDPs.  Technical report DCS-TR-577, Department of Computer Science, Rutgers University, May 2005.
  7. Lihong Li, Vadim Bulitko, and Russell Greiner: Focus of attention in sequential decision making. AAAI technical report WS-04-08, pages 43-48, July 2004. Also presented at an AAAI-04 workshop.