Lihong Li

[Interests] [Education] [Publications] [Work Experience] [Software/Datasets] [Professional Activities] [Teaching Experience]

Interests

In general, I am interested in making intelligent computer programs. Much of my research focuses on machine learning that involves interaction of various types. This includes reinforcement learning, multi-armed bandits, online learning, active learning, and their numerous applications on the Internet.

---

Education

• 01/2005 - 05/2009, PhD, Computer Science, Rutgers University, New Brunswick, NJ, USA
  • Advisor: Michael L. Littman
  • Thesis: A Unifying Framework for Computational Reinforcement Learning Theory
• 09/2002 - 07/2004, M.Sc., Computing Science, University of Alberta, Edmonton, AB, Canada
  • Advisors: Vadim Bulitko and Russell Greiner
  • Thesis: Focus of Attention in Reinforcement Learning
• 09/1998 - 07/2002, B.Eng., Computer Science and Technology, Tsinghua University, Beijing, P.R. China

---

Work Experience

• 06/2012 - present, Researcher at Microsoft Research
• 09/2010 - 06/2012, Research Scientist at Yahoo! Research
• 06/2009 - 08/2010, Postdoctoral Scientist at Yahoo! Research
• 05/2008 - 08/2008, Research Intern at AT&T Shannon Labs (mentored by Jason D. Williams and Suhrid Balakrishnan)
• 05/2007 - 08/2007, Research Intern at Yahoo! Research NYC (mentored by John Langford)
• 05/2006 - 08/2006, Engineer Intern at Google NYC (mentored by Lee Laroco)
• 01/2005 - 06/2009, Graduate Research Assistant at the Rutgers University
• 09/2002 - 07/2004, Teaching and Research Assistant at the University of Alberta

---

Software and Datasets

• Vowpal Wabbit, a C++ implementation of fast online-learning algorithms (started with John Langford and Alex Strehl)
• R6: Yahoo! Front Page Today Module User Click Log Dataset, the first large-scale dataset from real-life data that supports unbiased evaluation of multi-armed bandit algorithms (with Wei Chu)

---

Professional Activities

• Reviewer for journals:
  • ACM Transactions on Intelligent Systems and Technology (2013)
  • Advances in Complex Systems (2011)
  • Artificial Intelligence (2010 - present)
  • Artificial Intelligence Communications (2009)
  • Data Mining and Knowledge Discovery (2008)
  • IEEE Transactions on Automatic Control (2011)
  • IEEE Transactions on Knowledge and Data Engineering (2012)
  • IEEE Transactions on Neural Networks (2010)
  • IEEE Transactions on Wireless Communications (2009)
  • Journal of Artificial Intelligence Research (2008 - present)
  • Journal of Computer Science and Technology (2009)
  • Journal of Machine Learning Research (2005 - present)
  • Machine Learning (2009 - present)
  • Neural Computation (2009)
  • Neurocomputing (2009)
• Reviewer or senior program committee member or area chair for conferences:
  • AAAI: 2006, 2008, 2010
  • AISTATS: 2011
  • COLT: 2010, 2011, 2012
  • ECML: 2009
  • KDD: 2012
  • ICML: 2009, 2010, 2011, 2012 (AC), 2013 (AC)
  • IJCAI: 2007, 2011 (SPC)
  • NIPS: 2008, 2009, 2010, 2011, 2012
  • UAI: 2010
  • UbiComp: 2011
  • WSDM: 2012, 2013
  • WWW: 2012
• Program committeee for workshops:
  • IJCAI-05 workshop on planning and learning
  • ICML-09 workshop on abstraction in RL
  • NIPS-11 workshop on Bayesian optimization, experimental design and bandits
  • ICML-12 workshop on online advertising
  • NIPS-12 workshop on Bayesian optimization and decision making
• Workshop organizers:
  • 2009 Reinforcement Learning Competition (ICML/UAI/COLT'09).
  • PASCAL2 Exploration \& Exploitation Challenge (ICML'12)
  • Workshop on large-scale online learning and decision making (Cumberland Lodge, 2012)

---

Teaching Experience

• In Spring 2009, I gave a guest lecture on least-squares policy iteration in Michael Littman's graduate class, "Learning and Sequential Decision Making"
• In Fall 2007, I helped organize a light seminar on Planning in Learned Environments
• In Summer 2005, I organized a seminar on Abstraction and Hierarchies for Learning and Planning
• From Fall 2002 to Winter 2004, I was a teaching assistant for discrete mathematics courses (Formal Systems and Logic in Computer Science), and won a University of Alberta UTS Teaching Award in 2004

---

Visitors:

---

Created: 12/22/2007
Last modified: 01/24/2013