RL^3 Summer 2005 Reading:

Abstractions and Hierarchies for Learning and Planning

We are going to cover some papers on abstractions and hierarchies for learning and planning.
A growing paper list can be found here (last updated: 2005-08-22).

Some useful links:
Papers related to option discovery (RLAI, Alberta)
Abstraction Reading Group (Stanford)

Time: 3PM, Thursday (see table below for temporary changes)

Place: Hill 482

Schedule
Date Presenter Paper Title Related Links
05-19 Carlos The MaxQ Method for Hierarchical Reinforcement Learning (ICML'98) JAIR'00 version
NIPS'99 paper
Dietterich's Talk
05-26 Lihong Between MDPs and semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning (AIJ'99) ECML'98 paper
NIPS'97 paper
ICML'95 paper
06-02 Tom Reinforcement Learning with Hierarchies of Machines (NIPS'97) Parr's thesis
Brooks' paper
06-16 Ali Discovering Hierarchy in Reinforcement Learning with HEXQ (ICML'02) Hengst's thesis
ECML'04 paper
06-23 Tom
Alex 		Q-Cut - Dynamic Discovery of Sub-goals in Reinforcement Learning (ECML'02)
Dynamic Abstraction in Reinforcement Learning via Clustering (ICML'04) 		Ratio cut
clustering
06-30 Lihong Exponentiated Gradient Methods for Reinforcement Learning (ICML'97) An EG paper
Littlestone paper
07-06 Lihong Least-Squares Temporal Difference Learning (ICML'99)
Model-Free Least-Squares Policy Iteration (NIPS'01)
Lazy Approximation for Solving Continuous Finite-Horizon MDPs (AAAI'05) 		LSTD MLJ version
LSPI JMLR version
LS for RL (SETN'02)
LSPE (DEDS'03)
07-20 Tom Interpolation-based Q-learning (ICML'04) Gordon ICML'95
07-27 Alex
Carlos 		A Theoretical Analysis of Model-Based Interval Estimation (ICML'05)
Automatic Discovery of Subgoals in Reinforcement Learning using Diverse Density (ICML'01) 		McGovern's thesis
08-04 Michael Multi-Value-Functions: Efficient Automatic Action Hierarchies for Multiple Goal MDPs (IJCAI'99) HDG paper
08-11 Ali State Abstraction Discovery from Irrelevant State Variables (IJCAI'05) McCallum's thesis
SMDP Homomorphism
08-18 Tom Distributed Planning in Hierarchical Factored MDPs (UAI'02) GKP paper
Ch4 of Gordon's thesis
08-25 Lihong Metrics for Finite Markov Decision Processes (UAI'04) Bisimulation (AIJ'03)
Homomorphism (IJCAI'03)
08-29 Nick Jong State Abstraction Discovery from Irrelevant State Variables (IJCAI'05)


For any comments or questions, please email me.
Last revised: 2005-08-29