Oct 21, 1999: R.S. Sutton, D. McAllester, S. Singh and Y. Mansour: "Policy Gradient Methods for Reinforcement Learning with Function Approximation". Advances in Neural Information Processing Systems 11 (NIPS), 1999 (compressed postscript, 7 pages).
Oct 28, 1999: P.L. Lanzi and S.W. Wilson: "Optimal classifier system performance in non-Markov environments". Technical Report N. 99.36, Politecnico di Milano, 1999 (pdf, 26 pages).
Nov 4, 1999: "Variable Resolution Discretization in Optimal Control", a talk by Remi Munos.
Nov 18, 1999: J. Hu and M.P. Wellman: "Multiagent reinforcement learning in stochastic games", submitted, 1999 (postscript, 36 pages) (see an abstract of the discussion).
Dec 2, 1999: A.Y. Ng, D. Harada, and S. Russell, "Policy invariance under reward transformations: Theory and application to reward shaping". Machine Learning: Proceedings of the Sixteenth International Conference (ICML), 1999 (postscript, 10 pages).
Dec 9, 1999: J.A. Boyan: "Least-Squares Temporal Difference Learning". Machine Learning: Proceedings of the Sixteenth International Conference (ICML), 1999 (postscript, 8 pages).