Adaptive Problem Framing and Long-life, Life-long Learning:

This talk is about the challenges of building and analyzing agents embedded in complex environments: agents that are expected to live long, productive lives in environments that expose them to varied experience. In particular, I will be talking about the advantages and disadvantages of restricted computational architectures in building and analyzing embedded agents, how agents appear relatively static or adaptively constrained and environments relatively dynamic (nonstationary, technically) against the background of a long and varied life, models, data and the constant need for reformulating the decision-making problem that the agent is faced with, the difficulties of trying to amortize computational effort over a lifetime of experiences, how a collection of hacks can suddenly appear quite elegant when judged in the context of a fixed architecture and limited computational resources, how certain properties of the environment dictate different computational strategies and how agents might test for these properties in the process of exploring their environment, how exploration and rehearsal constitute forms of programming, and how the quest for a universal bias flies in the face of intuition and technical results such as the so-called no-free-lunch theorems. In the process of explaining why I'm interested in these challenges and quandaries I'll briefly recap a decade of work, justifying some threads of research, apologizing for others, but mainly illustrating how I've come to consider these challenges important and interesting for myself and for the field of artificial intelligence.