Drew Mellor is pleased to announce the publication of his new LCS book. Reinforcement Learning, Logic and Evolutionary Computation: A Learning Classifier System Approach to Relational Reinforcement Learning, published by Lambert Academic Publishing (ISBN 978-3-8383-0196-9).
Abstract Reinforcement learning (RL) consists of methods that automatically adjust behaviour based on numerical rewards and penalties. While use of the attribute-value framework is widespread in RL, it has limited expressive power. Logic languages, such as first-order logic, provide a more expressive framework, and their use in RL has led to the field of relational RL. This thesis develops a system for relational RL based on learning classifier systems (LCS). In brief, the system generates, evolves, and evaluates a population of condition-action rules, which take the form of definite clauses over first-order logic. Adopting the LCS approach allows the resulting system to integrate several desirable qualities: model-free and “tabula rasa” learning; a Markov Decision Process problem model; and importantly, support for variables as a principal mechanism for generalisation. The utility of variables is demonstrated by the system’s ability to learn genuinely scalable behaviour – ! behaviour learnt in small environments that translates to arbitrarily large versions of the environment without the need for retraining.