Abstract Reinforcement learning (RL) consists of methods that automatically adjust behaviour based on numerical rewards and penalties. While use of the attribute-value framework is widespread in RL, it has limited expressive power. Logic languages, such as first-order logic, provide a more expressive framework, and their use in RL has led to the field of relational RL. This thesis develops a system for relational RL based on learning classifier systems (LCS). In brief, the system generates, evolves, and evaluates a population of condition-action rules, which take the form of definite clauses over first-order logic. Adopting the LCS approach allows the resulting system to integrate several desirable qualities: model-free and “tabula rasa” learning; a Markov Decision Process problem model; and importantly, support for variables as a principal mechanism for generalisation. The utility of variables is demonstrated by the system’s ability to learn genuinely scalable behaviour – ! behaviour learnt in small environments that translates to arbitrarily large versions of the environment without the need for retraining.
I am pleased to share with you that the Journal of Artificial Evolution and Applications has recently published my LCS Review paper entitled, “Learning Classifier Systems: A Complete Introduction, Review, and Roadmap”. I wrote this from the perspective of a non-computer scientist, to introduce the basic LCS concept, as well as the variation represented in different LCS implementations that have been tasked to different problem domains. It was my goal and hope that this review might provide a reasonable starting point for outsiders interested in understanding or getting involved in the LCS community. This paper may be viewed using the following link: Thanks! I enjoyed listening to the many excellent GBML talks given at GECCO this year.
Below you may find the slides of the GECCO 2009 tutorial that Jaume Bacardit and I put together. Hope you enjoy it.
Slides
Abstract
We are living in the peta-byte era.We have larger and larger data to analyze, process and transform into useful answers for the domain experts. Robust data mining tools, able to cope with petascale volumes […]
We are living in the peta-byte era.We have larger and larger data to analyze, process and transform into useful answers for the domain experts. Robust data mining tools, able to cope with petascale volumes and/or high dimensionality producing human-understandable solutions are key on several domain areas. Genetics-based machine learning (GBML) techniques are perfect candidates for this task, among others, due to the recent advances in representations, learning paradigms, and theoretical modeling. If evolutionary learning techniques aspire to be a relevant player in this context, they need to have the capacity of processing these vast amounts of data and they need to process this data within reasonable time. Moreover, massive computation cycles are getting cheaper and cheaper every day, allowing researchers to have access to unprecedented parallelization degrees. Several topics are interlaced in these two requirements: (1) having the proper learning paradigms and knowledge representations, (2) understanding them and knowing when are they suitable for the problem at hand, (3) using efficiency enhancement techniques, and (4) transforming and visualizing the produced solutions to give back as much insight as possible to the domain experts are few of them.
This tutorial will try to answer this question, following a roadmap that starts with the questions of what large means, and why large is a challenge for GBML methods. Afterwards, we will discuss different facets in which we can overcome this challenge: Efficiency enhancement techniques, representations able to cope with large dimensionality spaces, scalability of learning paradigms. We will also review a topic interlaced with all of them: how can we model the scalability of the components of our GBML systems to better engineer them to get the best performance out of them for large datasets. The roadmap continues with examples of real applications of GBML systems and finishes with an analysis of further directions.
After coming back from GECCO I just uploaded the last of the NIGEL 2006 talks at LCS & GBML Central. This last talk was by Jaume Bacardit and GBML for protein structure prediction.
Related posts:NIGEL 2006 Part V: Bernardó vs. LanziNIGEL 2006 Part IV: Llorà vs. CasillasNIGEL 2006 Part III: Butz vs. Barry
After coming back from GECCO I just uploaded the last of the NIGEL 2006 talks at LCS & GBML Central. This last talk was by Jaume Bacardit and GBML for protein structure prediction.
After the vacation break, two more NIGEL 2006 talks are available at LCS & GBML Central. This week Ester Bernardó presents how LCS can perform in the presence of class imbalance, whereas Lanzi continues his quest on computed predictions.
Related posts:NIGEL 2006 Part IV: Llorà vs. CasillasTranscoding NIGEL 2006 videosNIGEL 2006 Part III: Butz […]
After the vacation break, two more NIGEL 2006 talks are available at LCS & GBML Central. This week Ester Bernardó presents how LCS can perform in the presence of class imbalance, whereas Lanzi continues his quest on computed predictions.
Two more NIGEL 2006 talks are available at LCS & GBML Central. This week Xavier Llorà presents how linkage learning can be achieve in Pittsburgh LCS, whereas Jorge Casillas reviews his work using XCS and Fuzzy LCS.
Related posts:NIGEL 2006 Part III: Butz vs. BarryNIGEL 2006 Part II: Dasgupta vs. BookerNIGEL 2006 Part V: […]
Two more NIGEL 2006 talks are available at LCS & GBML Central. This week Xavier Llorà presents how linkage learning can be achieve in Pittsburgh LCS, whereas Jorge Casillas reviews his work using XCS and Fuzzy LCS.
