Reinforcement Learning, Logic and Evolutionary Computation

Drew Mellor is pleased to announce the publication of his new LCS book. Reinforcement Learning, Logic and Evolutionary Computation: A Learning Classifier System Approach to Relational Reinforcement Learning, published by Lambert Academic Publishing (ISBN 978-3-8383-0196-9).

Abstract Reinforcement learning (RL) consists of methods that automatically adjust behaviour based on numerical rewards and penalties. While use of the attribute-value framework is widespread in RL, it has limited expressive power. Logic languages, such as first-order logic, provide a more expressive framework, and their use in RL has led to the field of relational RL. This thesis develops a system for relational RL based on learning classifier systems (LCS). In brief, the system generates, evolves, and evaluates a population of condition-action rules, which take the form of definite clauses over first-order logic. Adopting the LCS approach allows the resulting system to integrate several desirable qualities: model-free and “tabula rasa” learning; a Markov Decision Process problem model; and importantly, support for variables as a principal mechanism for generalisation. The utility of variables is demonstrated by the system’s ability to learn genuinely scalable behaviour – ! behaviour learnt in small environments that translates to arbitrarily large versions of the environment without the need for retraining.

New book Essentials of Metaheuristics by Sean Luke available online

A new book Essentials of Metaheuristics by Sean Luke is available online. The book can be downloaded for free on its web site. Information about the book from the author’s web site:

This is an open set of lecture notes on metaheuristics algorithms, intended for undergraduate students, practitioners, programmers, and other non-experts. It was developed as […]

A new book Essentials of Metaheuristics by Sean Luke is available online. The book can be downloaded for free on its web site. Information about the book from the author’s web site:

This is an open set of lecture notes on metaheuristics algorithms, intended for undergraduate students, practitioners, programmers, and other non-experts. It was developed as a series of lecture notes for an undergraduate course I taught at GMU. The chapters are designed to be printable separately if necessary. As it’s lecture notes, the topics are short and light on examples and theory. It’s best when complementing other texts. With time, I might remedy this.

Slides from my GECCO-2009 presentations

I just put the slides from my GECCO-2009 presentations online both on the MEDAL Publications page and on the slideshare.net. The slideshare.net versions are embedded below:
Initial-Population Bias in the Univariate Estimation of Distribution Algorithm
View more documents from pelikan.

Performance of Evolutionary Algorithms on NK Landscapes with Nearest Neighbor Interactions and Tunable Overlap
View more documents from pelikan.

Analysis […]

I just put the slides from my GECCO-2009 presentations online both on the MEDAL Publications page and on the slideshare.net. The slideshare.net versions are embedded below:

New issue of SIGEVOlution available now

The new issue of SIGEVOlution is now available for you to download from http://www.sigevolution.org. For me, the main highlight of the issue is the interview with John H. Holland with an introduction by Lashon Booker.

The new issue of SIGEVOlution is now available for you to download from http://www.sigevolution.org. For me, the main highlight of the issue is the interview with John H. Holland with an introduction by Lashon Booker.

John Holland to give a keynote at GECCO-2009 in Montreal, Canada

John H. Holland will give a keynote speech at GECCO-2009 on July 12, 2009 (Sunday), 10:40am-11:40am. The talk is entitled Genetic Algorithms: Long Ago [Past] and Far Away [Future] and the abstract of the talk follows:

It was in the mid-50’s of the 20th century when I realized that Fisher’s fundamental theorem could be extended […]

John H. Holland

John H. Holland will give a keynote speech at GECCO-2009 on July 12, 2009 (Sunday), 10:40am-11:40am. The talk is entitled Genetic Algorithms: Long Ago [Past] and Far Away [Future] and the abstract of the talk follows:

It was in the mid-50’s of the 20th century when I realized that Fisher’s fundamental theorem could be extended from individual alleles to co-adapted sets of alleles, without linearization. That led to a realization that recombination, rather than mutation, was the main mechanism providing grist for the natural selection mill. There was little theory concerning recombination in those days, but now recombination is a standard explanation for biological innovations, such as swine flu.

Much later, in the early 1990’s, GA’s provided the “adaptive” part of rule-based models of complex adaptive systems (CAS), such as the artificial stock market pioneered at the Santa Fe Institute. Tag-based signal processing occurs in systems as different as biological cells, language acquisition, and ecosystems. CAS models offer a unified way to study the on-going co-evolution of boundary and tag networks in these systems.

Another keynote speaker at GECCO-2009 is Demetri Terzopoulos, who will give the talk Artificial Life Simulation of Humans and Lower Animals: From Biomechanics to Intelligence on July 11 (Saturday) at 4.10pm-5.50pm. As if this wasn’t enough, GECCO-2009 will also feature an invited talk of Hans-Paul Schwefel at the Learning from Failures in Evolutionary Computation (LFFEC) Workshop, which is entitled Failures as stepping stones to success or per aspera ad astra.

More details can be found on GECCO-2009 webpage.

Highlight from Simulated Car Racing Competition at CEC-2009

Pier Luca Lanzi just posted a few video highlights from the Simulated Car Racing Competition at CEC-2009. The winner of the competition was Thies Lonneker and Martin Butz. Congratulations!
Here’s one of the posted videos:

See also Pier Luca’s post at IlliGAL Blogging.

Pier Luca Lanzi just posted a few video highlights from the Simulated Car Racing Competition at CEC-2009. The winner of the competition was Thies Lonneker and Martin Butz. Congratulations!

Here’s one of the posted videos:

See also Pier Luca’s post at IlliGAL Blogging.

Hans-Paul Schwefel to give a talk at GECCO-2009

I just found out that Hans-Paul Schwefel, one of the evolutionary computation pioneers, is going to give a talk at the Genetic and Evolutionary Computation Conference (GECCO-2009) in Montreal, Canada (July 8-12, 2009). The talk will be part of the Learning from Failures in Evolutionary Computation (LFFEC) Workshop.
The title of the talk is failures […]

Hans-Paul Schwefel

I just found out that Hans-Paul Schwefel, one of the evolutionary computation pioneers, is going to give a talk at the Genetic and Evolutionary Computation Conference (GECCO-2009) in Montreal, Canada (July 8-12, 2009). The talk will be part of the Learning from Failures in Evolutionary Computation (LFFEC) Workshop.

The title of the talk is failures as stepping stones to success or per aspera ad astra. The abstract follows:

The implicit thesis of this talk’s title will be underpinned with some examples from (my) real life. A first example leads back to the 1960s, when I simulated the (1+1)-ES with discrete mutations on a two-dimensional parabolic ridge by means of a Z23 computer. The result – getting stuck in certain search directions – led to making use of Gaussian variations. The second example comes from experimental investigations to determine the shape of a hot water flashing nozzle, the water being really hot and not simulated on a computer. In search for a multimembered evolutionary algorithm with effective self-adaptation of the mutation strengths, a couple of failures occurred. These, however, rendered deep insight into basic prerequisites to achieve the goal. And finally, some theory will be re-presented about the optimal failure rate in two black-box situations.

Of course, GECCO-2009 will feature many other interesting presentations, workshops, and other events and for more information about this conference, you should visit its web page here. GECCO is organized by ACM SIGEVO (Special Interest Group on Genetic and Evolutionary Computation).

GECCO ranks 11th among conferences in AI/ML

According to the estimated impact factor, GECCO (Genetic and Evolutionary Computation Conference) ranks 11th in computer science conferences on artificial intelligence, machine learning, and human-computer interaction. The source: http://www.cs-conference-ranking.org/conferencerankings/topicsii.html.

According to the estimated impact factor, GECCO (Genetic and Evolutionary Computation Conference) ranks 11th in computer science conferences on artificial intelligence, machine learning, and human-computer interaction. The source: http://www.cs-conference-ranking.org/conferencerankings/topicsii.html.