Genetic Programming With Niching for Uncertain Capacitated Arc Routing Problem

The uncertain capacitated arc routing problem is an important optimization problem with many real-world applications. Genetic programming is considered a promising hyper-heuristic technique to automatically evolve routing policies that can make effecti…

The uncertain capacitated arc routing problem is an important optimization problem with many real-world applications. Genetic programming is considered a promising hyper-heuristic technique to automatically evolve routing policies that can make effective real-time decisions in an uncertain environment. Most existing research on genetic programming hyper-heuristic for the uncertain capacitated arc routing problem only focused on the test performance aspect. As a result, the routing policies evolved by genetic programming are usually too large and complex, and hard to comprehend. To evolve effective, smaller, and simpler routing policies, this article proposes a novel genetic programming approach, which simplifies the routing policies during the evolutionary process using a niching technique. The simplified routing policies are stored in an external archive. We also developed new elitism, parent selection, and breeding schemes for generating offspring from the original population and the archive. The experimental results show that the newly proposed approach can achieve significantly better test performance than the current state-of-the-art genetic programming algorithms for the uncertain capacitated arc routing problem. The evolved routing policies are smaller, and thus potentially more interpretable.

Self-Adjusting Multitask Particle Swarm Optimization

Particle swarm optimization algorithm has become a promising approach in solving multitask optimization (MTO) problems since it can transfer knowledge with easy implementation and high searching efficiency. However, in the process of knowledge transfer…

Particle swarm optimization algorithm has become a promising approach in solving multitask optimization (MTO) problems since it can transfer knowledge with easy implementation and high searching efficiency. However, in the process of knowledge transfer, negative transfer is common because it is difficult to evaluate whether knowledge is effective for population evolution. Therefore, how to obtain and transfer the effective knowledge to curb the negative transfer is a challenging problem in MTO. To deal with this problem, a self-adjusting multitask particle swarm optimization (SA-MTPSO) algorithm is designed to improve the convergence performance in this article. First, a knowledge estimation metric, combining the decision space knowledge and the target space knowledge for each task, is designed to describe the effectiveness of knowledge. Then, the effective knowledge is obtained to promote the knowledge transfer process. Second, a self-adjusting knowledge transfer mechanism, based on the effective knowledge and the self-adjusting transfer method, is developed to achieve effective knowledge transfer. Then, the ineffective knowledge is removed to solve the negative transfer problem. Third, the convergence analysis is given to guarantee the effectiveness of the SA-MTPSO algorithm theoretically. Finally, the proposed algorithm is compared with some existing MTO algorithms. The results show that the performance of the proposed algorithm is superior to most algorithms on negative transfer suppression and convergence.

Frames-of-Reference-Based Learning: Overcoming Perceptual Aliasing in Multistep Decision-Making Tasks

Perceptual aliasing challenges reinforcement learning agents. They struggle to learn stable policies by failing to identify and disambiguate perceptually identical states in the environment that require different actions to reach a goal. As the agent o…

Perceptual aliasing challenges reinforcement learning agents. They struggle to learn stable policies by failing to identify and disambiguate perceptually identical states in the environment that require different actions to reach a goal. As the agent often has only a local frame of reference, it cannot represent the global environment. Frame-of-reference-based learning is a feature of vertebrate intelligence that allows multiple simultaneous representations of an environment at different levels of abstraction. This enables the resolution of patterns that are made up of patterns that are made up of features. The evolutionary computation technique of learning classifier systems has shown promise in learning nested patterns in single-step domains. This work uses the frame-of-reference concept within a learning classifier system to learn stable policies in non-Markov multistep domains. Considering aliased states at a constituent level enables the system to place them appropriately in holistic-level policies. Instead of enumerating a huge search space, evolution computation empowers the novel system to evolve fitter rules and policies. The experimental results show that the novel system effectively solves complex aliasing patterns in non-Markov environments that have been challenging to artificial agents. For example, the novel system utilizes only 6.5, 3.71, and 3.22 steps to resolve Maze10, Littman57, and Woods102, respectively.