Author: Community

Neural architecture search (NAS) is a challenging problem in the design of deep learning due to its nonconvexity. To address this problem, an adaptive scalable NAS method (AS-NAS) is proposed based on the reinforced I-Ching divination evolutionary algorithm (IDEA) and variable-architecture encoding strategy. First, unlike the typical reinforcement learning (RL)-based and evolutionary algorithm (EA)-based NAS methods, a simplified RL algorithm is developed and used as the reinforced operator controller to adaptively select the efficient operators of IDEA. Without the complex actor–critic parts, the reinforced IDEA based on simplified RL can enhance the search efficiency of the original EA with lower computational cost. Second, a variable-architecture encoding strategy is proposed to encode neural architecture as a fixed-length binary string. By simultaneously considering variable layers, channels, and connections between different convolution layers, the deep neural architecture can be scalable. Through the integration with the reinforced IDEA and variable-architecture encoding strategy, the design of the deep neural architecture can be adaptively scalable. Finally, the proposed AS-NAS are integrated with the ${L}_{1/2}$ regularization to increase the sparsity of the optimized neural architecture. Experiments and comparisons demonstrate the effectiveness and superiority of the proposed method.

Multimodal representation learning is beneficial for many multimedia-oriented applications, such as social image recognition and visual question answering. The different modalities of the same instance (e.g., a social image and its corresponding description) are usually correlational and complementary. Most existing approaches for multimodal representation learning are not effective to model the deep correlation between different modalities. Moreover, it is difficult for these approaches to deal with the noise within social images. In this article, we propose a deep learning-based approach named evolutionary adversarial attention networks (EAANs), which combines the attention mechanism with adversarial networks through evolutionary training, for robust multimodal representation learning. Specifically, a two-branch visual-textual attention model is proposed to correlate visual and textual content for joint representation. Then adversarial networks are employed to impose regularization upon the representation by matching its posterior distribution to the given priors. Finally, the attention model and adversarial networks are integrated into an evolutionary training framework for robust multimodal representation learning. Extensive experiments have been conducted on four real-world datasets, including PASCAL, MIR, CLEF, and NUS-WIDE. Substantial performance improvements on the tasks of image classification and tag recommendation demonstrate the superiority of the proposed approach.

This article is concerned with the multitask multiagent allocation problem via many-objective optimization for multiagent systems (MASs). First, a novel layered MAS model is constructed to address the multitask multiagent allocation problem that includes both the original task simplification and the many-objective allocation. In the first layer of the model, the deep Q-learning method is introduced to simplify the prioritization of the original task set. In the second layer of the model, the modified shift-based density estimation (MSDE) method is put forward to improve the conventional strength Pareto evolutionary algorithm 2 (SPEA2) in order to achieve many-objective optimization on task assignments. Then, an MSDE-SPEA2-based method is proposed to tackle the many-objective optimization problem with objectives including task allocation, makespan, agent satisfaction, resource utilization, task completion, and task waiting time. As compared with the existing allocation methods, the developed method in this article exhibits an outstanding feature that the task assignment and the task scheduling are carried out simultaneously. Finally, extensive experiments are conducted to: 1) verify the validity of the proposed model and the effectiveness of two main algorithms and 2) illustrate the optimal solution for task allocation and efficient strategy for task scheduling under different scenarios.

Feature extraction is a critical issue in many machine learning systems. A number of basic fusion operators have been proposed and studied. This article proposes an evolutionary algorithm, called evolutionary deep fusion method, for searching an optimal combination scheme of different basic fusion operators to fuse multiview features. We apply our proposed method to chemical structure recognition. Our proposed method can directly take images as inputs, and users do not need to transform images to other formats. The experimental results demonstrate that our proposed method can achieve a better performance than those designed by human experts on this real-life problem.

This article investigates deep neural networks (DNNs)-based lung nodule classification with hyperparameter optimization. Hyperparameter optimization in DNNs is a computationally expensive problem, and a surrogate-assisted evolutionary algorithm has been recently introduced to automatically search for optimal hyperparameter configurations of DNNs, by applying computationally efficient surrogate models to approximate the validation error function of hyperparameter configurations. Different from existing surrogate models adopting stationary covariance functions (kernels) to measure the difference between hyperparameter points, this article proposes a nonstationary kernel that allows the surrogate model to adapt to functions whose smoothness varies with the spatial location of inputs. A multilevel convolutional neural network (ML-CNN) is built for lung nodule classification, and the hyperparameter configuration is optimized by the proposed nonstationary kernel-based Gaussian surrogate model. Our algorithm searches with a surrogate for optimal setting via a hyperparameter importance-based evolutionary strategy, and the experiments demonstrate our algorithm outperforms manual tuning and several well-established hyperparameter optimization methods, including random search, grid search, the tree-structured parzen estimator (TPE) approach, Gaussian processes (GP) with stationary kernels, and the recently proposed hyperparameter optimization via RBF and dynamic (HORD) coordinate search.

Presents a listing of the editorial board, board of governors, current staff, committee members, and/or society editors for this issue of the publication.

Presents a listing of the editorial board, board of governors, current staff, committee members, and/or society editors for this issue of the publication.

Kriging models, also known as Gaussian process models, are widely used in surrogate-assisted evolutionary algorithms (SAEAs). However, the cubic time complexity of the standard Kriging models limits their usage in high-dimensional optimization. To tackle this problem, we propose an incremental Kriging model for high-dimensional surrogate-assisted evolutionary computation. The main idea is to update the Kriging model incrementally based on the equations of the previously trained model instead of building the model from scratch when new samples arrive, so that the time complexity of updating the Kriging models can be reduced to quadratic. The proposed incremental learning scheme is very suitable for online SAEAs since they evaluate new samples in each one or several generations. The proposed algorithm is able to achieve competitive optimization results on the test problems compared with the standard Kriging-assisted evolutionary algorithm and is significantly faster than the standard Kriging approach. The proposed algorithm also shows competitive or better performances compared with four fast Kriging-assisted evolutionary algorithms and four state-of-the-art SAEAs. This work provides a fast way of employing Kriging models in high-dimensional surrogate-assisted evolutionary computation.

Convolutional neural networks (CNNs) have achieved state-of-the-art performance in many image classification tasks. However, training a deep CNN requires a massive amount of training data, which can be expensive or unobtainable in practical applications, such as defect inspection and medical diagnosis. Transfer learning has been developed to address this issue by transferring knowledge learned from source domains to target domains. A common approach is fine-tuning, which adapts the parameters of a trained neural network for the new target task. Nevertheless, the network architecture remains designed for the source task rather than the target task. To optimize the network architecture in transfer learning, we propose a two-stage evolutionary neural architecture search for transfer learning (EvoNAS-TL), which searches for an efficient subnetwork of the source model for the target task. EvoNAS-TL features two search stages: 1) structure search and 2) local enhancement. The former conducts a coarse-grained global search for suitable neural architectures, while the latter acts as a fine-grained local search to refine the models obtained. In this study, neural architecture search (NAS) is formulated as a multiobjective optimization problem that concurrently minimizes the prediction error and model size. The knee-guided multiobjective evolutionary algorithm, a modern multiobjective optimization approach, is employed to solve the NAS problem. In this study, several experiments are conducted to examine the effectiveness of EvoNAS-TL. The results show that applying EvoNAS-TL on VGG-16 can reduce the model size by 52%–85% and simultaneously improve the testing accuracy by 0.7%–6.9% in transferring from ImageNet to CIFAR-10 and NEU surface detection datasets. In addition, EvoNAS-TL performs comparably to or better than state-of-the-art methods on the CIFAR-10, NEU,-
and Office-31 datasets.

This article concerns the study of mixed Pareto-lexicographic multiobjective optimization problems where the objectives must be partitioned in multiple priority levels (PLs). A PL is a group of objectives having the same importance in terms of optimization and subsequent decision making, while between PLs a lexicographic ordering exists. A naive approach would be to define a multilevel dominance relationship and apply a standard EMO/EMaO algorithm, but the concept does not conform to a stable optimization process as the resulting dominance relationship violates the transitive property needed to achieve consistent comparisons. To overcome this, we present a novel approach that merges a custom nondominance relation with the Grossone methodology, a mathematical framework to handle infinite and infinitesimal quantities. The proposed method is implemented on a popular multiobjective optimization algorithm (NSGA-II), deriving a generalization of it called by us PL-NSGA-II. We also demonstrate the usability of our strategy by quantitatively comparing the results obtained by PL-NSGA-II against other priority and nonpriority-based approaches. Among the test cases, we include two real-world applications: one 10-objective aircraft design problem and one 3-objective crash safety vehicle design task. The obtained results show that PL-NSGA-II is more suited to solve lexicographical many-objective problems than the general purpose EMaO algorithms.