LCS & GBML Central

As an important approach to solving complex sequential decision problems, reinforcement learning (RL) has been widely studied
in the community of artificial intelligence and machine learning. However, the generalization ability of RL is still an open
problem and it is difficult for existing RL algorithms to solve Markov decision problems (MDPs) with both continuous state
and action spaces. In this paper, a novel RL approach with fast policy search and adaptive basis function selection, which
is called Continuous-action Approximate Policy Iteration (CAPI), is proposed for RL in MDPs with both continuous state and
action spaces. In CAPI, based on the value functions estimated by temporal-difference learning, a fast policy search technique
is suggested to search for optimal actions in continuous spaces, which is computationally efficient and easy to implement.
To improve the generalization ability and learning efficiency of CAPI, two adaptive basis function selection methods are developed
so that sparse approximation of value functions can be obtained efficiently both for linear function approximators and kernel
machines. Simulation results on benchmark learning control tasks with continuous state and action spaces show that the proposed
approach not only can converge to a near-optimal policy in a few iterations but also can obtain comparable or even better
performance than Sarsa-learning, and previous approximate policy iteration methods such as LSPI and KLSPI.

In a previous paper, we generalized to the mixed strategy case the γ model of coalition formation (introduced by Hart and
Kurz in Econometrica 51(4):1047–1064, 1983) for situations in which players have ambiguous expectations about the formation of the coalitions in which they are not
involved; then we analyzed the corresponding evolutionary games. In this paper, we embody into the model rationality of the
players; it follows that allowing for mixed strategies makes it impossible to construct unequivocally a von Neumann–Morgestein
expected utility function coherent (in the sense of de Finetti B in Sul Significato Soggettivo della Probabilità, Fundamenta Mathematicae, T, vol XVIII, pp
298–329, 1931) to every strategy profile. We find out that if the multiplicity of coherent beliefs problem is approached by considering
“ambiguity loving” players then existence results for classical static equilibria can be obtained in this model. Moreover,
we provide conditions for the game to be dynamically playable and we find how the coalition structure beliefs might evolve
coherently (according) to the evolution of the strategies.

The study of biological processes within cells is based on the measurement of the activity of different molecules, in particular
genes and proteins whose activities are strictly related. The activity of genes is measured through a systematic investigation
carried out by microarrays. Such technology enables the investigation of all the genes of an organism in a single experiment,
encoding meaningful biological information. Nevertheless, the preprocessing of raw microarray data needs automatic tools that
standardise such phase in order to: (a) avoiding errors in analysis phases, and (b) making comparable the results of different
laboratories. The preprocessing problem is as much relevant as considering results obtained from analysis platforms of different
vendors. Nevertheless, there is currently a lack of tools that allow to manage and preprocess multivendor dataset. This paper
presents a software platform (called GSAT, General-purpose Summarisation and Annotation Tool) able to manage and preprocess
microarray data. The GSAT allows the summarisation, normalisation and annotation of multivendor microarray data, using web
services technology. First experiments and results on Affymetrix data samples are also discussed. GSAT is available online
at http://bioingegneria.unicz.it/m-cs as a standalone application or as a plugin of the TMEV microarray data analysis platform.

In this paper, definitions of

K

automata,

K

regular languages,

K

regular expressions and

K

regular grammars based on lattice-ordered semirings are given. It is shown that

K

NFA is equivalent to

K

DFA under some finite condition, the Pump Lemma holds if

K

is finite, and

Ke

NFA is equivalent to

K

NFA. Further, it is verified that the concatenation of

K

regular languages remains a

K

regular language. Similar to classical cases and automata theory based on lattice-ordered monoids, it is also found that

K

NFA,

K

regular expressions and

K

regular grammars are equivalent to each other when

K

is a complete lattice.