Author: nuria

Call for Contest Participation – Classifier domains of competence: The landscape contest (ICPR 2010)

Classifier domains of competence: The landscape contest is a research competition aimed at finding out the relation between data complexity and the performance of learners. Comparing your techniques to those of other participants on targeted-complexity problems may contribute to enrich our understanding of the behavior of machine learning techniques and open further research lines.

The contest will take place on August 22, during the 20th International Conference on Pattern Recognition (ICPR 2010) at Istanbul, Turkey.

We encourage everyone to participate and share with us your work! For further details about dates and submission, please see http://www.salle.url.edu/ICPR10Contest/.

SCOPE OF THE CONTEST

The landscape contest involves the running and evaluation of classifier systems over synthetic data sets. Over the last two decades, the pattern recognition and machine learning communities have developed many supervised learning techniques. Nevertheless, the competitiveness of such techniques has always been claimed over a small and repetitive set of problems. This contest provides a new and configurable testing framework, reliable enough to test the robustness of each technique and detect its limitations.

INSTRUCTION FOR PARTICIPANTS

Contest participants are allowed to use any type of technique. However, we highly encourage and appreciate the use of novel algorithms.

Participants are required to submit the results by email to the organizers.
Submission e-mail: nmacia@salle.url.edu
Meet the submission deadline: Wednesday May 26, 2010

The contest is divided into two phases: (1) offline test and (2) live test. For the offline test, participants should run their algorithms over two sets of problems, S1 and S2. However, the real competition, the live test, will take place during the conference. Two more collections of problems, S3 and S4, will be presented.

S1: Collection of data sets spread along the complexity space to train the learner. All the instances will be duly labeled.

S2: Collection of data sets spread along the complexity space with no class labeling to test the learner performance.

S3: Collection of data sets with no class labeling, like S2 to be run for a limited period of time.

S4: Collection of data sets with no class labeling covering specific regions of the complexity space to determine the neighborhood dominance.

For the offline test, the results report consists of:

1. Labeling the data sets of the collection S2.

The procedure is the following:

1. Train the learner using Dn-trn.arff in S1.
2. Provide the rate of the correctly classified instances over a 10-fold cross validation.
3. Label the corresponding data set Dn-tst.arff in S2.
4. Store the n models generated for each data set to perform the live contest on August 22. Be ready to load them on this day.

2. Describing the techniques used.

A brief summary (1~2 pages) of the machine learning technique/s used in the experiments must be submitted. We expect details such as the learning paradigm, configuration parameters, strength and limitations, and computational cost.

IMPORTANT DATES

* May 26, 2010: Deadline for submission of the results and technical report

* May 29, 2010: Notification of participation

* Aug 22, 2010: Release of S3 and S4

* Aug 22, 2010: ICPR 2010 – Interactive Session

CONTACT DETAILS

Dr. Tin Kam Ho – tkh at research.bell-labs.com
Núria Macià – nmacia at salle.url.edu
Prof. Albert Orriols Puig – aorriols at salle.url.edu
Prof. Ester Bernadó Mansilla – esterb at salle.url.edu

Classifier domains of competence: The landscape contest is a research competition aimed at finding out the relation between data complexity and the performance of learners. Comparing your techniques to those of other participants may contribute to enrich our understanding of the behavior of machine learning and open further research lines. Contest participants are allowed to use any type of technique. However, we highly encourage and appreciate the use of novel algorithms.

The contest will take place on August 22, during the 20th International Conference on Pattern Recognition (ICPR 2010) at Istanbul, Turkey.

We are planning to have a day workshop during the ICPR 2010, so that participants will be able to present and discuss their results.

We encourage everyone to participate and share with us your work! For further details about dates and submission, please visit The landscape contest webpage.

Classifier domains of competence: The landscape contest is a research competition aimed at finding out the relation between data complexity and the performance of learners. Comparing your techniques to those of other participants may contribute to enrich our understanding of the behavior of machine learning and open further research lines. Contest participants are allowed to use any type of technique. However, we highly encourage and appreciate the use of novel algorithms.

The contest will take place on August 22, during the 20th International Conference on Pattern Recognition (ICPR 2010) at Istanbul, Turkey.

We are planning to have a day workshop during the ICPR 2010, so that participants will be able to present and discuss their results.

We encourage everyone to participate and share with us your work! For further details about dates and submission, please visit The landscape contest webpage.

It seems that the artificial intelligence related in science fiction is not as far from reality as we used to think. The main character of the film AI, a little boy belonging to a robot series capable of emulating human behavior, is now a model to reach in current scientific projects, which aim at providing machines with consciousness, thoughts, and emotions to interact with human beings. Thus, the world described in Blade Runner, a world where humans and robots coexist and cannot be distinguished with the naked eye, may be just behind the corner.

The advances in AI field, however, start to raise some serious concerns about robot autonomy and its social status as well as how to face this social disruption, and the three Laws elaborated by Asimov to protect humans from machines start to make sense for other than computer geeks. Scientifics are concerned about the “loss of human control of computer-based intelligences”, and the past February, the Association for the Advancement of Artificial Intelligence organized a conference in Asilomar (not a casual place) to discuss the limits of the research in this field. Development of machines that are close to kill autonomously are worth a discussion by those involved in the creation of the brain of such devices. The news of this event has leaked in the Markoff’s article in the New York Times.

On the other hand, who will be responsible for damages caused by these autonomous friends? Themselves or the corresponding designer? In this sense, philosophy should play a leading role in the design and integration of these “future citizens” since they should have a moral system allowing them to learn ethics from experience and people, and also find their place in our society. The latter implies to create a legal framework that defines machine’s civic rights and duties which is a proposal under study (see the news published by “El Periódico”, in Spanish language).

Finally, one may ask whether or not we are ready to live with human emulators. In my view, we are not. Although in the past years we have been skillful to adapt to new and challenging situations, and our experience with immigration integration and race conflicts should help us to welcome these new electronic neighbors, I tend to think that coexistence with robots will be one of the greatest challenges mankind has ever faced. Anyway, we will need to figure out the way to overcome it because the individualism and loneliness ruling our current society is leading us unrelentingly to a future with custom-made roommates.

It seems that the artificial intelligence related in science fiction is not as far from reality as we used to think. The main character of the film AI, a little boy belonging to a robot series capable of emulating human behavior, is now a model to reach in current scientific projects, which aim at providing machines with consciousness, thoughts, and emotions to interact with human beings. Thus, the world described in Blade Runner, a world where humans and robots coexist and cannot be distinguished with the naked eye, may be just behind the corner.

The advances in AI field, however, start to raise some serious concerns about robot autonomy and its social status as well as how to face this social disruption, and the three Laws elaborated by Asimov to protect humans from machines start to make sense for other than computer geeks. Scientifics are concerned about the “loss of human control of computer-based intelligences”, and the past February, the Association for the Advancement of Artificial Intelligence organized a conference in Asilomar (not a casual place) to discuss the limits of the research in this field. Development of machines that are close to kill autonomously are worth a discussion by those involved in the creation of the brain of such devices. The news of this event has leaked in the Markoff’s article in the New York Times.

On the other hand, who will be responsible for damages caused by these autonomous friends? Themselves or the corresponding designer? In this sense, philosophy should play a leading role in the design and integration of these “future citizens” since they should have a moral system allowing them to learn ethics from experience and people, and also find their place in our society. The latter implies to create a legal framework that defines machine’s civic rights and duties which is a proposal under study (see the news published by “El Periódico”, in Spanish language).

Finally, one may ask whether or not we are ready to live with human emulators. In my view, we are not. Although in the past years we have been skillful to adapt to new and challenging situations, and our experience with immigration integration and race conflicts should help us to welcome these new electronic neighbors, I tend to think that coexistence with robots will be one of the greatest challenges mankind has ever faced. Anyway, we will need to figure out the way to overcome it because the individualism and loneliness ruling our current society is leading us unrelentingly to a future with custom-made roommates.