Category Archives: Data-Intensive Computing

Parallel and Distributed Computational Intelligence book is out for pre-order

“Parallel and Distributed Computational Intelligence” edited by Francisco Fernández de Vega & Erick Cantú-Paz and published by Springer is out for pre-order. The first chapter “When Huge is Routine: Scaling Genetic Algorithms and Estimation of Distribution Algorithms via Data-Intensive Computing”

Continue reading

Posted in Books, Data-Intensive Computing, Estimation of distribution algorithms, Genetic algorithms, Publications | Tagged , | Comments Off on Parallel and Distributed Computational Intelligence book is out for pre-order

Meandre 2.0 Alpha Preview = Scala + MongoDB

A lot of water under the bridge has gone by since the first release of Meandre 1.4.X series. In January I went back to the drawing board and start sketching what was going to be 1.5.X series. The slide deck

Continue reading

Posted in Crochet, Data-Intensive Computing, meandre, mongodb, Presentations, RDF, Research, scala, Software | Tagged , , , | Comments Off on Meandre 2.0 Alpha Preview = Scala + MongoDB

Scaling eCGA Model Building via Data-Intensive Computing

I just uploaded the technical report of the paper we put together for CEC 2010 on how we can scale up eCGA using a MapReduce approach. The paper, besides exploring the Hadoop implementation, it also presents some very compelling results obtained with MongoDB (a document based store able to perform parallel MapReduce tasks via sharding). […]

Related posts:

  1. Scaling Genetic Algorithms using MapReduce
  2. Data-Intensive Computing for Competent Genetic Algorithms: A Pilot Study using Meandre
  3. Data-Intensive Computing for Competent Genetic Algorithms: A Pilot Study using Meandre

Continue reading

Posted in Data-Intensive Computing, eCGA, Estimation of distribution algorithms, hadoop, map-reduce, mongodb, pro, Research, Software | Comments Off on Scaling eCGA Model Building via Data-Intensive Computing

Soaring the Clouds with Meandre

You may find the slide deck and the abstract for the presentation we delivered today at the “Data-Intensive Research: how should we improve our ability to use data” workshop in Edinburgh. Abstract This talk will focus a highly scalable data intensive infrastructure being developed at the National Center for Supercomputing Application (NCSA) at the University […]

Related posts:

  1. Meandre: Semantic-Driven Data-Intensive Flows in the Clouds
  2. Data-Intensive Computing for Competent Genetic Algorithms: A Pilot Study using Meandre
  3. [BDCSG2008] Clouds and ManyCores: The Revolution (Dan Reed)

Continue reading

Posted in cloud computing, Data-Intensive Computing, hadoop, meandre, Notes, Research, ZigZag | Comments Off on Soaring the Clouds with Meandre

Scaling Genetic Algorithms using MapReduce

Below you may find the abstract to and the link to the technical report of the paper entitled “Scaling Genetic Algorithms using MapReduce” that will be presented at the Ninth International Conference on Intelligent Systems Design and Applications (ISDA) 2009 by Verma, A., Llorà, X., Campbell, R.H., Goldberg, D.E. next month. Abstract:Genetic algorithms(GAs) are increasingly […]

Related posts:

  1. Scaling eCGA Model Building via Data-Intensive Computing
  2. Data-Intensive Computing for Competent Genetic Algorithms: A Pilot Study using Meandre
  3. Data-Intensive Computing for Competent Genetic Algorithms: A Pilot Study using Meandre

Continue reading

Posted in Conferences, Data-Intensive Computing, Estimation of distribution algorithms, Genetic algorithms, hadoop, map-reduce, Publications, Research, Technical Reports | Comments Off on Scaling Genetic Algorithms using MapReduce

Temporary storage for Meandre’s distributed flow execution

Designing the distributed execution of a generic Meandre flow involves several moving pieces. One of those is the temporary storage required by the computing nodes (think of it as one node as one isolated component of a flow) to keep up with the data generated by a component, and also be able to replicate such […]

Related posts:

  1. Easy, reliable, and flexible storage for Python
  2. ZooKeeper and distributed applications
  3. Meandre: Semantic-Driven Data-Intensive Flow Engine

Continue reading

Posted in Data-Intensive Computing, data-intensive flows, java, meandre, Notes, python, Software, storage, tokyo cabinet | Comments Off on Temporary storage for Meandre’s distributed flow execution

Liquid: RDF endpoint for FluidDB

A while ago I wrote some thoughts about how to map RDF to and from FluidDB. There I explored how you could map RDF onto FluidDB, and how to get it back. That got me thinking about how to get a simple endpoint you could query for RDF. Imagine that you could pull FluidDB data […]

Related posts:

  1. Liquid: RDF meandering in FluidDB
  2. Temporary storage for Meandre’s distributed flow execution
  3. Efficient serialization for Java (and beyond)

Continue reading

Posted in cloud computing, Data-Intensive Computing, FluidDB, meandre, Notes, RDF, Software, sparql, storage | Comments Off on Liquid: RDF endpoint for FluidDB

Liquid: RDF meandering in FluidDB

Meandre (NCSA pushed data-intensive computing infrastructure) relies on RDF to describe components, flows, locations and repositories. RDF has become the central piece that makes possible Meandre’s flexibility and reusability. However, one piece still remains largely sketchy and still has no clear optimal solution: How can we facilitate to anybody sharing, publishing and annotating flows, components, […]

Related posts:

  1. Liquid: RDF endpoint for FluidDB
  2. Meandre: Semantic-Driven Data-Intensive Flows in the Clouds
  3. Meandre 1.4.0 final release candidate tagged

Continue reading

Posted in cloud computing, Data-Intensive Computing, FluidDB, meandre, Notes, RDF, Research, Social Networks, Software, storage | Comments Off on Liquid: RDF meandering in FluidDB