NERSCPowering Scientific Discovery Since 1974

Powered by NERSC, a Database of Billions of Genes and Counting!

With More than a Billion Microbial genes, IMG/M Breaks a Record

January 26, 2012 | Tags: Joint Genome Institute

Linda Vu, lvu@lbl.gov, +1 510 495 2402

IMG/M team celebrates the recording of 1 billionth gene.

Microbes are microscopic organisms that live in every nook and cranny of our planet. Without them, plants wouldn’t grow, garbage wouldn’t decay, humans wouldn’t digest food, and there would literally be no life on Earth, or at least as we know it. By examining the genetic makeup of these “bugs,” scientists hope to understand how they work, and how they can be used to solve a variety of important problems like identifying new sources of clean energy.

One important tool for this analysis is the IMG/M (Integrated Microbial Genomes with Microbiome Samples) data management system, which supports the analysis of microbial communities sequenced by the Department of Energy’s Joint Genome Institute (JGI). With computing and storage support from the National Energy Research Scientific Computing Center (NERSC), this system currently contains more than 3 billion microbial genes—more than any other similar system in the world.

“Last December IMG/M crossed the boundary of 1 billion genes recorded in the system, and we wouldn’t have been able to reach this important milestone without NERSC,” says Victor Markowitz, Chief Informatics Officer and Associate Director at JGI. Markowitz also heads the Lawrence Berkeley National Laboratory’s Biological Data Management and Technology Center (BDMTC). Since the December milestone, billions more genes have been recorded in the system.

A Fire Hose of Data

No microbe is an island. In fact, these organisms live in communities of thousands or more. By studying the aggregate genome, or metagenome, of these communities, researchers can better understand how the different organisms interact with each other and their host, as well as how they adapt to different environments.

According to Nikos Kyrpides, who heads JGI’s Microbial Genome and Metagenome Super Program, the number and size of metagenome sequence datasets are growing incredibly fast. This is primarily due to advancements in DNA sequencing technologies that allow for rapid and relatively cheap sequencing of microbial communities. Unlocking these interactions is crucial for developing clean energy sources or understanding the mechanism of disease.

Kyrpides also notes that a series of innovative data processing methods developed by JGI’s Omics group has paved the way in finding a way of coping with the deluge of metagenomics data. The majority of data obtained from current sequencing technologies is comprised of short fragments, and the new tools allow researchers to predict genes on these fragments.

"These new methods allow us for the first time to explore the full genetic capacity of the sequenced data,” says Kostas Mavrommatis, who leads the Omics group; but he adds that at the same time, the comparative analysis and data management of billions of genes poses a tremendous computational challenge.

According to Markowitz, the rapidly growing size of metagenome sequence datasets requires using supercomputers for interpreting the biological function of hundreds of millions, and even billions of genes.  As part of the NERSC and JGI partnership, JGI tools and pipelines can now be utilized on NERSC’s Cray XE6 “Hopper” system, currently the eighth most powerful supercomputer in the world.  Providing scientists with the ability of analyzing metagenome sequence datasets has also required new data management methods that were developed by BDMTC’s Amy Chen and NERSC’s Shane Canon.

 “NERSC has a long history of successfully managing computing clusters, and its management of JGI’s cluster together with the added capacity of Hopper has sped up the biological interpretation process and allowed IMG/M to reach this milestone of billions genes so quickly, ”says Markowitz.

Both NERSC and JGI are managed by the Berkeley Lab on behalf of the Department of Energy, and IMG/M was developed with support from the Berkeley Lab’s Computational Research Division.


About NERSC and Berkeley Lab
The National Energy Research Scientific Computing Center (NERSC) is the primary high-performance computing facility for scientific research sponsored by the U.S. Department of Energy's Office of Science. Located at Lawrence Berkeley National Laboratory, the NERSC Center serves more than 4,000 scientists at national laboratories and universities researching a wide range of problems in combustion, climate modeling, fusion energy, materials science, physics, chemistry, computational biology, and other disciplines. Berkeley Lab is a U.S. Department of Energy national laboratory located in Berkeley, California. It conducts unclassified scientific research and is managed by the University of California for the U.S. DOE Office of Science. »Learn more about computing sciences at Berkeley Lab.