Variation exists in microbiome sequencing and analysis

Microbiome sequencing and analysis is at the heart of microbiome science.  While established protocols do exist in order to identify the bacteria in samples, there is still a lot of variability between labs, individuals, reagents, analytics techniques, and instruments.  This makes comparing data across groups difficult.  The microbiome quality control project (MBQC) was carried out in order to determine the sources of variability that exist.  During the project identical samples were sequenced by various labs, and the results were compared.  The project’s conclusions were published this past week in the journal Genome Biology.

There was a lot of variation from many of the sources during experimentation.  Overall, DNA extraction technique proved to be a major source of error.  On the other hand, sample storage protocols, such as length of time the sample spent in the freezer, appeared to only play a small role in variability. Another encouraging result was that the the bioinformatics pipelines, i.e. the software programs used to determine the bacteria from the raw data, displayed consistent results.  The project also included negative controls that should have contained no bacterial DNA at all, however many labs reported seeing non-trivial sequences.  In addition, the samples with known compositions often times had spurious DNA from bacteria that should not have been there.  Sometimes the results showed upwards of 7x more bacteria organizational taxonomic units (OTUs) than expected.

This initial MBQC project accomplished two of its major goals, and therefore should be considered a success.  First, it demonstrated a need for quality control within microbiome scientists.  Second, it helped narrow down the variables that need to be studied more robustly in future MBQC projects.  For now though, we must just acknowledge that error does exist in microbiome studies, and to keep that in mind when interpreting results.

Quality control is critical to microbiome research

We just got back from two days in Rockville, MD for a meeting of the Microbiome Quality Control Project (MBQC).  The MBQC is a project that seeks to understand variability in microbiome data from sampling to analysis and make recommendations as to best practices.  Its steering committee includes some of the leading scientists in the microbiome field, Rob Knight, Curtis Huttenhower, and Owen White (all members of the AMI Scientific Advisory Board), along with epidemiologists from the National Cancer Institute Rashmi Sinha and Christian Abnet.  

Labs from around North America signed up to participate in two different aspects of the study, microbiome handling and bioinformatic data analysis. The labs that were involved in the handling aspect of the study received anonymized samples from a central repository and were tasked with extracting DNA from the samples and sequencing them.  They then made their data available (still anonymized) for the groups who signed up for the bioinformatics phase of the projects to analyze the data. Because all of the groups were handling and sequencing the same samples, differences in results meant that the variability was being caused by some aspect of the handling process or differences in the analysis.

At the meeting the steering committee presented the results of the study followed by a day and a half of discussion of the results and what steps could be taken to develop more standardized results, protocols, and reference materials for future microbiome studies. The results were exciting to see but even forgetting the results, the overall initiative was a huge success. This was a project that took place for over a year and all of the project's participants volunteered an enormous amount of time and conducted their research studies without receiving any funding.  It was wonderful to see a group of the world's leading scientists to come together like this with the sole goal of bettering future microbiome research.  

The AMI supports the MBQC and looks forward to being involved in the future phases of this work. As we learned at the meeting, there is so much left to be done to improve microbiome quality control and this was just the beginning.  More information about the results of the study and the meeting will be published in the coming months.