Microarray Quality Control (MAQC) Consortium
Research Areas
At a Glance
- Status: Active Consortium
- Year Launched: 2005
- Initiating Organization: U.S. Food and Drug Administration
- Initiator Type: Industry
- No disease focus
- Location: International
Abstract
Microarrays and next-generation sequencing represent core technologies in pharmacogenomics and toxicogenomics; however, before these technologies can successfully and reliably be used in clinical practice and regulatory decision-making, standards and quality measures need to be developed. The Microarray Quality Control (MAQC) Consortium is helping to improve the microarray and next-generation sequencing technologies and to foster their proper applications in discovery, development, and review of U.S. Food and Drug Administration (FDA)–regulated products.
Mission
The first phase of the MAQC project (MAQC-I) aimed to:
-
Provide quality control (QC) tools to the microarray community to avoid procedural failures
-
Develop guidelines for microarray data analysis by providing the public with large reference datasets along with readily accessible reference ribonucleic acid (RNA) samples
-
Establish QC metrics and thresholds for objectively assessing the performance achievable by various microarray platforms
-
Evaluate the advantages and disadvantages of various data analysis methods
The second phase of the MAQC project (MAQC-II) aimed to:
-
Assess the capabilities and limitations of various data analysis methods in developing and validating microarray-based predictive models
-
Reach consensus on the “best practices” for development and validation of predictive models based on microarray gene expression and genotyping data for personalized medicine
The third phase of the MAQC project (MAQC-III), also called Sequencing Quality Control (SEQC), aimed at assessing the technical performance of next-generation sequencing platforms by generating benchmark datasets with reference samples and evaluating advantages and limitations of various bioinformatics strategies in RNA and deoxyribonucleic acid (DNA) analyses. The project aimed to:
-
Examine the latest tools for measuring gene activity (RNA-seq)
-
Establish best practices for reproducibility across different technologies and laboratories
-
Evaluate the utility of these technologies in clinical and safety assessments
Consortium History
Feb. 11, 2005: Phase I of the MAQC project (MAQC-I) on microarray technical performance launched
June 5, 2006: MAQC-I manuscripts submitted
Sept. 8, 2006: MAQC-I results published in September 2006 issue of Nature Biotechnology
Sept. 8, 2006: MAQC-I datasets made publicly available
Sept. 21, 2006: MAQC-II on predictive models (signatures) launched
Aug. 28, 2007: “Pharmacogenomic Data Submissions — Companion Guidance” released
Dec. 16-17, 2008: MAQC-III (or SEQC) on next-generation sequencing launched
March 2009: MAQC-II manuscripts submitted
Aug. 2010: MAQC-II results published in August 2010 issues of Nature Biotechnology and Pharmacogenomics
Structure & Governance
MAQC involves six FDA Centers, major providers of microarray platforms and RNA samples, National Institutes of Health, Environmental Protection Agency (EPA), National Institute of Standards and Technology (NIST), academic laboratories, and other stakeholders.
MAQC contains the following working groups: clinical, toxicogenomics, titration, regulatory biostatistics, genome-wide association, copy number variation, and coordination of the entire MAQC project.
Impact/Accomplishment
MAQC has had many publications since its start in 2006. Additionally, Nature Biotechnology and Pharmacogenomics Journal have published four journal issues featuring MAQC.
The impact and accomplishments of each phase of the study are below.
MAQC-I involved six FDA Centers, major providers of microarray platforms and RNA samples, EPA, NIST, academic laboratories, and other stakeholders. Two human reference RNA samples were selected, and differential gene expression levels between the two samples were calibrated with microarrays and other technologies (e.g., QRT-PCR). The resulting microarray datasets were used for assessing the precision and cross-platform/laboratory comparability of microarrays, and the QRT-PCR datasets enabled evaluation of the nature and magnitude of any systematic biases that may exist between microarrays and QRT-PCR. The availability of the well-characterized RNA samples combined with the resulting microarray and QRT-PCR datasets, which were made readily accessible to the scientific community, allow individual laboratories to more easily identify and correct procedural failures.
During MAQC-II, 36 teams developed classifiers for 13 endpoints — some easy, some difficult to predict, from six relatively large training datasets. These analyses collectively produced more than 18,000 models that were challenged by independent and blinded validation sets generated for MAQC-II. The cross-validated performance estimates for models developed under good practices are predictive of the blinded validation performance. The achievable prediction performance is largely determined by the intrinsic predictability of the endpoint, and simple data analysis methods often perform as well as more complicated approaches. Multiple models of comparable performance can be developed for a given endpoint, and the stability of gene lists correlates with endpoint predictability. Importantly, similar conclusions were reached when more than 12,000 new models were generated by swapping the original training and validation sets.
In MAQC-III (also known as SEQC) specifically, three RNA-seq platforms (Illumina HiSeq, Life Technologies SOLiD, and Roche 454) were tested at multiple sites for reproducibility, accuracy, and information content. The project also extensively compared RNA-seq to microarray technology and evaluated the transferability of predictive models and signature genes between microarray and RNA-Seq data. The impact of various bioinformatics approaches on the downstream biological interpretations of RNA-seq results was also comprehensively examined, and the utility of RNA-seq in clinical application and safety evaluation was assessed. The project was completed by the end of 2014 and generated many manuscripts (visit MAQC Publications under 2014 for a list). Most of the publications are available in a Nature Collections special issue.
Links/Social Media Feed
Homepage |
http://www.fda.gov/ScienceResearch/BioinformaticsTools/MicroarrayQualityControlProject/ |
Please address questions and suggestions about the MicroArray Quality Control project to Weida Tong at Weida.Tong@fda.hhs.gov.
Sponsors & Partners
Abbott Laboratories, Inc. |
Affymetrix Inc. |
Agencourt Bioscience, A Bechman Coulter Company |
Agilent Technologies Inc. |
Albert Einstein College of Medicine |
Almac Diagnostics |
Aster Data Systems |
Asuragen Inc. |
Baylor College of Medicine |
BGI Shenzhen |
Biogen Idec |
BioMath solutions, LLC |
BloodCenter of Wisconsin |
Bristol-Myers Squibb |
Burnham Institute |
Cancer Research UK |
CapialBio Corporation |
CBM S.c.r.l. |
Cedars-Sinai Medical Center |
Centro de Investigación Principe Felipe (CIPF) |
Cogenics, a Divisiion of Clinical Data Inc. |
Columbia University |
Complete Genomics Inc. |
Cornell University |
DNASTAR Inc. |
DNAVision SA |
Dover, A Danaher Motion Company |
Duke University |
East China Normal University |
École Polytechnique Fédérale de Lausanne (EPFL) |
Emory University |
Entelos Inc. |
Environmental Protection Agency (EPA) |
Eppendorf Array Technologies |
Expression Analysis Inc. |
F. Hoffmann-La Roche Ltd. |
FDA/Center for Biologics Evaluation and Research (CBER) |
FDA/Center for Devices and Radiological Health (CDRH) |
FDA/Center for Drug Evalutaion and Research (CDER) |
FDA/Center for Food Safety and Applied Nutrition(CFSAN) |
FDA/Center for Veterinary Medicine (CVM) |
FDA/National Center for Toxicological Research (NCTR) |
Fondazione Bruno Kessler |
Full Moon BioSystems Inc. |
GE Healthcare |
Gene Express, Inc. |
Genedata Inc. |
GeneGo Inc. |
Genemarkers LLC |
Genomatix Software GmbH |
Genomatix Software Inc. |
GenomeQuest Inc. |
Genomes United Inc. |
GenUs BioSystems Inc. |
Georgia Institute of Technology |
Geospiza Inc. |
German Cancer Research Center (DKFZ) |
Ghent University Hospital |
GlaxoSmithKline |
GlobeImmune Inc. |
Golden Helix Inc. |
Harvard University |
Hefei Teachers College |
Helicos BioSciences Corporation |
Illumina Inc. |
Institut Jules Bordet |
Johns Hopkins University |
LGC |
Life Technologies Corporation |
Ligand Pharmaceuticals Inc. |
Lilly Singapore Centre for Drug Discovery |
Luminex Corporation |
Mayo Clinic |
MD Anderson Cancer Center |
Memorial Sloan-Kettering Cancer Center |
Millennium Pharmaceuticals Inc. |
Monsanto Co. |
National Institute of Standards and Technology (NIST) |
National Institutes of Health/National Cancer Institute (NCI) |
National Institutes of Health/National Center for Biotechnology Information (NCBI) |
National Institutes of Health/National Institute of Environmental Health Sciences (NIEHS) |
New York University |
NextBio Inc. |
North Carolina State University |
Northeast Forestry University |
Northwestern University |
Norwegian Microarray Consortium |
Novartis Oncology |
Novartis Pharma AG |
Nuvera Biosciences Inc. |
Operon Biotechnologies Inc. |
OpGen Inc. |
Oregon Health & Science University |
Pacific Biosciences Inc. |
Panomics |
Pathwork Diagnostics Inc. |
Point Judith Capital |
Princeton University |
Purdue University |
RainDance Technologies Inc. |
READNA |
Riverside Center Care Center |
Roche Diagnostics Systems |
Roche Molecular Systems |
Roche NimbleGen Inc. |
Roche Palo Alto LLC |
Roche/454 Life Sciences |
Rosetta Biosoftware |
RTI International |
Russian Academy of Sciences |
Rutgers, The State University of New Jersey |
SABiosciences Corp. |
Science Applications International Corporation (SAIC) |
SeqWright Inc. |
Sichuan University |
South Dakota State University |
Spheromics |
SRA International (EMMES) |
St Mary’s Hospital |
Stanford University |
Statistical Analysis Software (SAS) Institute Inc. |
Stratagene |
SUNY at Stony Brook |
Swiss Institute of Bioinformatics |
Systems Analytics Inc. |
Takeda Global Research & Development Center Inc. |
TeleChem ArrayIt |
The Hamner Institutes for Health Sciences |
The Jackson Laboratory |
The Salk Institute for Biological Studies |
Tsinghua Universtiy |
Umeå University |
University of Arkansas for Medical Sciences |
University of Bayreuth |
University of California Los Angeles (UCLA)/Cedars-Sinai |
University of California San Francisco (UCSF) |
University of Cologne |
University of Copenhagen |
University of Essex |
University of Illinois Urbana-Champaign (UIUC) |
University of Kansas |
University of Massachusetts Boston |
University of Massachusetts Lowell |
University of Michigan |
University of Missouri |
University of North Carolina |
University of Southern California |
University of Southern Mississippi |
University of Texas at Dallas |
University of Texas Southwestern Medical Center (UTSW) |
University of Toledo |
Vanderbilt University |
ViaLogy Inc. |
Virginia Bioninformatics Institute |
Virginia Commonwealth University |
VirginiaTech |
Wake Forest University |
Weill Medical College of Cornell University |
Yale University |
Zhejiang University |
Z-Tech, an ICF International Company at NCTR/FD |