Tag Archives: differential gene expression

Benchmarking Differential Gene Expression Tools

Posted on by

In a recent study, Schurch et al., 2015 closely examine 9 differential gene expression (DGE) tools (baySeq , cuffdiff , DESeq , edgeR , limma , NOISeq , PoissonSeq , SAMSeq, DEGSeq) and rate their performance as a function of replicates in an RNA-Seq experiment. The group highlights edgeR and DESeq as the most widely used tools in the field and conclude that they along with limma perform the best in studies with high and low numbers of biological replicates. The study goes further, making the specific recommendation that experiments with greater than 12 replicates should use DESeq, while those with fewer than 12 replicates should use edgeR. As for the number of replicates needed, Schurch et al recommend at least 6 replicates/condition in an RNA-seq experiment, and up to 12 in studies where identifying the majority of differentially expressed genes is critical. 

With each technical replicate having only 0.8-2.8M reads, this paper and others (Rapaport et al., 2013) continue to suggest that more replicates in an RNA-seq experiment are preferred over simply increasing the number of sequencing reads. Several other papers, including differential expression profiling recommendations in our Sequencing Coverage Guide recommend at least 10M reads per sample, but do not make recommendations on the numbers of replicates needed. The read/sample number disparity is related to the relatively small and well annotated S. cerevisiae genome in this study and the more complex, multiple transcript isoforms in mammalian tissue. By highlighting studies that carefully examine the number of replicates that should be used, we hope to improve RNA-seq experimental design on Genohub. 

So why don’t researchers use an adequate number of replicates? 1) Sequencing cost, 2) Inexperience in differential gene expression analysis. We compare the costs between 6 and 12 replicates in yeast and human RNA-Seq experiments using 1 and 10M reads/sample to show that in many cases adding more replicates in an experiment can be affordable. 

 

6 replicates

12 replicates

Human (10M reads/sample)

$2,660

$4,470

Yeast (1M reads/sample)

$2,810

$4,470

*Prices are in USD and are inclusive of both sequencing and library prep cost. Click on prices in the table to see more project specific detail.

The table shows that the main factor in the price difference is related to library preparation costs. Sequencing on the Illumina Miseq or Hiseq at the listed sequencing depths do not play as significant a role in cost, due to the sequencing capacity of those instruments. 

To accurately determine the sequencing output required for your RNA-seq study, simply change the number of reads/sample in our interactive Project Page. 

 

References:

Evaluation of tools for differential gene expression analysis by RNA-seq on a 48 biological replicate experiment. Nicholas J. Schurch, Pieta Schofield, Marek Gierliński, Christian Cole, Alexander Sherstnev, Vijender Singh, Nicola Wrobel, Karim Gharbi, Gordon G. Simpson, Tom Owen-Hughes, Mark Blaxter, Geoffrey J. Barton

Comprehensive evaluation of differential gene expression analysis methods for RNA-seq data. Franck Rapaport, Raya Khanin, Yupu Liang, Mono Pirun, Azra Krek, Paul Zumbo,Christopher E Mason, Nicholas D Socci and Doron Betel

How Many Replicates are Sufficient for Differential Gene Expression?

Posted on by

In a nicely done, convincing 2 condition, 48 replicate RNA-Seq experiment, researchers from the University of Dundee aimed to answer a frequently asked question in the field and on Genohub.com, ‘How many replicates are necessary for differential gene expression (DGE)?’. In their study they examined three statistical models to see which best represented read-count distribution of genes from commonly used DGE tools.

Using the statistical power of 48 replicates they determined that inter-lane variability does not play a large role in DGE results. Assuming even loading and amplification their results showed a Poisson distribution of counts from individual genes. The authors also determined that read count distribution was consistent with a negative binomial model, an assumption in widely used tools such as edgR, DESeq, cuffdiff and baySeq. Performing a goodness-of-fit test for log-normal, negative binomial and normal distributions the authors demonstrated that inclusion of ‘bad replicates’ made results inconsistent with the statistical models they tested, complicating the interpretation of differential expression results. A bad replicate was defined as 1) one that poorly correlated with other replicates, 2) a replicate with atypical read counts, 3) one having non-uniform read depth profiles. 

So how many replicates are sufficient for differential gene expression? The authors sequenced 96 mRNA samples in 7 1×50 HiSesq lanes. The cost for this on Genohub today is $28k USD. When the authors removed 6-8 bad replicates from their pool of 48 samples, their data became consistent with a negative binomial distribution. Assuming experimental variability similar to the authors, this indicates at least 6 replicates in a DGE experiment is good practice. The cost for the preparation of 6 RNA-seq libraries and sequencing is $2,500 USD.  

Doing a literature search and observing client behavior on Genohub, we estimate ~80% of those studying DGE use 3 replicates in their experiments, which with dropouts and variation is unsatisfactory. A final point we’d like to make is the authors used ~11M, 1×50 reads/sample, which goes to show that with DGE, replicates can be more important than read depth. This is further discussed in our Coverage and Read Depth Guide. 

At Genohub, we help consult and design sequencing experiments with users. Determining the replicates needed for a study is a common question that needs to be answered. Unfortunately, too few studies examine these fundamental elements in sequencing design. We hope this article gets the recognition it deserves. 

Reference: 

Statistical models for RNA-seq data derived from a two-condition 48-replicate experiment Marek Gierliński, Christian Cole, Pietà Schofield, Nicholas J. Schurch, Alexander Sherstnev, Vijender Singh, Nicola Wrobel, Karim Gharbi, Gordon Simpson, Tom Owen-Hughes, Mark Blaxter, Geoffrey J. Barton