From f1e78720aab9340dc551208fe75b16fc4986b7dd Mon Sep 17 00:00:00 2001 From: Heng Li Date: Sat, 27 Jun 2020 14:22:39 -0400 Subject: [PATCH] more getting started; added NA12878 and links --- README.md | 78 +++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 55 insertions(+), 23 deletions(-) diff --git a/README.md b/README.md index d40520d..dda3035 100644 --- a/README.md +++ b/README.md @@ -4,8 +4,20 @@ # Install hifiasm (requiring g++ and zlib) git clone https://github.com/chhylp123/hifiasm cd hifiasm && make -# Assembly -./hifiasm -o NA12878.asm -t 32 NA12878.fq.gz + +# Run on test data (use -f0 for small datasets) +wget https://github.com/chhylp123/hifiasm/releases/download/v0.7/chr11-2M.fa.gz +./hifiasm -o test -t4 -f0 chr11-2M.fa.gz 2> test.log # this takes ~90 sec + +# Assemble inbred/homozygous genomes (-l0 disables duplication purging) +hifiasm -o CHM13.asm -t32 -l0 CHM13-HiFi.fa.gz +# Assemble heterozygous with built-in duplication purging +hifiasm -o HG002.asm -t32 HG002-file1.fq.gz HG002-file2.fq.gz + +# Trio binning assembly (requiring https://github.com/lh3/yak) +yak count -b37 -t16 -o pat.yak <(cat pat_1.fq.gz pat_2.fq.gz) <(cat pat_1.fq.gz pat_2.fq.gz) +yak count -b37 -t16 -o mat.yak <(cat mat_1.fq.gz mat_2.fq.gz) <(cat mat_1.fq.gz mat_2.fq.gz) +hifiasm -o HG002.asm -t32 -1 pat.yak -2 mat.yak HG002-HiFi.fa.gz ``` ## Introduction @@ -28,7 +40,7 @@ outputs consist of: 4. Alternate assembly contig graph (*prefix*.a\_ctg.gfa). This graph consists of all assemblies that are discarded in primary contig graph. -For trio assembly, the input of hifiasm is the PacBio Hifi reads in fasta/fastq format, and the paternal/maternal trio indexes generated by `yak count` (see https://github.com/lh3/yak). The outputs consist of: +For trio assembly, the input of hifiasm is the PacBio Hifi reads in fasta/fastq format, and the paternal/maternal trio indexes generated by [yak count](https://github.com/lh3/yak). The outputs consist of: 1. Haplotype-resolved raw [unitig][unitig] graph in [GFA][gfa] format (*prefix*.r\_utg.gfa). This graph keeps all haplotype information. @@ -39,37 +51,59 @@ For trio assembly, the input of hifiasm is the PacBio Hifi reads in fasta/fastq maternal/haplotype2 assembly. - In addition, hifiasm also outputs three binary files that save all overlap information (*prefix*.ec.bin, *prefix*.ovlp.reverse.bin, *prefix*.ovlp.source.bin). With these files, hifiasm can avoid the time-consuming all-to-all overlap calculation step, and do the assembly directly and quickly. This might be helpful when you want to get an optimized assembly by multiple rounds of experiments with different parameters. Hifiasm is a standalone and lightweight assembler, which does not need external libraries (except zlib). For large genomes, it can generate high-quality primary -assembly in a few hours. Hifiasm has been tested on various large and complex datasets. +assembly in several hours. Hifiasm has been tested on various large and complex datasets. The results are as follows: -|Dataset|GSize|Cov|Asm options|CPU time|Wall time|RAM| N50| +|Dataset|Size|Cov.|Asm options|CPU time|Wall time|RAM| N50| |:---------------|-----:|-----:|:---------------------|-------:|--------:|----:|----------------:| -|[\[Mouse (C57/BL6J)\]](https://www.ncbi.nlm.nih.gov/sra/?term=SRR11606870)|2.7Gb|x25|-t48 -l0|172.9h|4.8h|76G|21.1Mb| -|[\[Maize (B73)\]](https://www.ncbi.nlm.nih.gov/sra/?term=SRR11606869)|2.2Gb|x22|-t48 -l0|203.2h|5.1h|68G|36.7Mb| -|[\[Strawberry\]](https://www.ncbi.nlm.nih.gov/sra/?term=SRR11606867)|0.8Gb|x36|-t48 -D10|152.7h|3.7h|91G|17.8Mb| -|[\[Mountain yellow-legged frog\]](https://www.ncbi.nlm.nih.gov/sra?term=(SRR11606868)%20OR%20SRR12048570)|9.0Gb|x29|-t48|2834.3h|69.0h|463G|9.3Mb| -|[\[Redwood\]](https://www.ncbi.nlm.nih.gov/sra/?term=SRP251156)|26.5Gb|x28|-t64|3890.3h|65.5h|699G|5.4Mb| -|[\[CHM13\]](https://www.ncbi.nlm.nih.gov/sra?term=(((SRR11292120)%20OR%20SRR11292121)%20OR%20SRR11292122)%20OR%20SRR11292123)|3.1Gb|x32|-t48 -l0|310.7h|8.2h|114G|88.9Mb| -|[\[HG00733\]](https://www.ebi.ac.uk/ena/data/view/ERX3831682)|3.1Gb|x33|-t48|269.1h|6.9h|135G|69.9Mb| -|[\[HG002\]](https://www.ncbi.nlm.nih.gov/sra?term=(((SRR10382244)%20OR%20SRR10382245)%20OR%20SRR10382248)%20OR%20SRR10382249)|3.1Gb|x36|-t48|305.4h|7.7h|137G|98.7Mb| +|[Mouse (C57/BL6J)][mouse-data]|2.6Gb |×25|-t48 -l0 |172.9h |4.8h |76G |21.1Mb| +|[Maize (B73)][maize-data] |2.2Gb |×22|-t48 -l0 |203.2h |5.1h |68G |36.7Mb| +|[Strawberry][strawberry-data] |0.8Gb |×36|-t48 -D10|152.7h |3.7h |91G |17.8Mb| +|[Frog][frog-data] |9.5Gb |×29|-t48 |2834.3h|69.0h|463G|9.3Mb| +|[Redwood][redwood-data] |35.6Gb|×28|-t80 |3890.3h|65.5h|699G|5.4Mb| +|[Human (CHM13)][CHM13-data] |3.1Gb |×32|-t48 -l0 |310.7h |8.2h |114G|88.9Mb| +|[Human (HG00733)][HG00733-data]|3.1Gb|×33|-t48 |269.1h |6.9h |135G|69.9Mb| +|[Human (HG002)][NA24385-data] |3.1Gb |×36|-t48 |305.4h |7.7h |137G|98.7Mb| +[mouse-data]: https://www.ncbi.nlm.nih.gov/sra/?term=SRR11606870 +[maize-data]: https://www.ncbi.nlm.nih.gov/sra/?term=SRR11606869 +[strawberry-data]: https://www.ncbi.nlm.nih.gov/sra/?term=SRR11606867 +[frog-data]: https://www.ncbi.nlm.nih.gov/sra?term=(SRR11606868)%20OR%20SRR12048570 +[redwood-data]: https://www.ncbi.nlm.nih.gov/sra/?term=SRP251156 +[CHM13-data]: https://www.ncbi.nlm.nih.gov/sra?term=(((SRR11292120)%20OR%20SRR11292121)%20OR%20SRR11292122)%20OR%20SRR11292123 Hifiasm also can produce high-quality fully resolved assembly. We tested it on the following trio-binning datasets: -|Dataset|GSize|Cov|CPU time|Wall time|RAM| N50| -|:---------------|-----:|-----:|-------:|--------:|----:|----------------:| -|HG00733: [\[child\]](https://www.ebi.ac.uk/ena/data/view/ERX3831682), [\[father\]](https://www.ebi.ac.uk/ena/data/view/ERR3241754), [\[mother\]](https://www.ebi.ac.uk/ena/data/view/ERR3241755)|3.1Gb|x33 (child), x30 (parent)|269.1h|6.9h|135G|35.1Mb (hap1), 34.9Mb (hap2)| -|HG002: [\[child\]](https://www.ncbi.nlm.nih.gov/sra?term=(((SRR10382244)%20OR%20SRR10382245)%20OR%20SRR10382248)%20OR%20SRR10382249), parent from GIAB[1]|3.1Gb|x36 (child), x30 (parent)|305.4h|7.7h|137G|41.0Mb (hap1), 40.8Mb (hap2)| - -[1] GIAB’s “Homogeneity Run01” short-read runs were used for the HG002 trio. These HG002 reads were downsampled to 30-fold coverage. +|Dataset|Cov.|CPU time|Elapsed time|RAM| N50| +|:---------------|-----:|-------:|--------:|----:|----------------:| +|[HG00733][HG00733-data], [\[father\]][HG00731-data], [\[mother\]][HG00732-data]|×33|269.1h|6.9h|135G|35.1Mb (paternal), 34.9Mb (maternal)| +|[HG002][NA24385-data], [\[father\]][NA24149-data], [\[mother\]][NA24143-data]|×36|305.4h|7.7h|137G|41.0Mb (paternal), 40.8Mb (maternal)| +|[NA12878][NA12878-data], [\[father\]][NA12891-data], [\[mother\]][NA12892-data]|×30|180.8h|4.9h|123G|27.7Mb (paternal), 27.0Mb (maternal)| +[HG00733-data]: https://www.ebi.ac.uk/ena/data/view/ERX3831682 +[HG00731-data]: https://www.ebi.ac.uk/ena/data/view/ERR3241754 +[HG00732-data]: https://www.ebi.ac.uk/ena/data/view/ERR3241755 +[NA24385-data]: https://www.ncbi.nlm.nih.gov/sra?term=(((SRR10382244)%20OR%20SRR10382245)%20OR%20SRR10382248)%20OR%20SRR10382249 +[NA24149-data]: https://ftp-trace.ncbi.nlm.nih.gov/giab/ftp/data/AshkenazimTrio/HG003_NA24149_father/NIST_HiSeq_HG003_Homogeneity-12389378/HG003Run01-13262252/ +[NA24143-data]: https://ftp-trace.ncbi.nlm.nih.gov/giab/ftp/data/AshkenazimTrio/HG004_NA24143_mother/NIST_HiSeq_HG004_Homogeneity-14572558/HG004Run01-15133132/ +[NA12878-data]: https://ftp-trace.ncbi.nlm.nih.gov/giab/ftp/data/NA12878/PacBio_SequelII_CCS_11kb/ +[NA12891-data]: https://www.ebi.ac.uk/ena/data/view/ERR194160 +[NA12892-data]: https://www.ebi.ac.uk/ena/data/view/ERR194161 +Except NA12878, the assemblies above were produced by hifiasm v0.7 and can be +downloaded at +```txt +ftp://ftp.dfci.harvard.edu/pub/hli/hifiasm/submission/v0.7/ +``` +NA12878 was assembled with a more recent version of hifiasm and is available at +```txt +ftp://ftp.dfci.harvard.edu/pub/hli/hifiasm/NA12878-r253/ +``` ## Usage @@ -98,12 +132,10 @@ the assembly quality, adapters should be removed by `-z` as follow: In this example, hifiasm will remove 20 bases from both ends of each read. For trio assembly, first the trio indexes of paternal/maternal should be generated by -`yak count` (see https://github.com/lh3/yak): +[yak count](https://github.com/lh3/yak): ```sh ./yak count -k31 -b37 -t16 -o mat.yak mat.fq.gz -``` -```sh ./yak count -k31 -b37 -t16 -o pat.yak pat.fq.gz ```