A fast, lock-free approach for efficient parallel counting of occurrences of k-mers
Guillaume Marçais,Carl Kingsford +1 more
TL;DR: This work proposes a new k-mer counting algorithm and associated implementation, called Jellyfish, which is fast and memory efficient, based on a multithreaded, lock-free hash table optimized for counting k-mers up to 31 bases in length.
read more
Abstract: Motivation: Counting the number of occurrences of every k-mer (substring of length k) in a long string is a central subproblem in many applications, including genome assembly, error correction of sequencing reads, fast multiple sequence alignment and repeat detection. Recently, the deep sequence coverage generated by next-generation sequencing technologies has caused the amount of sequence to be processed during a genome project to grow rapidly, and has rendered current k-mer counting tools too slow and memory intensive. At the same time, large multicore computers have become commonplace in research facilities allowing for a new parallel computational paradigm.
Results: We propose a new k-mer counting algorithm and associated implementation, called Jellyfish, which is fast and memory efficient. It is based on a multithreaded, lock-free hash table optimized for counting k-mers up to 31 bases in length. Due to their flexibility, suffix arrays have been the data structure of choice for solving many string problems. For the task of k-mer counting, important in many biological applications, Jellyfish offers a much faster and more memory-efficient solution.
Availability: The Jellyfish software is written in C++ and is GPL licensed. It is available for download at http://www.cbcb.umd.edu/software/jellyfish.
Contact: [email protected]
Supplementary information:Supplementary data are available at Bioinformatics online.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
The genome of the black cutworm Agrotis ipsilon
Yaohui Wang,Gangqi Fang,Xien Chen,Yanghui Cao,Ningning Wu,Qian Cui,Chenxu Zhu,Lansa Qian,Yongping Huang,Shuai Zhan +9 more
TL;DR: The black cutworm (BCW), Agrotis ipsilon, is a worldwide polyphagous and underground pest that causes a high level of economic loss to a wide range of crops through the damage of roots as mentioned in this paper.
16
Genome survey sequencing and genetic diversity of cultivated Akebia trifoliata assessed via phenotypes and SSR markers.
TL;DR: In this article, the size and characteristics of the genome of A. trifoliata were reported for the first time, which greatly enriched the genomic resources of this plant for further research and utilization.
16
Structural variation in the pangenome of wild and domesticated barley
Murukarthick Jayakodi,Qiongxian Lu,Hélène Pidon,M. Timothy Rabanus‐Wallace,Micha Bayer,Thomas Lux,Yu Guo,Benjamin Jaegle,Ana Badea,Wubishet A. Bekele,Gurcharn S. Brar,Katarzyna Braune,Boyke Bunk,K. J. Chalmers,Brett Chapman,Morten Egevang Jørgensen,Jia‐Wu Feng,Manuel Feser,Anne Fiebig,Heidrun Gundlach,Wenbin Guo,Georg Haberer,Mats Hansson,Axel Himmelbach,Iris Hoffie,Robert E. Hoffie,Haifei Hu,Sachiko Isobe,Patrick König,Sandip M. Kale,Nadia Kamal,Gabriel Keeble‐Gagnère,Beat Keller,Manuela Knauft,Ravi Koppolu,Simon G. Krattinger,Jochen Kumlehn,Peter Langridge,Chengdao Li,Marina Püpke Marone,Andreas Maurer,Klaus Mayer,Michael Melzer,Gary J. Muehlbauer,Emiko Murozuka,Sudharsan Padmarasu,Dragan Perović,Klaus Pillen,Pierre A. Pin,Curtis Pozniak,Luke Ramsay,Pai Pedas,Twan Rutten,Shun Sakuma,Kazuhiro Sato,Danuta Schüler,Thomas Schmutzer,Uwe Scholz,Miriam Schreiber,Kenta Shirasawa,Craig G. Simpson,Birgitte Skadhauge,M. Spannagl,Brian J. Steffenson,Hanne Cecilie Thomsen,Josquin Tibbits,Martin Toft Simmelsgaard Nielsen,Corinna Trautewig,Dominique Vequaud,Cynthia Voss,Penghao Wang,Robbie Waugh,Sharon Westcott,Magnus Wohlfahrt Rasmussen,Runxuan Zhang,Xiao‐Qi Zhang,Thomas Wicker,Christoph Dockter,Martin Mascher,Nils Stein +79 more
TL;DR: This study examines structural variation in the pangenome of wild and domesticated barley, comparing annotated genome sequences of multiple individuals to understand genetic differences and evolutionary relationships between the two species.
16
RapMap: A Rapid, Sensitive and Accurate Tool for Mapping RNA-seq Reads to Transcriptomes
TL;DR: It is demonstrated how quasi-mapping can be successfully applied to the problems of transcript-level quantification from RNA-seq reads and the clustering of contigs from de novo assembled transcriptomes into biologically-meaningful groups.
16
The genetic architecture of phenotypic diversity in the betta fish (Betta splendens)
Zhang W,Haobin Wang,Brandt Dyc,Hu B,Sheng J,Wang M,Luo H,Guo S,Sheng B,Zeng Q,Peng K,Zhao D,Jian S,Wu D,Jianbin Wang,van Esch Jhm,Shi W,Ren J,Rasmus Nielsen,Rasmus Nielsen,Hong Y +20 more
TL;DR: In this article, a high-quality genome assembly and re-sequencing of 727 individuals representing diverse morphologies of the betta fish was performed using GWAS, identifying the genetic basis of multiple traits, including several coloration phenotypes, sex-determination which they map to DMRT1 and the long-fin phenotype which maps to KCNJ15.
References
MUSCLE: multiple sequence alignment with high accuracy and high throughput
TL;DR: MUSCLE is a new computer program for creating multiple alignments of protein sequences that includes fast distance estimation using kmer counting, progressive alignment using a new profile function the authors call the log-expectation score, and refinement using tree-dependent restricted partitioning.
45.1K
•Book
Introduction to Algorithms
Thomas H. Cormen,Charles E. Leiserson,Ronald L. Rivest +2 more
- 01 Jan 1990
TL;DR: The updated new edition of the classic Introduction to Algorithms is intended primarily for use in undergraduate or graduate courses in algorithms or data structures and presents a rich variety of algorithms and covers them in considerable depth while making their design and analysis accessible to all levels of readers.
24.8K
MapReduce: simplified data processing on large clusters
Jeffrey Dean,Sanjay Ghemawat +1 more
- 06 Dec 2004
TL;DR: This paper presents the implementation of MapReduce, a programming model and an associated implementation for processing and generating large data sets that runs on a large cluster of commodity machines and is highly scalable.
MapReduce: simplified data processing on large clusters
Jeffrey Dean,Sanjay Ghemawat +1 more
TL;DR: This presentation explains how the underlying runtime system automatically parallelizes the computation across large-scale clusters of machines, handles machine failures, and schedules inter-machine communication to make efficient use of the network and disks.
A Whole-Genome Assembly of Drosophila
Eugene W. Myers,Granger G. Sutton,Arthur L. Delcher,Ian M. Dew,Dan P. Fasulo,Michael Flanigan,Saul A. Kravitz,Clark M. Mobarry,Knut Reinert,Karin A. Remington,Eric L. Anson,Randall Bolanos,Hui-Hsien Chou,Catherine Jordan,Aaron L. Halpern,Stefano Lonardi,Ellen M. Beasley,Rhonda C. Brandon,Lin Chen,Patrick J. Dunn,Zhongwu Lai,Yong Liang,Deborah R. Nusskern,Ming Zhan,Qing Zhang,Xiangqun Zheng,Gerald M. Rubin,Mark Raymond Adams,J. Craig Venter +28 more
TL;DR: The quality of a whole-genome assembly of Drosophila melanogaster and the nature of the computer algorithms that accomplished it are reported on and should be of substantial value to the scientific community.
1.6K