Biomedical data are growing exponentially in both volume and levels of complexity, due to the rapid advancement of technologies and research methodologies. Analyzing these large datasets, referred to collectively as "big data," has become an integral component of research that guides experimentation-driven discovery and a new engine of discovery itself as it uncovers previously unknown connections through mining of existing data. To fully realize the potential of big data, biomedical researchers need access to high-performance-computing (HPC) resources. However, supporting on-premises infrastructure that keeps up with these consistently expanding research needs presents persistent financial and staffing challenges, even for well-resourced institutions. For other institutions, including primarily undergraduate institutions and minority serving institutions, that educate a large portion of the future workforce in the USA, this challenge presents an insurmountable barrier. Therefore, new approaches are needed to provide broad and equitable access to HPC resources to biomedical researchers and students who will advance biomedical research in the future. 

NIGMS Sandbox: a learning platform toward democratizing cloud computing for biomedical research

Non-volatile memory (NVM) offers lower costs per capacity and higher total capacities than DRAM. However, NVM cannot simply be used as a drop-in replacement for DRAM in database management systems due to its different performance characteristics. We thus investigate the placement of column-store data structures in a hybrid hierarchy of DRAM and NVM, with the goal of placing as much data as possible in NVM without compromising performance. After analyzing how different memory access patterns affect query runtimes when columns are placed in NVM, we propose a heuristic that leverages lightweight access counters to suggest which structures should be placed in DRAM and which in NVM. Our evaluation using TPC-H shows that more than 80% of the data touched by queries can be placed in NVM with almost no slowdown, while naively placing all data in NVM would increase runtime by 53%.

https://dl.acm.org/doi/pdf/10.1145/3465998.3466008

Workload-Driven Placement of Column-Store Data Structures on DRAM and NVM

By utilizing identity-based signatures rather than hash-based protocols, the goal is to enhance the reliability of cloud storage services. The innovative identity-based signature and the hash-based protocol are two components of this paradigm that provide universal access to a mutual pool of adjustable computer resources. This access can be granted on demand. Several publications that have been published are used to compile the dataset for trusted cloud storage services. Using the cloud analyst, the analysis was carried out and the implementation was conducted. There are two groups for when there are two algorithms. With a G power of 80%, a threshold of 0.05, and a confidence interval of 95%, the sample size for each group amount to 36.IBM SPSS is the program that was utilized in order to achieve the results. The data indicated that there was a substantial disparity between the implementation of the Identity Based Signature technique and its Minimized signing time. Given that the significance value of the independent sample $\mathbf{T}$ test is 0.000 $(\mathrm{p} < 0.05)$, it may be concluded that there is a statistically significant difference between the two algorithms of interest. That is a reduction of thirty to forty percent in comparison to the Hash Based Protocol. On the basis of the findings, it is possible to draw the conclusion that the Hash Board Protocol Algorithm (0.0027 MB/Sec) is less secure than the Identity Based Signature Algorithm $(0.0000\ \text{MB}/\text{Sec})$. •

Improving Trusted Cloud Storage Services using Identity Based Signature Over Hash Based Protocol

Evaluating Lightweight Integer Compression Algorithms in Column-Oriented In-Memory DBMS.

We study the problem of optimizing data storage and access costs on the cloud while ensuring that the desired performance or latency is unaffected. We first propose an optimizer that optimizes the data placement tier (on the cloud) and the choice of compression schemes to apply, for given data partitions with temporal access predictions. Secondly, we propose a model to learn the compression performance of multiple algorithms across data partitions in different formats to generate compression performance predictions on the fly, as inputs to the optimizer. Thirdly, we propose to approach the data partitioning problem fundamentally differently than the current default in most data lakes where partitioning is in the form of ingestion batches. We propose access pattern aware data partitioning and formulate an optimization problem that optimizes the size and reading costs of partitions subject to access patterns. We study the various optimization problems theoretically as well as empirically, and provide theoretical bounds as well as hardness results. We propose a unified pipeline of cost minimization, called SCOPe that combines the different modules. We extensively compare the performance of our methods with related baselines from the literature on TPC-H data as well as enterprise datasets (ranging from GB to PB in volume) and show that SCOPe substantially improves over the baselines. We show significant cost savings compared to platform baselines, of the order of 50% to 83% on enterprise Data Lake datasets that range from terabytes to petabytes in volume.

pdf/towards-optimizing-storage-costs-on-the-cloud-19b0vh4w.pdf

Towards Optimizing Storage Costs on the Cloud

Enterprises increasingly move their databases into the cloud. As a result, database-as-a-service providers are challenged to meet the performance guarantees assured in service-level agreements (SLAs) while keeping hardware costs as low as possible. Being cost-effective is particularly crucial for cloud databases where the provisioned amount of DRAM dominates the hardware costs. A way to decrease the memory footprint is to leverage access skew in the workload by moving rarely accessed cold data to cheaper storage layers and retaining only frequently accessed hot data in main memory. In this paper, we present SAHARA, an advisor that proposes a table partitioning for column stores with minimal memory footprint while still adhering to all performance SLAs. SAHARA collects lightweight workload statistics, classifies data as hot and cold, and calculates optimal or near-optimal range partitioning layouts with low optimization time using a novel cost model. We integrated SAHARA into a commercial cloud database and show in our experiments for real-world and synthetic benchmarks a memory footprint reduction of 2.5 × while still fulfilling all performance SLAs provided by the customer or advertised by the DBaaS provider.

SAHARA: Memory Footprint Reduction of Cloud Databases with Automated Table Partitioning

Precise, Compact, and Fast Data Access Counters for Automated Physical Database Design.

As biomedical research data grow, researchers need reliable and scalable solutions for storage and compute. There is also a need to build systems that encourage and support collaboration and data sharing, to result in greater reproducibility. This has led many researchers and organizations to use cloud computing [1]. The cloud not only enables scalable, on-demand resources for storage and compute, but also collaboration and continuity during virtual work, and can provide superior security and compliance features. Moving to or adding cloud resources, however, is not trivial or without cost, and may not be the best choice in every scenario. The goal of this workshop is to explore the benefits of using the cloud in biomedical and computational research, and considerations (pros and cons) for a range of scenarios including individual researchers, collaborative research teams, consortia research programs, and large biomedical research agencies / organizations.

https://doi.org/10.1142/9789811270611_0049

Biomedical research in the Cloud: considerations for researchers and organizations moving to (or adding) cloud computing resources.

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Nick Weber

Author Tools

Chat about Author

Papers

SAHARA: Memory Footprint Reduction of Cloud Databases with Automated Table Partitioning

Precise, Compact, and Fast Data Access Counters for Automated Physical Database Design.

Biomedical research in the Cloud: considerations for researchers and organizations moving to (or adding) cloud computing resources.