Distributed storage systems play an increasingly critical role in data centers to meet the ever-increasing data growth demand. Heterogeneous storage systems, with the coexistence of hard disk drives (HDDs) and solid state drives (SSDs), can be an attractive distributed store solution due to the balanced performance, large capacity, and economic cost. The consistent hashing distribution algorithm that is widely used in distributed storage systems can achieve scalable and flexible mappings, but do not take full advantages of different characteristics of heterogeneous devices. In this research, we propose a hierarchical consistent hashing (HiCH) algorithm to better manage data distribution in a heterogeneous object-based storage system and better explore the potential of heterogeneous devices. HiCH divides heterogeneous storage devices into different buckets and applies separate consistent hashing rings for each bucket. It places data into various hashing rings according to the hotness, access time, and other data access patterns. The evaluation based on the Sheepdog, a distributed objectbased storage system, confirms that HiCH can improve the performance of storage systems, and also make better utilization of heterogeneous storage devices.

Hierarchical Consistent Hashing for Heterogeneous Object-Based Storage

In Internet of Things (IoT) environments, there are multiple sensors and devices monitoring different metrics and producing massive amounts of data. Monitoring of metrics can be done at different frequencies. Systems and applications that consume monitoring data typically use constrained IT resources, e.g., constrained network facilities, storage, display, processing/computing power, and energy. Given the limited quantity of resources used by these monitoring systems and applications, it is impossible to be able to collect data of all metrics in the application’s context with a very high monitoring frequency. Additionally, changes in the IoT environmental context may affect the choice of metrics that should be monitored and their monitoring frequencies. To address these issues, we propose in this paper a novel approach based on optimization model to optimally determine the metrics that should be monitored and the frequencies at which these metrics are to be monitored. Our approach is an adaptive iterative approach in which the metric frequencies are re-optimized whenever an environmental event or a monitored metric value triggers the need for such re-optimization. We also present a proof-of-concept implementation of our approach that shows the efficiency of adopting it.

An Optimization Approach for Adaptive Monitoring in IoT Environments

SSDs are emerging storage devices which unlike HDDs, do not have mechanical parts and therefore, have superior performance compared to HDDs. Due to the high cost of SSDs, entirely replacing HDDs with SSDs is not economically justified. Additionally, SSDs can endure a limited number of writes before failing. To mitigate the shortcomings of SSDs while taking advantage of their high performance, SSD caching is practiced in both academia and industry. Previously proposed caching architectures have only focused on either performance or endurance and neglected to address both parameters in suggested architectures. Moreover, the cost, reliability, and power consumption of such architectures is not evaluated. This paper proposes a hybrid I/O caching architecture that while offers higher performance than previous studies, it also improves power consumption with a similar budget. The proposed architecture uses DRAM, Read-Optimized SSD, and Write-Optimized SSD in a three-level cache hierarchy and tries to efficiently redirect read requests to either DRAM or RO-SSD while sending writes to WO-SSD. To provide high reliability, dirty pages are written to at least two devices which removes any single point of failure. The power consumption is also managed by reducing the number of accesses issued to SSDs. The proposed architecture reconfigures itself between performance- and endurance-optimized policies based on the workload characteristics to maintain an effective tradeoff between performance and endurance. We have implemented the proposed architecture on a server equipped with industrial SSDs and HDDs. The experimental results show that as compared to state-of-the-art studies, the proposed architecture improves performance and power consumption by an average of 8% and 28%, respectively, and reduces the cost by 5% while increasing the endurance cost by 4.7% and negligible reliability penalty.

An Efficient Hybrid I/O Caching Architecture Using Heterogeneous SSDs

Systems, methods, and computer programs are disclosed for scheduling tasks in a heterogeneous processor cluster architecture in a portable computing device. One embodiment is a system comprising a first processor cluster and a second processor cluster. The first processor cluster comprises a first shared cache, and the second processor cluster comprises a second shared cache. The system further comprises a controller in communication with the first and second processor clusters for performing task migration between the first and second processor clusters. The controller initiates execution of a task on a first processor in the first processor cluster. The controller monitors a processor workload for the first processor and a cache demand associated with the first shared cache while the task is running on the first processor in the first processor cluster. The controller migrates the task to the second processor cluster based on the processor workload and the cache demand.

Systems and methods for scheduling tasks in a heterogeneous processor cluster architecture using cache demand monitoring

JointCloud is a cross-cloud cooperation architecture for integrated Internet service customization. The customized cross-cloud storage service based on this architecture is called JointCloud storage. Storing the Internet of Things (IoT) big data in erasure-coded JointCloud storage systems ensures that data can be accessed when several cloud services interrupt. However, because existing erasure codes cannot adapt the generator matrix and data placement scheme to different network environments and encoding parameters, they usually incur a large network resource consumption (NRC) for repairing data in JointCloud storage systems. As a result, the availability of IoT applications running on JointCloud storage systems is impaired. In this article, to minimize the NRC of repairing data, we propose an adaptive erasure code for JointCloud storage of IoT big data called ACIoT. Specifically, we first propose the concept of average weighted locality (AWL) of a stripe of erasure-coded data, which is proportional to the average NRC of repairing this stripe in JointCloud storage systems. Then, we propose an active parallel trial-and-error algorithm to calculate the optimal generator matrix and data placement scheme to achieve the lowest AWL, under different network environments and encoding parameters. By encoding and placing each stripe of data with the optimal generator matrix and data placement scheme, ACIoT can achieve the minimum NRC. The experiments show that, compared with several state-of-the-art erasure codes, ACIoT reduces the NRC by 26.4%–44.7%.

An Adaptive Erasure Code for JointCloud Storage of Internet of Things Big Data

Virtualization technology enables multiple virtual machines (VMs) to share a single physical server. Commercial servers increasingly use the nonuniform memory access (NUMA) architecture due to its scalable memory performance. However, multiple VMs running on a NUMA physical server will cause performance overheads such as remote memory access latency and shared microarchitectural resource contention, which makes the VM performance less efficient and stable. These performance overheads are mainly caused by memory traffic from data-intensive workloads. In this paper, we propose a traffic-aware VM optimization (TAVO) scheme on NUMA systems. Based on the performance monitoring of the data traffic and CPU/memory resource usages in the system, TAVO addresses VM memory access locality and shared resource contention problems via automatic VM initial placement and NUMA-aware VM online scheduling. Our experimental results show that TAVO improves VM performance in terms of benchmark runtime by up to 22.6% compared with the default KVM CFS scheduler. TAVO also achieves a much stable performance with benchmark's average runtime variation under 3%.

Performance-Monitoring-Based Traffic-Aware Virtual Machine Deployment on NUMA Systems

The fault-tolerance property in most cloud storage systems are designed within the scale of a single datacenter. The single datacenter as a whole may be unreachable or crashed due to severe problems, such as broken network links, power supply interruptions, and natural disasters, etc. Therefore, the design of an effective cross-datacenter fault-tolerant storage system is important to protect data security in the cloud. However, building a cross-datacenter fault-tolerant system faces great challenges, such as high latency, low throughput, high costs of bandwidth resources between datacenters. In this paper, we propose a practical cross-datacenter fault-tolerant (CDFT) algorithm in the cloud storage system. Our fault-tolerant algorithm design considers the difficult tradeoffs among fault tolerance, latency, throughput, network and storage costs. We propose the Domain Fault Codes (DFC) and the topology-aware scheduling techniques, which can tolerate the whole datacenter breakdown. We implemented the DFC-CDFT algorithm in a prototype cloud storage system. The experimental results showed that the proposed DFC-CDFT algorithm can effectively recover data blocks from the single datacenter failure while achieves low storage and bandwidth costs.

A practical cross-datacenter fault-tolerance algorithm in the cloud storage system

The invention discloses a virtual machine coexisting scheduling method based on processor performance monitoring. The virtual machine coexisting scheduling method comprises the following steps: based on performance events provided by a hardware processor performance monitoring unit and based on pressure values and tolerance degrees of VCPUs (Virtual Central Processing Units) running on a multi-core physical server, calculating to obtain the pressure coefficients and tolerance coefficients of the VCPUs, determining a VPCU to be moved out from high-load physical CPUs (Central Processing Units) according to the pressure coefficients and the tolerance coefficients, and through coexisting coefficient prediction, moving the VCPU to be moved out to a physical CPU with a minimum coexisting coefficient. When the virtual machine coexisting scheduling method disclosed by the invention is used for predicting the performance of the VCPU coexisting combination, VCPU coexisting coefficients are introduced to represent the performance of different VCPU coexisting combinations, a coexisting coefficient prediction method based on the VCPU coexisting combinations is provided, favorable decision information is provided for virtual machine scheduling, a virtual machine in match with the use situation of a shared cache is scheduled to the same CPU of the shared cache for execution, and the purpose of improving the operation efficiency of the virtual machine is achieved.

Virtual machine coexisting scheduling method based on processor performance monitoring

The invention discloses a virtual machine scheduling method based on coexisting coefficient balance. The virtual machine scheduling method comprises the following steps: based on pressure values and tolerance degrees of VCPUs (Virtual Central Processing Units) running on a multi-core physical server, calculating to obtain the pressure coefficients and tolerance coefficients of the VCPUs, calculating the existing coefficients of the VPCUs on shared caches and the coexisting coefficients of the shared caches according to the pressure coefficients and the tolerance coefficients, judging whether the coexisting coefficients of the shared cache in a virtual machine system are in balance or not according to the coexisting coefficients and set thresholds; when the coexisting coefficients are not in balance, performing VCPU scheduling according to the coexisting coefficients of the VCPUs, so as to keep the coexisting coefficients of the shared caches in balance. By adopting the virtual machine scheduling method, the problem that virtual machines compete with one another to obtain shared cache resource, a memory controller and memory bandwidth resource in a multi-core processor is effectively solved, the situation that the virtual machines compete for and use the shared resource is identified, the operation environment of the virtual machines is optimized, and the service quality of the virtual machines is improved.

Virtual machine scheduling method based on coexisting coefficient balance

The invention discloses a virtual machine scheduling multi-strategy selection method based on micro-architecture perception. The virtual machine scheduling multi-strategy selection method comprises the following steps: acquiring performance characteristic information of virtual machines, setting scheduling algorithms for the virtual machines from higher priority to lower priority of set performance characteristics according to operation performance characteristic information of the virtual machines, dividing the virtual machines to corresponding queues according to the scheduling algorithms, judging the change of each queue according to a set queue priority, and scheduling corresponding scheduling algorithms according to the change of the queues. As the virtual machine scheduling multi-strategy selection method effectively utilizes a potential performance improvement space in a NUMA (Non Uniform Memory Access) multi-core system, the overall performance of a virtualization system is improved, the physical resource use cost is lowered, and meanwhile other management expenses such as energy expense are reduced.

Virtual machine scheduling multi-strategy selection method based on micro-architecture perception

Popular Tasks used by Researchers

How can I help with your research?

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

We encountered an error while loading the panel. Please try refreshing the page.

Something went wrong

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

Anyone with the link can view this chat. You can switch it back to private anytime.

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Xinjie Yu

Author Tools

Chat about Author