The advent of cloud computing promises to provide computational resources to customers like public utilities such as water and electricity. To deal with dynamically fluctuating resource demands, market-driven resource allocation has been proposed and recently implemented by public Infrastructure-as-a-Service (IaaS) providers like Amazon EC2. In this environment, cloud resources are offered in distinct types of virtual machines (VMs) and the cloud provider runs an auction-based market for each VM type with the goal of achieving maximum revenue over time. However, as demand for each type of VMs can fluctuate over time, it is necessary to adjust the capacity allocated to each VM type to match the demand in order to maximize total revenue while minimizing the energy cost. In this paper, we consider the case of a single cloud provider and address the question how to best match customer demand in terms of both supply and price in order to maximize the providers revenue and customer satisfactions while minimizing energy cost. In particular, we model this problem as a constrained discrete-time optimal control problem and use Model Predictive Control (MPC) to find its solution. Simulation studies using real cloud workloads indicate that under dynamic workload conditions, our proposed solution achieves higher net income than static allocation strategies and minimizes the average request waiting time.

/pdf/dynamic-resource-allocation-for-spot-markets-in-cloud-2vos8611lh.pdf

Dynamic Resource Allocation for Spot Markets in Cloud Computing Environments

Recently introduced spot instances in the Amazon Elastic Compute Cloud (EC2) offer low resource costs in exchange for reduced reliability; these instances can be revoked abruptly due to price and demand fluctuations. Mechanisms and tools that deal with the cost-reliability tradeoffs under this schema are of great value for users seeking to lessen their costs while maintaining high reliability. We study how mechanisms, namely, checkpointing and migration, can be used to minimize the cost and volatility of resource provisioning. Based on the real price history of EC2 spot instances, we compare several adaptive checkpointing schemes in terms of monetary costs and improvement of job completion times. We evaluate schemes that apply predictive methods for spot prices. Furthermore, we also study how work migration can improve task completion in the midst of failures while maintaining low monetary costs. Trace-based simulations show that our schemes can reduce significantly both monetary costs and task completion times of computation on spot instance.

Monetary Cost-Aware Checkpointing and Migration on Amazon Cloud Spot Instances

Amazon introduced Spot Instance Market to utilize the idle resources of Amazon Elastic Compute Cloud (EC2) more efficiently. The price of a spot instance changes dynamically according to the current supply and demand for cloud resources. Users can bid for a spot instance and the job request will be granted if the current spot price falls below the bid, whereas the job will be terminated if the spot price exceeds the bid. In this paper, we investigate the problem of designing a bidding strategy from a cloud service broker's perspective, where the cloud service broker accepts job requests from cloud users, and leverages the opportunistic yet less expensive spot instances for computation in order to maximize its own profit. In this context, we propose a profit aware dynamic bidding (PADB) algorithm, which observes the current spot price and selects the bid adaptively to maximize the time average profit of the cloud service broker. We show that our bidding strategy achieves a near-optimal solution, i.e., (1−∈) of the optimal solution to the profit maximization problem, where ∈ can be arbitrarily small. The proposed dynamic bidding algorithm is self-adaptive and requires no a priori statistical knowledge on the distribution of random job sizes from cloud users.

Optimal bidding in spot instance market

Modern frameworks, such as Hadoop, combined with abundance of computing resources from the cloud, offer a significant opportunity to address long standing challenges in distributed processing. Infrastructure-as-a-Service clouds reduce the investment cost of renting a large data center while distributed processing frameworks are capable of efficiently harvesting the rented physical resources. Yet, the performance users get out of these resources varies greatly because the cloud hardware is shared by all users. The value for money cloud consumers achieve renders resource sharing policies a key player in both cloud performance and user satisfaction. In this paper, we employ microeconomics to direct the allotment of cloud resources for consumption in highly scalable master-worker virtual infrastructures. Our approach is developed on two premises: the cloud-consumer always has a budget and cloud physical resources are limited. Using our approach, the cloud administration is able to maximize per-user financial profit. We show that there is an equilibrium point at which our method achieves resource sharing proportional to each user's budget. Ultimately, this approach allows us to answer the question of how many resources a consumer should request from the seemingly endless pool provided by the cloud.

/pdf/flexible-use-of-cloud-resources-through-profit-maximization-3rff99nbq3.pdf

Flexible use of cloud resources through profit maximization and price discrimination

Spot virtual-machine (VM) instances, such as Amazon EC2 Spot VMs, are a class of VMs that are purchased through a market mechanism of price-bids submitted by cloud users. Spot VMs can be obtained at substantially lower cost than other VM classes such as Reserved and On-demand instances, but they do not have guaranteed availability since it depends on the submitted price bids and the fluctuating spot VM price. Many applications with large computing requirements but no real-time availability constraints, such as scientific computing, financial modelling and large data analysis, can be carried out at a significantly lower cost using spot VMs. For such jobs, an important question that arises is what should the submitted price bids be so that the computation is completed within a fixed time interval while the cost is minimized. Towards this goal, we model a job as a fixed computation request with a deadline constraint and formulate the problem of designing a dynamic bidding policy that minimizes the average cost of job completion. We obtain analytical and closed-form results for the optimal strategy under a Markov spot price evolution, and then evaluate the performance of the algorithms on the actual spot price history of Amazon EC2 Spot VMs.

Optimal Bids for Spot VMs in a Cloud for Deadline Constrained Jobs

We present a practical, market-based solution to the resource provisioning problem in a set of heterogeneous resource clusters. We focus on provisioning rather than immediate scheduling decisions to allow users to change long-term job specifications based on market feedback. Users enter bids to purchase quotas, or bundles of resources for long-term use. These requests are mapped into a simulated clock auction which determines uniform, fair resource prices that balance supply and demand. The reserve prices for resources sold by the operator in this auction are set based on current utilization, thus guiding the users as they set their bids towards under-utilized resources. By running these auctions at regular time intervals, prices fluctuate like those in a real-world economy and provide motivation for users to engineer systems that can best take advantage of available resources. These ideas were implemented in an experimental resource market at Google. Our preliminary results demonstrate an efficient transition of users from more congested resource pools to less congested resources. The disparate engineering costs for users to reconfigure their jobs to run on less expensive resource pools was evidenced by the large price premiums some users were willing to pay for more expensive resources. The final resource allocations illustrated how this framework can lead to significant, beneficial changes in user behavior, reducing the excessive shortages and surpluses of more traditional allocation methods.

/pdf/using-a-market-economy-to-provision-compute-resources-across-55ie6tidkp.pdf

Using a market economy to provision compute resources across planet-wide clusters

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Jim Winget

Author Tools

Chat about Author

Papers

Using a market economy to provision compute resources across planet-wide clusters