We describe PNUTS, a massively parallel and geographically distributed database system for Yahoo!'s web applications. PNUTS provides data storage organized as hashed or ordered tables, low latency for large numbers of concurrent requests including updates and queries, and novel per-record consistency guarantees. It is a hosted, centrally managed, and geographically distributed service, and utilizes automated load-balancing and failover to reduce operational complexity. The first version of the system is currently serving in production. We describe the motivation for PNUTS and the design and implementation of its table storage and replication layers, and then present experimental results.

/pdf/pnuts-yahoo-s-hosted-data-serving-platform-tirb704wyn.pdf

PNUTS: Yahoo!'s hosted data serving platform

Database systems serving cloud platforms must serve large numbers of applications (or tenants). In addition to managing tenants with small data footprints, different schemas, and variable load patterns, such multitenant data platforms must minimize their operating costs by efficient resource sharing. When deployed over a pay-per-use infrastructure, elastic scaling and load balancing, enabled by low cost live migration of tenant databases, is critical to tolerate load variations while minimizing operating cost. However, existing databases---relational databases and Key-Value stores alike---lack low cost live migration techniques, thus resulting in heavy performance impact during elastic scaling. We present Albatross, a technique for live migration in a multitenant database serving OLTP style workloads where the persistent database image is stored in a network attached storage. Albatross migrates the database cache and the state of active transactions to ensure minimal impact on transaction execution while allowing transactions active during migration to continue execution. It also guarantees serializability while ensuring correctness during failures. Our evaluation using two OLTP benchmarks shows that Albatross can migrate a live tenant database with no aborted transactions, negligible impact on transaction latency and throughput both during and after migration, and an unavailability window as low as 300 ms.

/pdf/albatross-lightweight-elasticity-in-shared-storage-databases-1ddsqtymol.pdf

Albatross: lightweight elasticity in shared storage databases for the cloud using live data migration

Computer clusters with the shared-nothing architecture are the major computing platforms for big data processing and analysis. In cluster computing, data partitioning and sampling are two fundamental strategies to speed up the computation of big data and increase scalability. In this paper, we present a comprehensive survey of the methods and techniques of data partitioning and sampling with respect to big data processing and analysis. We start with an overview of the mainstream big data frameworks on Hadoop clusters. The basic methods of data partitioning are then discussed including three classical horizontal partitioning schemes: range, hash, and random partitioning. Data partitioning on Hadoop clusters is also discussed with a summary of new strategies for big data partitioning, including the new Random Sample Partition (RSP) distributed model. The classical methods of data sampling are then investigated, including simple random sampling, stratified sampling, and reservoir sampling. Two common methods of big data sampling on computing clusters are also discussed: record-level sampling and block-level sampling. Record-level sampling is not as efficient as block-level sampling on big distributed data. On the other hand, block-level sampling on data blocks generated with the classical data partitioning methods does not necessarily produce good representative samples for approximate computing of big data. In this survey, we also summarize the prevailing strategies and related work on sampling-based approximation on Hadoop clusters. We believe that data partitioning and sampling should be considered together to build approximate cluster computing frameworks that are reliable in both the computational and statistical respects.

/pdf/a-survey-of-data-partitioning-and-sampling-methods-to-5anrhdwver.pdf

A survey of data partitioning and sampling methods to support big data analysis

A mobile terminal including a display configured to display information; a short range communication module configured to exchange a signal with an external control device; and a controller configured to receive the signal from the external control device, determine a context at a timing point of receiving the signal, and control an operation corresponding to the signal to be performed in the determined context. Further, the operation includes at least one of activation/deactivation of the display, an activation/deactivation of a lock mode of the mobile terminal, a volume operation and a camera operation.

Mobile terminal and controlling method thereof

Techniques for managing information in a computing environment. Information associated with components of the computing environment is obtained. Then, from at least a portion of the obtained information, a determination is made as to the existence of one or more relationships associated with at least a portion of the components of the computing environment. The determination of the existence of one or more relationships is capable of accounting for a full lifecycle (e.g., including deployment, installation and runtime) associated with at least one component of the computing environment. Thus, techniques for managing runtime dependencies between the various components of computing systems are disclosed which provide a level of abstraction from individual systems and allow the computation of service/component (wherein the component may, for example, be an application, middleware, hardware, a device driver, an operating system and a system associated with the computing environment) dependencies that are related to end-to-end services, as perceived by a customer. By way of example, the inventive techniques may be applied to a distributed computing environment. The computing environment may also be an autonomic computing environment.

Methods and apparatus for managing dependencies in distributed systems

Cloud SQL Server is a relational database system designed to scale-out to cloud computing workloads. It uses Microsoft SQL Server as its core. To scale out, it uses a partitioned database on a shared-nothing system architecture. Transactions are constrained to execute on one partition, to avoid the need for two-phase commit. The database is replicated for high availability using a custom primary-copy replication scheme. It currently serves as the storage engine for Microsoft's Exchange Hosted Archive and SQL Azure.

Adapting microsoft SQL server for cloud computing

Described herein is a data collection and warehousing system that collects log files from a plurality of servers, and that transforms the entries of the log files prior to using them to populate a database. Nightly, the system provides a pre-processing program to each of the servers. The servers executing the pre-processing component to filter the logged data before sending it on to the data collection and warehousing system. The data collection and warehousing system pre-parses each received log file to create dimension and fact tables. During this process, strings are replaced by keys. These keys are calculated on-the-fly by taking a hash of each string. The resulting tables are then further parsed to create further dimension and fact tables that correspond to each of the dimension and fact tables of a relational database schema. After further processing, these tables are loaded into a data warehousing system.

Populating a data warehouse using a pipeline approach

An approach is described for maintaining multiple presence objects associated with a single entity. The different presence objects correspond to different respective aspects of the entity's presence. Each presence object includes subscriber information and permission information associated therewith. The subscriber information identifies the subscribers that subscribe to receive notifications for a particular presence object. The permission information identifies the users who are entitled to receive notifications for the particular presence object. By virtue of the use of multiple objects, the entity can control access to its presence information with a high degree of granularity.

Providing Access to Presence Information Using Multiple Presence Objects

In one embodiment, a user authentication server may use geo-location tracking to determine whether to present an enhanced identity challenge. A communication interface 180 may receive a user login attempt by a user and a current location of the user login attempt. A data storage 150 may store a user location profile of the user. A processor 120 may execute a comparison of the current location to the user location profile. The communication interface 180 may present the user with an enhanced identity challenge before allowing user access based on the comparison.

Protection from unfamiliar login locations

This paper discusses technical problems that arise in supporting large-scale 24×7 web services basedon experience at MSN with Windows Live TM services. Issues covered include multi-tier architecture,costs of commodity vs. premium servers, managing replicas, managing sessions, use of materializedviews, and controlling checkpointing. We ﬁnish with some po ssible research directions. 1 Introduction Large-scale web services supported by fee-for-service or advertising revenue are heavy users of database tech-nology. Such services include email, instant messaging, social networking, blogs, photo sharing, maps, shop-ping, and classiﬁeds. They also include search services, th ough these typically use custom indexing engines, notrelational database systems.By large-scale, we mean systems with millions of users and thousands of servers. For example, WindowsLive Messenger has thousands of servers and tens of millions of simultaneous online connections. It processeshundreds of millions of logins per day and has over 250 million unique users per month. Each login translatesinto queries to backend database servers for authentication and authorization. Users send billions of messagesper day. The message transfer itself takes place independently of the database. Similarly large numbers havebeen reported for Hotmail

Data Management Issues in Supporting Large-Scale Web Services

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Ramesh Manne

Author Tools

Chat about Author