Real-time accurate recommendation of large-scale recommender systems is a challenging task. Matrix factorization (MF), as one of the most accurate and scalable techniques to predict missing ratings, has become popular in the collaborative filtering (CF) community. Currently, stochastic gradient descent (SGD) is one of the most famous approaches for MF. However, it is non-trivial to parallelize SGD for large-scale CF MF problems due to the dependence on the user and item pair, which can cause parallelization over-writing. To remove the dependence on the user and item pair, we propose a multi-stream SGD (MSGD) approach, for which the update process is theoretically convergent. On that basis, we propose a Compute Unified Device Architecture (CUDA) parallelization MSGD (CUMSGD) approach. CUMSGD can obtain high parallelism and scalability on ${Graphic \;Processing\; Units}$ (GPUs). On Tesla K20m and K40c GPUs, the experimental results show that CUMSGD outperforms prior works that accelerated MF on shared memory systems, e.g., DSGD, FPSGD, Hogwild!, and CCD++. For large-scale CF problems, we propose multiple GPUs (multi-GPU) CUMSGD (MCUMSGD). The experimental results show that MCUMSGD can improve MSGD performance further. With a K20m GPU card, CUMSGD can be 5-10 times as fast compared with the state-of-the-art approaches on shared memory platform.

MSGD: A Novel Matrix Factorization Approach for Large-Scale Collaborative Filtering Recommender Systems on GPUs

In the past decade, Graphics Processing Units have played an important role in the field of high-performance computing and they still advance new fields such as IoT, autonomous vehicles, and exascale computing. It is therefore important to understand how to extract performance from these processors, something that is not trivial. This survey discusses various optimization techniques found in 450 articles published in the last 14 years. We analyze the optimizations from different perspectives which shows that the various optimizations are highly interrelated, explaining the need for techniques such as auto-tuning.

Optimization Techniques for GPU Programming

Big Data and Recommender Systems

Matrix Factorization (MF) has been widely applied in machine learning and data mining. Due to the large computational cost of MF, we aim to improve the efficiency of SGD-based MF computation by utilizing the massive parallel processing power of heterogeneous multiprocessors. The main challenge in parallel SGD algorithms on heterogeneous CPU-GPU systems lies in the strategy to assign tasks. We design a novel strategy to divide the matrix into a set of blocks by considering two aspects. First, we observe that the matrix should be divided nonuniformly, and relatively large blocks should be assigned to GPUs to saturate the computing power of GPUs. In addition to exploiting the characteristics of hardware, the workloads assigned to two types of hardware should be balanced. We design a cost model tailored for our problem to accurately estimate the performance of hardware on different data sizes. Extensive experiments show that our proposed algorithm achieves high efficiency with a high quality of training quality.

Efficient Matrix Factorization on Heterogeneous CPU-GPU Systems

The increasing amount of network data and the development of big data technology have brought development opportunities and challenges to recommendation systems. Model-based collaborative filtering has become a mainstream algorithm in recommendation systems. For example, the Funk singular- value decomposition (FunkSVD) algorithm. However, in big data calculations, data sparseness and iterative oscillations often affect the accuracy of the FunkSVD algorithm. Moreover, when the data volume is in units of GB or more, the FunkSVD algorithm runs slowly and is ineffective. Therefore, we propose an improved FunkSVD algorithm (IFABG) based on the root-mean-square prop (RMSProp) and graphics processing unit (GPU) to solve this problem. First, we used RMSProp algorithm to improve the traditional FunkSVD algorithm, alleviate data sparseness and iterative shock, and improve the prediction accuracy of the algorithm. Next, we implemented parallelization of the improved FunkSVD algorithm in the GPU, which increased the calculation speed of the algorithm. Finally, we verify the IFABG algorithm using the Movielens dataset. The experimental results show that the IFABG algorithm is very suitable for processing sparse data, and it alleviates the iterative shock. The prediction accuracy rate is approximately 30% higher than that of the traditional FunkSVD algorithm. The experimental results also demonstrate that the IFABG algorithm has a good acceleration effect. For the same size dataset, the IFABG algorithm is faster than the traditional FunkSVD, and the acceleration ratio can be as high as 19.27. 

Parallel Algorithm of Improved FunkSVD Based on GPU

Matrix factorization is one of the leading techniques for many applications such as social network-based recommendation systems. As of today, many parallel stochastic gradient descent SGD methods have been proposed to address the matrix factorization issue on shared-memory multi-core systems and distributed systems. However, these methods cannot be improved significantly on graphics processing unit GPU because the serious over-writing problem and thread divergence may occur. The fundamental reason for such undesired results is that GPU is a parallel single instruction multiple data device, which only can greatly improve the applications with fine-grained parallelism. In this paper, we propose an efficient GPU algorithm, named GPUSGD, to solve the matrix factorization problem based on SGD method. The major advantage of the proposed GPUSGD is that such method not only can handle the over-writing problem but also can avoid the performance loss caused by the thread divergence. The experimental results show that GPUSGD performs much better in accelerating the matrix factorization compared with the existing state-of-the-art parallel methods. To the best of our knowledge, this is the first work that develops a parallel SGD method to improve the matrix factorization on GPU. Copyright © 2015 John Wiley & Sons, Ltd.

GPUSGD: A GPU-accelerated stochastic gradient descent algorithm for matrix factorization

With the development of information technology, graph computing becomes an increasingly important tool for information processing. Recently, GPU has been adopted to accelerate various graph processing algorithms. However, since the architecture of GPU is very different from traditional computing model, the learning threshold for developing GPU-based applications is high. In this paper, we propose a GPU-based parallel graph processing system named GPregel. GPregel harnesses a lightweight complier to hide the underlying complexity of the parallel details and simplifies programming. It greatly reduces the difficulty in utilizing the GPU to solve graph computing problems. We also design a special storage model for BSP model running on GPU, which overcomes the execution divergence and irregular memory access by coarse-grained designs. Experiments demonstrate that GPregel can achieve high performance with little work for developers.

GPregel: A GPU-Based Parallel Graph Processing Model

Large-scale graph processing plays an increasingly important role for many data-related applications. Recently GPU has been adopted to accelerate various graph processing algorithms. However, since the architecture of GPU is very different from traditional computing model, the learning threshold for developing GPU-based applications is high. In this paper, we propose a GPU-based parallel graph processing system named GPregel to tackle this challenge. GPregel is a BSP model in graph processing such as Pregel from Google. It harnesses a lightweight compiler to hide the underlying complexity of the parallel processing details and simplifies programming, so that it greatly reduces the difficulty in utilizing the GPU to solve graph computing problems. Moreover, GPregel develops several optimizations for enhancing the performance, including (1) a special storage model for BSP model running on GPU, which overcomes the execution divergence and irregular memory access by coarse-grained designs; (2) a warp-level optimal strategy Parallelized-Messages-Sending and a thread-level optimal strategy Threads-Merge-Executing to accelerate the computations of high degree vertexes and low degree vertexes respectively; (3) messages copy mechanism optimization that utilizes a shared array and a rolling array to speed up the messages copy. Experiments demonstrate that GPregel can achieve high performance with little work for developers.

A BSP model graph processing system on many cores

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Siyan Lai

Author Tools

Chat about Author

Papers

GPUSGD: A GPU-accelerated stochastic gradient descent algorithm for matrix factorization

GPregel: A GPU-Based Parallel Graph Processing Model

A BSP model graph processing system on many cores