A Scalable Reinforcement Learning-based System Using On-Chain Data for Cryptocurrency Portfolio Management

Question

1. How can on-chain data be utilized in quantitative trading systems?

2. What are the primary units of CryptoRLPM?

3. What is the role of DFU in CryptoRLPM?

4. How are on-chain metrics selected for each crypto in the DRU?

Accepted Answer

On-chain data, which includes typical metrics such as circulating supply, exchange flows, and balance on exchanges, can be utilized in quantitative trading systems to predict crypto prices. These real-time, sequentially recorded metrics provide comprehensive records of a blockchain network's operational details and metrics. By incorporating on-chain data into quantitative trading systems, traders can gain insights into the factors influencing cryptocurrency valuations. This integration allows for more accurate price prediction and informed trading decisions. The use of on-chain data in quantitative trading systems is expected due to the significant impact of factors like hash rate on crypto prices. However, the implementation of on-chain metrics in reinforcement learning-based systems for cryptocurrency prediction and trading (PM) has not been extensively explored. The proposed CryptoRLPM system, which incorporates on-chain data for cryptocurrency PM, demonstrates positive accumulated rate of return (ARR), daily rate of return (DRR), and Sortino ratio (SR), outperforming baseline systems. This highlights the potential of on-chain data in enhancing the performance of quantitative trading systems in the cryptocurrency market.

Accepted Answer

CryptoRLPM consists of five primary units: Data Feed Unit (DFU), Data Refinement Unit (DRU), Portfolio Agent Unit (PAU), Live Trading Unit (LTU), and Agent Updating Unit (AUU). Each unit has distinct responsibilities, with DFU and DRU handling data generation, PAU training RL agents, LTU managing live trading, and AUU maintaining agents and reallocating portfolios. The architecture of CryptoRLPM is illustrated in Figure 1, showing the interrelation of these units in the overall process from information comprehension to trading order execution.

Accepted Answer

The Data Feed Unit (DFU) is the most fundamental unit of CryptoRLPM, controlling the acquisition of data for initial model training and ongoing data feed requirements during live trading and model retraining. It retrieves historical price data and on-chain metrics using Binance REST API and Santiment's SanAPI, storing them in separate SQLite databases. The DFU's system design is displayed in Figure 2. The Data Refinement Unit (DRU) fetches on-chain metrics, which provide insights into the real-time status and activities of blockchains. On-chain metrics, due to blockchain's decentralized nature, offer more accurate and transparent measurements than traditional company fundamentals. However, the integration of on-chain data into RL-based PM systems remains unexplored. Available metrics include those available under the SanAPI Basic Subscription Plan, which vary depending on different cryptocurrencies. On-chain and social metrics are often intertwined on API platforms and in practical applications, and both are considered as on-chain metrics.

Accepted Answer

In the DRU, on-chain metrics are selected by examining the linear relationship between three-period returns and the metrics for a specific crypto. Pearson's correlation coefficients are calculated between the returns and metrics, and the coefficients are divided into three groups. Metrics within each group are sorted based on their correlation coefficients, and the highest and lowest five from each group are selected. The selected metrics from all three groups are then ranked by their appearance frequency, and the top-10 metrics are used as valid features to construct the agents' environment in the PAU. This process ensures that the metrics are specific to each crypto, mitigating the issue of ineffective metrics in existing studies.

Accepted Answer

The Evolving Agent Module (EAM) in PAU generates asset-specific signal-comprised information. It utilizes a Deep Q-Network (DQN) agent to process data and produce valuable trading signals. These signals, along with new OHLCV data, are then transferred to the Strategic Agent Module (SAM) for decision-making. The EAM plays a crucial role in enhancing the portfolio's performance by providing high-quality trading signals that are essential for optimizing the portfolio's allocation of assets.

Accepted Answer

The EAM's role in CryptoRLPM is to interact with the environment, which is composed of historical OHLCV and refined on-chain metrics of the designated crypto. It uses a Deep Q-network (DQN) agent to interact with its environment and selects an action (buy, sell, or hold) for the designated crypto at every time step. These actions establish the crypto's trading signal, which is later fed into the SAM within the same CM. The EAM collaborates with the SAM to establish the weight of a specific crypto within the dedicated CM.

Accepted Answer

The Live Trading Unit (LTU) is a component of CryptoRLPM that manages live reallocation of the cryptocurrency portfolio at 10-to-30-minute intervals. It relies on specific exchange APIs for implementation and sends reallocation requests to exchanges when portfolio weights are updated and formatted. The LTU's system design is illustrated in Figure 5, and the reallocation weight is formalized as the portfolio's weight at time step EQUATION. This weight is then transformed into the required format for the designated exchange's API, such as Binance. The LTU plays a crucial role in the end-to-end system for cryptocurrency portfolio management provided by CryptoRLPM.

Accepted Answer

CryptoRLPM consistently outperforms the baselines across all three portfolios, achieving positive values in terms of ARR, DRR, and SR. Specifically, when compared to Bitcoin, CryptoRLPM achieves at least an 83.14% improvement on ARR, a 0.5603% improvement on DRR, and a 2.1767 improvement on SR. These results demonstrate the strong performance of CryptoRLPM in generating capital returns, and validate the system's potential applicability in crypto PM. Table 2 details CryptoRLPM's outperformance over the baselines in terms of ARR, DRR, and SR. It is worth noting that CryptoRLPM achieves promising SR for all portfolios, indicating its robust ability at profit-making and adaptability to the ever-changing market. The anticipation is that including additional baselines for benchmarking, such as conventional PM strategies or RL-based methods, may further enhance CryptoRLPM's performance (ARR, DRR, and SR).

A Scalable Reinforcement Learning-based System Using On-Chain Data for Cryptocurrency Portfolio Management

Chat with Paper

AI Agents for this Paper

Most frequently asked questions

1. How can on-chain data be utilized in quantitative trading systems?

2. What are the primary units of CryptoRLPM?

3. What is the role of DFU in CryptoRLPM?

4. How are on-chain metrics selected for each crypto in the DRU?

5. What is the role of Evolving Agent Module (EAM) in PAU?

6. What is the role of EAM in CryptoRLPM?

7. What is the Live Trading Unit (LTU) in CryptoRLPM?

8. How does CryptoRLPM compare to baselines in terms of ARR, DRR, and SR?

Related Papers (5)

A review paper on cryptocurrency & portfolio management

A Modelling of Novel Approach for Stock Portfolio Management Using Unsupervised Algorithm and Deep Reinforcement Technique

Deep reinforcement learning for portfolio management of markets with a dynamic number of assets

Deep Reinforcement Learning for Portfolio Management

A Two-level Reinforcement Learning Algorithm for Ambiguous Mean-variance Portfolio Selection Problem