A wide variety of deep reinforcement learning (DRL) models have recently been proposed to learn profitable investment strategies. The rules learned by these models outperform the previous strategies specially in high frequency trading environments. However, it is shown that the quality of the extracted features from a long-term sequence of raw prices of the instruments greatly affects the performance of the trading rules learned by these models. Employing a neural encoder-decoder structure to extract informative features from complex input time-series has proved very effective in other popular tasks like neural machine translation and video captioning in which the models face a similar problem. The encoder-decoder framework extracts highly informative features from a long sequence of prices along with learning how to generate outputs based on the extracted features. In this paper, a novel end-to-end model based on the neural encoder-decoder framework combined with DRL is proposed to learn single instrument trading strategies from a long sequence of raw prices of the instrument. The proposed model consists of an encoder which is a neural structure responsible for learning informative features from the input sequence, and a decoder which is a DRL model responsible for learning profitable strategies based on the features extracted by the encoder. The parameters of the encoder and the decoder structures are learned jointly, which enables the encoder to extract features fitted to the task of the decoder DRL. In addition, the effects of different structures for the encoder and various forms of the input sequences on the performance of the learned strategies are investigated. Experimental results showed that the proposed model outperforms other state-of-the-art models in highly dynamic environments.

中文翻译：

---

一种基于强化学习的编解码器框架，用于学习股票交易规则

最近提出了各种各样的深度强化学习（DRL）模型，以学习有利可图的投资策略。这些模型学到的规则特别是在高频交易环境中优于以前的策略。但是，结果表明，从工具原始价格的长期序列中提取的特征的质量极大地影响了这些模型学习的交易规则的性能。在其他流行的任务（如神经机器翻译和视频字幕）中，采用神经编码器/解码器结构从复杂的输入时间序列中提取信息特征已被证明非常有效，其中模型面临着类似的问题。编码器-解码器框架从一长串价格中提取出高度信息化的功能，同时学习如何基于提取的功能来生成输出。本文提出了一种基于神经编码器-解码器框架并结合DRL的新型端到端模型，以从长期的工具原始价格序列中学习单一工具的交易策略。所提出的模型由一个编码器和一个解码器组成，该编码器是负责从输入序列中学习信息特征的神经结构，而DRL模型则是一种基于编码器提取的特征来学习获利策略的DRL模型。共同学习编码器和解码器结构的参数，这使编码器能够提取适合于解码器DRL任务的特征。此外，研究了编码器的不同结构以及各种形式的输入序列对学习策略性能的影响。实验结果表明，在高度动态的环境中，所提出的模型优于其他最新模型。