Summarizing user reviews and classifying user sentiment are two critical tasks for modern e-commerce platforms. These two tasks can benefit each other by capturing the shared linguistic features. However, such a relationship has not been fully exploited by existing research on domain-specific contextual representations. This work explores a win-win strategy for a multi-task framework with three stages: general pre-training, adaptive pre-training, and collaborative fine-tuning. The task-adaptive continual pre-training on a language model can obtain domain-specific contextual representations, further used to improve two related tasks, sentiment classification and review summarization during the collaborative fine-tuning. Meanwhile, to effectively capture sentiment-oriented domain-specific contextual representations, we introduce a novel task-adaptive pre-training procedure, which adds a sentiment prediction task during the adaptive pre-training. Extensive experiments conducted on two adaption scenarios of a general-to-single domain and a general-to-multiple domain show that our framework outperforms state-of-the-art methods.

中文翻译：

---

自适应预训练和协作微调：改进评论分析任务的双赢策略


总结用户评论和分类用户情绪是现代电子商务平台的两项关键任务。这两项任务可以通过捕获共享的语言特征来互惠互利。然而，现有的针对特定领域上下文表示的研究尚未充分利用这种关系。这项工作探索了多任务框架的双赢策略，分为三个阶段：通用预训练、自适应预训练和协作微调。对语言模型进行任务自适应持续预训练可以获得特定领域的上下文表示，并在协作微调期间进一步用于改进两个相关任务：情感分类和评论摘要。同时，为了有效捕获面向情感的特定领域上下文表示，我们引入了一种新颖的任务自适应预训练过程，该过程在自适应预训练期间添加了情感预测任务。对通用到单域和通用到多域的两种适应场景进行的广泛实验表明，我们的框架优于最先进的方法。