Text representation can aid machines in text understanding. Previous work on text representation often focuses on the so-called forward implication, i.e., preceding words are taken as the context of later words for creating representations, effective it is, yet ignoring the fact that the semantics of a text segment is a product of the mutual implication of words in the text: later words contribute to the meaning of preceding words. To bridge this gap, we introduce the concept of interaction and propose a two-perspective interaction representation, in which it encapsulates a local and a global interaction representation. Here, a local interaction representation is one that interacts among words with parent-children relationships on the syntactic trees whereas a global interaction interpretation is one that interacts among all the words in a sentence. We combine these two interaction representations to develop a Hybrid Interaction Representation (HIR).

Inspired by existing feature-based and fine-tuning-based pretrain-finetuning approaches to language models, we integrate the merits of feature-based and fine-tuning-based methods to propose the Pre-train, Interact, Fine-tune (PIF) architecture.

We evaluate our proposed models on five widely-used datasets for text classification tasks. It turns out that our ensemble method, HIR_P, outperforms state-of-the-art baselines with improvements ranging from 2.03% to 3.15% in terms of error rate. In addition, we find that, the improvements of PIF against most state-of-the-art methods is not affected by increasing of the text length.

文本表示可以帮助机器理解文本。先前关于文本表示的工作通常集中于所谓的前向蕴涵，即，将前面的单词作为后面的单词的上下文，以创建表示，这是有效的，但忽略了文本段的语义是以下内容的产物：文本中单词的相互暗示：后几个单词有助于前一个单词的含义。为了弥合这种差距，我们介绍了交互的概念，并提出了两种视角的交互表示，其中封装了局部和全局交互表示。在这里，局部交互表示是指在句法树上具有亲子关系的单词之间进行交互，而全局交互表示则是交互解释是一种在句子中所有单词之间进行交互的解释。我们结合这两个交互表示形式来开发混合交互表示形式（HIR）。

受现有的基于特征和基于微调的预训练-微调方法对语言模型的启发，我们结合了基于特征和基于微调的方法的优点，提出了预训练，交互，微调（PIF）建筑。

我们在五个广泛用于文本分类任务的数据集上评估了我们提出的模型。事实证明，我们的集成方法HIR _P优于最新的基准，错误率从2.03％到3.15％有所提高。此外，我们发现，与大多数最新技术相比，PIF的改进不受文本长度增加的影响。