The work presented in this paper investigates the ability of BERT neural language model pretrained in Italian to embed syntactic dependency relationships into its layers, by approximating a Dependency Parse Tree. To this end, a structural probe, namely a supervised model able to extract linguistic structures from a language model, has been trained leveraging the contextual embeddings from the layers of BERT. An experimental assessment has been performed using an Italian version of BERT-base model and a set of datasets for Italian labelled with Universal Dependencies formalism. The results, achieved using standard metrics of dependency parsers, have shown that a knowledge of the Italian syntax is embedded in central-upper layers of the BERT model, according to what observed in literature for the English case. In addition, the probe has been also used to experimentally evaluate the BERT model behaviour in case of two specific syntactic phenomena in Italian, namely null-subject and subject-verb-agreement, showing better performance than an Italian state-of-the-art parser. These findings can open a path for the development of new hybrid approaches, exploiting the probe to integrate or improve limits or weaknesses in analysing articulated constructions of Italian syntax, traditionally complex to be parsed.

本文提出的工作通过近似依赖关系分析树，研究了在意大利语中经过预训练的BERT神经语言模型将语法依赖关系嵌入其层中的能力。为此，已经利用来自BERT层的上下文嵌入对结构探针（即能够从语言模型中提取语言结构的监督模型）进行了训练。使用意大利语版本的BERT-base模型和一组标有Universal Dependencies形式的意大利语数据集进行了实验评估。使用相关性解析器的标准度量获得的结果表明，根据对英语案例的文献观察，在BERT模型的中央-上层中嵌入了意大利语语法的知识。此外，在意大利语中存在两种特定的句法现象（即空主语和主语-动词一致）的情况下，该探针还用于实验性地评估BERT模型的行为，显示出比意大利语最新的解析器更好的性能。这些发现可以为开发新的混合方法开辟道路，利用该探针在分析传统上要解析的意大利语法的铰接式结构时综合或改善限制或弱点。