Despite a growing awareness of methodological issues, the literature on APPRAISAL has not so far provided adequate answers to some of the key challenges involved in reliably identifying and classifying evaluative language expressions. This article presents a step-wise method for the manual annotation of APPRAISAL in text that is designed to optimize reliability, replicability and transparency. The procedure consists of seven steps, from the creation of a context-specific annotation manual to the statistical analysis of the quantitative data derived from the manually-performed annotations. By presenting this method, the article pursues the twofold purpose of (i) providing a practical tool that can facilitate more reliable, replicable and transparent analyses, and (ii) fostering a discussion of the best practices that should be observed when manually annotating APPRAISAL. (Less)

中文翻译：

---

一种阶梯式的评价注解方法

尽管对方法论问题的认识不断提高，但迄今为止，关于评估的文献还没有为可靠地识别和分类评估性语言表达所涉及的一些关键挑战提供足够的答案。本文介绍了一种在文本中手动注释 APRAISAL 的逐步方法，旨在优化可靠性、可复制性和透明度。该过程包括七个步骤，从创建特定于上下文的注释手册到对从手动执行的注释中得出的定量数据进行统计分析。通过介绍这种方法，本文追求双重目的：(i) 提供一种实用工具，可以促进更可靠、可复制和透明的分析，(ii) 促进对手动注释评估时应遵守的最佳实践的讨论。（较少的）