This study introduces GAMET, which was developed to help writing researchers examine the types and percentages of structural and mechanical errors in texts. GAMET is a desktop application that expands LanguageTool v3.2 through a user-friendly, graphic user interface that affords the automatic assessment of writing samples for structural and mechanical errors. GAMET is freely available, works on a variety of operating systems, affords document batch processing, and groups errors into a number of structural and mechanical error categories. This study also tests LanguageTool’s validity using hand-coded assessment for accuracy and meaningfulness on first language (L1) and second language (L2) writing corpora. The study also examines how well LanguageTool replicates human coding of structural and mechanical errors in an L1 corpus as well as assesses associations between GAMET and human ratings of essay quality. Results indicate that LanguageTool can be used to successful locate errors within text. However, while the accuracy of LanguageTool is high, the recall of errors is low, especially in terms of punctuation errors. Nevertheless, the errors coded by LanguageTool show significant correlations with human ratings of writing and grammar and mechanics errors. Overall, the results indicate that while LanguageTool fails to flag a number of errors, the errors flagged provide an accurate profile of the structural and mechanical errors made by writers.

中文翻译：

---

使用人工判断来检查自动语法，语法和机械错误的有效性

这项研究介绍了GAMET，它旨在帮助写作研究人员检查文本中结构和机械错误的类型和百分比。GAMET是一个桌面应用程序，通过用户友好的图形用户界面扩展LanguageTool v3.2，该界面可自动评估书写样本的结构和机械错误。GAMET是免费提供的，可在各种操作系统上运行，提供文档批处理，并将错误分为许多结构和机械错误类别。这项研究还使用手动编码评估​​语言工具在第一语言（L1）和第二语言（L2）写作语料库上的准确性和意义，以此来检验LanguageTool的有效性。这项研究还研究了LanguageTool在L1语料库中如何很好地复制人类对结构和机械错误的编码，以及评估GAMET与人类论文质量评级之间的关联。结果表明LanguageTool可用于成功定位文本中的错误。但是，虽然LanguageTool的准确性很高，但错误的召回率却很低，尤其是在标点符号方面。但是，LanguageTool编码的错误与人类的写作水平，语法和力学错误显示出显着的相关性。总体而言，结果表明，虽然LanguageTool无法标记许多错误，但标记的错误提供了作者所犯的结构和机械错误的准确概况。结果表明LanguageTool可用于成功定位文本中的错误。但是，尽管LanguageTool的准确性很高，但错误的召回率却很低，尤其是在标点符号方面。但是，LanguageTool编码的错误与人类的写作水平，语法和力学错误显示出显着的相关性。总体而言，结果表明，虽然LanguageTool无法标记许多错误，但标记的错误提供了作者所犯的结构和机械错误的准确概况。结果表明LanguageTool可用于成功定位文本中的错误。但是，虽然LanguageTool的准确性很高，但错误的召回率却很低，尤其是在标点符号方面。但是，LanguageTool编码的错误与人类的写作水平，语法和力学错误显示出显着的相关性。总体而言，结果表明，虽然LanguageTool无法标记许多错误，但标记的错误提供了作者所犯的结构和机械错误的准确概况。