In the Dublin Institute of Technology, high threshold core skills assessments are run in mathematics for third year engineering students. Such tests require students to reach a threshold of 90% on a multiple-choice test based on a randomised question bank. The material covered by the test consists of the more important aspects of undergraduate engineering mathematics covered in the first two years of the Honours degree programme and/or the three years of the Ordinary degree programme . Students are allowed to re-sit the assessment as frequently as required until they pass. In order to measure the effectiveness of such an exercise a follow up assessment was given to students on their first day of fourth year. A comparison is made with the level of basic mathematical knowledge of these students on their first day in Third year, exactly a year previously. In addition students were surveyed on their view of, how much knowledge had been retained and how effective they felt that this approach had been. Introduction Dublin Institute of Technology offers students several different routes into engineering, via direct entry onto an Honours degree or alternatively entry into an ordinary degree followed by subsequent transfer to third year of the honours degree. However, these students have a wide range of mathematical abilities and prior knowledge, and many are missing the basic skills required for completion of a mathematics module at this level. In an attempt to resolve this problem, an Advanced Maths Diagnostic Test (Carr et al 2013,2013b) was introduced. This test covered many of the key concepts from the early years of Engineering Mathematics. A pass-mark of 90% was set in this assessment. Multiple re-sit opportunities were provided, and a weighting of 10% of the continuous assessment mark for the mathematics module was given to the diagnostic test. Online resources and special classes were also introduced. This procedure has been run for the last few years and normally 80-90% of the students achieve a mark of over 90% in these basic skills. In an attempt to assess the level of retention of these basic skills the Advanced Diagnostic tests is also given to the same students on the very first day of 4th year maths(3rd week of September). These students wouldn’t have studied any mathematics since the previous January, a break of 9 months although in theory they should have been using these skills in their other engineering modules. For most students this would be the longest break they would ever have had from studying mathematics. Methodology: The methodology for this study involved comparing the diagnostic test scores of a group of students when they are in 3rd year and again when they are in 4th year. A questionnaire was also given to the students once they have completed the 4th year diagnostic test. 83 students took the diagnostic test in 3rd year, 66 in 4th year and 41 students completed the questionnaire. The questionnaire was designed by the authors and was made up of seven questions with the aim of investigating student’s opinions regarding the process of continuous diagnostic testing. The diagnostic test is made up of 20 questions based on various mathematical topics. These topics include integration (6 questions), differentiation (4 questions), 1st Order Differential Equations (2questions), 2nd Order Differential Equations (2 questions), complex numbers (2 questions) and matrices (2 questions). The test is presented to students in their first mathematics lectures of 3rd and 4th year. Students are awarded 3 mark for a correct answer and -1 marks for an incorrect answer. No attempt marks are awarded. Diagnostic Test Quantitative Results Descriptive analysis of the results show that students performed better on the diagnostic test in their first lecture of 3rd year (Mean: 11.72, SD: 3.44) than they did in their first lecture of 4th year (Mean: 9.14, SD: 3.24). A paired samples T-test found that there was a statistically significant decrease in the mean scores of the students (t (22) = 3.812, p < .001) between their 3rd and 4th year. The following table and figures show a breakdown of the student’s performance in each question over both years. In comparison to their 3rd year results, 4th year students got lower scores in thirteen of the twenty questions. The main differences in scores between each year were in the in the 2nd ODE, matrices and complex number questions. Table 1: Breakdown of diagnostic results by question and year Question Question Type: 3rd Year % Mean 4th Year % Mean Diff (+/-) 1 Differentiation 100 100 No Diff 2 Differentiation 100 100 No Diff 3 2nd Order Differential Equations 27 8 -19 4 2nd Order Differential Equations 25 0 -25 5 Integration 93 94 +1 6 Integration 92 88 -4 7 Matrices 59 14 -45 8 Matrices 77 8 -69 9 1st Order Differential Equations 34 44 +10 10 1st Order Differential Equations 34 44 +10 11 Chain Rule 65 48 -17 12 Chain Rule 65 48 -17 13 Product Rule 72 48 -24 14 Product Rule 47 29 -18 15 Quotient Rule 34 61 +27 16 Quotient Rule 69 61 -8 17 Complex Numbers 51 14 -37 18 Complex Numbers 51 9 -42 19 Integration 63 48 -15 20 Integration 17 48 +31 These results are particularly interesting when they are compared with student’s responses to the questionnaire on the diagnostic testing process. Although the results from both tests show a fall off in performance from year 3 to year 4, the vast majority of students (71.5%) felt it will be somewhat easier or easier to restore knowledge as a result of engagement in the continuous diagnostic testing process. Figure 1: Students responses on whether it will be easier to restore knowledge from diagnostic testing A closer look at each specific topic show further discrepancies between the findings. Students were asked to reflect on the diagnostic test they took in 3rd and 4th year and to indicate their level of retention in each of the topics. The two topics which students indicated the highest levels of retention were complex numbers (69.1% somewhat / very high) and matrices (66.7% somewhat / very high). However, these were the topics in which there was the biggest drop in student performance between the 3rd and 4th year tests. Between the four questions on these two topics these was an average drop of 48 percentage points from the 3rd and 4th year tests. The majority of students (47.6%) were undecided on their level of retention for the Differential Equation questions. The comparisons of test results for these questions show two different cases. Students did show good levels of retention for the 1st Order Differential Equations but poor retention for the 2nd Order Differential Equations. Overall the results of the questionnaire indicate that students have a positive opinion of the diagnostic testing process. As Figure 2 depicts the majority of students feel that the continuous practising of skills through the diagnostic testing process has helped their performance in end of term examinations. Figure2: Students responses on whether diagnostic testing has helped exam performance Furthermore, 66.7% of students would like the diagnostic tests to be carried our more frequently. Although the pass mark for the diagnostic test is very high (90%), 47.6% of students feel that this mark is just right. Unsurprising the majority (50%) feel that it is too high. Qualitative Synopsis of Students’ Responses 1. Reflecting on the diagnostic test you took at the start of this academic year, what was your level of retention in each topic since the last diagnostic test you took in 3rd year. Explain. When asked to expand on question 1 relating to students’ perceptions of their retention levels of topics on the diagnostic test taken at the start of the academic year only 25% of respondents mentioned specific topics of difficulty the majority of these highlighting ODE’s as being an issue. Almost 44% of students who expanded on this question referred to “poor retention levels” due to “forgetting” material or “needing revision” but did not mention specific topics. The final significant theme which emerged from students who chose to expand on this question related to a positive attitude towards the diagnostic test as 12% of students felt it encouraged them to revise/improve their skills. 2. In areas where you had low levels of retention do you think it would be easier to restore your knowledge in the future as a result of your engagement in continuous diagnostic testing. Explain. Almost 60% of students who responded to this question highlighted the ability of the diagnostic test to aid memory and retention of topics which is a very positive finding one student describing it as something which helps you to “reinforce existing knowledge” (Male, Level 7). [Fewer students (36%) focussed on the ability of the diagnostic test to aid real understanding of the mathematical topics contained on the test by encouraging real engagement and study of material after the test was completed]. 3. Are you addressing the areas where you feel you had low levels of retention? If yes how?[ i think this question may have been ambiguous some of them are talking about 3rd year and some are talking about 4th year] Students who stated that they were addressing areas of weakness in the diagnostic test almost exclusively said they were doing so by engaging in self-study of old material and said that the diagnostic test encouraged them to study while giving them a focus and incentive for their study schedules. If not, why not? There were two major themes emerging from students’ responses who stated that they were not addressing areas of weakness in the test –53% of respondents maintained that they there was “not enough time in the day” (Female, Level 8) to address the areas of weakness highlighted by the test. The second theme centred around pressure which students felt due to the work load they have in other areas of their degree programmes : “I have to focus on other areas of study” (Male,Level 8)

中文翻译：

---

经过一系列高门槛在线数学测试后，工程专业学生如何保持核心数学知识？

在都柏林理工学院，高年级核心技能评估针对数学专业的三年级学生进行。此类测试要求学生在基于随机问题库的多项选择测试中达到90％的阈值。测试涵盖的材料包括荣誉学位课程的前两年和/或普通学位课程的三年内涵盖的本科工程数学的更重要方面。允许学生根据需要重新安排评估直到他们通过。为了衡量这种练习的有效性，在第四年的第一天对学生进行了跟踪评估。比较这些学生在三年级的第一天的基本数学知识水平，恰好一年前。此外，还对学生的观点进行了调查，了解他们保留了多少知识，并认为这种方法有效。简介都柏林理工学院为学生提供了几种不同的工程学途径，可以直接进入荣誉学位课程，也可以进入普通学位课程，然后再转入荣誉学位课程的三年级。但是，这些学生具有广泛的数学能力和先验知识，许多人缺少完成此级别的数学模块所需的基本技能。为了解决此问题，引入了高级数学诊断测试（Carr等人，2013,2013b）。该测试涵盖了工程数学早期的许多关键概念。此评估中的合格分数为90％。提供了多次重新安置的机会，并将数学模块连续评估分数的10％权重分配给了诊断测试。还介绍了在线资源和特殊课程。该程序已运行了最近几年，通常80-90％的学生在这些基本技能上的得分超过90％。为了评估这些基本技能的保留水平，还在四年级数学的第一天（9月的第3周）对同一名学生进行了高级诊断测试。自从上一月份以来，这些学生将没有学习任何数学，这是9个月的休息时间，尽管从理论上讲，他们应该在其他工程模块中使用这些技能。对于大多数学生来说，这将是他们学习数学以来最长的一次休息。方法：本研究的方法涉及比较一组学生在三年级时和四年级时的诊断测试成绩。完成四年级诊断测试的学生也将收到一份问卷。第三年有83名学生参加了诊断测试，第四年有66名学生参加了调查问卷，有41名学生完成了问卷。问卷是由作者设计的，由七个问题组成，目的是调查学生对连续诊断测试过程的看法。诊断测试由基于各种数学主题的20个问题组成。这些主题包括整合（6个问题），差异化（4个问题），一阶微分方程（2个问题），二阶微分方程（2个问题），复数（2个问题）和矩阵（2个问题）。该考试将在三年级和四年级的第一场数学讲座中介绍给学生。正确答案的学生将获得3分，错误答案的学生将获得-1分。不授予任何尝试分数。诊断测试定量结果对结果的描述性分析表明，与四年级第一堂课（平均值：9.14，SD）相比，学生在三年级第一堂课的诊断测试中表现更好（平均：11.72，标准差：3.44）。 ：3.24）。配对样本T检验发现，在第三年和第四年之间，学生的平均分数有统计学显着性下降（t（22）= 3.812，p <.001）。下表和图表显示了两年中每个问题的学生表现明细。与三年级的成绩相比，四年级的学生在二十个问题中有十三个得分较低。每年之间分数的主要差异在于第二个ODE，矩阵和复数问题。表1：按问题和年份划分的诊断结果分类问题问题类型：尽管两种测试的结果都显示从3年级到4年级的成绩有所下降，但绝大多数学生（71.5％）认为，由于参与了连续诊断测试过程，恢复知识的难度会有所提高。图1：学生对从诊断测试中恢复知识是否更容易的回答仔细研究每个特定主题，可以发现发现之间的进一步差异。要求学生对他们在第3年和第4年进行的诊断测试进行反思，并指出他们对每个主题的保留程度。学生表示保留率最高的两个主题是复数（69.1％或非常高）和矩阵（66.7％某些/非常高）。然而，这些是3年级和4年级学生考试成绩下降幅度最大的主题。在这两个主题的四个问题之间，第三年和第四年的考试平均下降了48个百分点。大多数学生（47.6％）对微分方程问题的保留程度不确定。这些问题的测试结果比较显示了两种不同的情况。学生确实表现出对一阶微分方程的良好保留，但对二阶微分方程的保留却很差。问卷调查的总体结果表明，学生对诊断测试过程持积极态度。如图2所示，大多数学生认为通过诊断测试过程不断进行技能练习有助于他们在期末考试中的表现。图2：学生对诊断测试是否有助于考试成绩的回应此外，有66.7％的学生希望诊断测试更频繁地进行。尽管诊断测试的及格分数很高（90％），但47.6％的学生认为该分数是正确的。毫不奇怪，大多数（50％）认为它太高了。学生反应的定性提要1.考虑到您在本学年开始时进行的诊断测试，自第三年进行上次诊断测试以来，您对每个主题的保留程度是多少。解释。当被问及关于问题1的问题时，只有25％的受访者提到了特定的困难主题，其中大多数强调ODE是一个问题， 。在这个问题上扩展的学生中，有近44％的学生由于“忘记了”材料或“需要修订”而提到了“保留率低”，但没有提及具体主题。最终的重要主题来自选择扩展此问题的学生，这与对诊断测试的积极态度有关，因为12％的学生认为这鼓励他们修改/提高自己的技能。2。您认为在保留水平较低的地区，由于您参与了连续的诊断测试，因此将来恢复知识更为容易。解释。回答此问题的学生中，近60％强调了诊断测试有助于记忆和保留主题的能力，这是一个非常积极的发现，一位学生将其描述为有助于您“加强现有知识”的事物（男，7级， ）。[更少的学生（36％）专注于诊断测试的能力，可通过鼓励测试完成后真正的参与和学习材料来帮助真正理解测试中包含的数学主题]。3。您是在解决您认为保留率低的领域吗？如果是，怎么办？[我认为这个问题可能是模棱两可的，其中有些人谈论三年级，有些人谈论四年级。]那些表示正在解决诊断测试弱点的学生几乎完全是说他们正在这样做。通过对旧材料进行自学，他说诊断测试鼓励他们学习，同时给予他们学习计划的重点和动力。如果没有，为什么不呢？学生的回答中出现了两个主要主题，它们表示他们没有解决测试中的弱点领域– 53％的受访者坚持认为“一天中没有足够的时间”（女性，8级）解决测试突出显示的弱点区域。