Please find attached a word document that summarises my thoughts and a PDF of the manuscript with further comments. 

I would be very happy to talk to you about my thoughts and comments if that would be helpful. 

General comments:

This paper introduces a new technique designed to help students explicitly identify and communicate the uncertainty inherent in geological fieldwork. Though created in the context of geologic mapping, the framework could be useful in virtual geological work and in other field-based geoscience disciplines.

The framework has six levels (ranging from no evidence to certain) that are applied to four key properties of an outcrop. One of the things I particularly like about the framework is how it separates out data from model uncertainty. I agree that there is a high level distinction between what a geoscientist is immediately observing and what they interpret from their observations. Other major strengths of the paper are the grounding of the framework in research and expert practice, and the ease with which it may potentially be used with students. Helping students to identify, manage, and communicate uncertainty in any type of geoscience research or practice is immensely important, and this technique has the potential to be a major contribution to field-based teaching practice.  Lastly, the figures (especially Figure 2) are excellent additions to the paper, which is clearly written and easy to follow.

My biggest issue with the paper is shared by the Referee 1 (R1), namely that the expert development and testing of the paper is not clearly explained and the data on which the framework is built are not shared. The paper could greatly benefit from the addition of a table or figure that shows how the framework is applied, ideally using some of the expert data. This will not only help readers to better understand how the framework is put into practice, but will also help to validate the design process. At present, the design of the framework is not replicable or accessible to other researchers because the raw data (quotes, examples of maps or notes, etc.) are not shared. I also have some comments related to the introductory arguments and claims, specified below. Overall, I recommend that paper for publication with the reviewer comments sufficiently addressed.

Specific comments:

Having read R1’s comments, I would agree that the claim about student not being explicitly taught uncertainly is overstated. In a follow-up to the 2009 study referenced in the paper, I was surprised by the range of tactics used by both experts and novices to depict uncertainty during geologic mapping – for example dotted, dashed, and solid lines; heavy and light shading with colored pencils; marking of outcrops on the map; use of question marks and other symbols; and markings in field notes. Perhaps the issue isn’t that students are never taught how to manage uncertainty, it is instead a lack of a systematic approach for teaching this skill. Anecdotally, the field courses I have worked with each had a set of tactics that they taught – so my sense (not supported by empirical research, since I’m not aware that this has been studied) is that geologists use whatever system they learned from their instructor or mentor. And if they were not taught a system, they made one up.

I’ll also echo R1’s comment that the paper needs to make a distinction between published geologic maps (or “final” maps that a student would submit for a grade) and working field maps and notes. Working maps are by their nature messy, subject to annotation, erasure, and multiple changes of mind. Published maps conform to community standards like the dotted-dashed-solid line notation discussed in the text.

I disagree somewhat with the claim (Lines 38-39) that the field is a student’s first expose to geologic uncertainty - other situations such as interpretation of geophysical data or remotely sensed imagery also require managing uncertainty. Though I do agree that the field is often a student’s first encounter with raw and messy geologic phenomena that do not match the tidy photographs in the textbook or the samples in the lab.

Lines 113-138 would be very well served with a figure or table that shows field examples and how those would be rated using the data and model categories and the uncertainty scale. This would really help readers to understand how the scale is applied in the field.

I really appreciate the argument on lines 147-158 that the framework could help students articulate exactly how they are uncertain. My experience as an instructor is that many students find themselves unable to say exactly how and why they are confused, and simply give up. An instructor could easily use this framework to prompt students to explain where they are stuck. And I agree that the framework could help students guard against getting so set on their geological model or interpretation that they disregard compelling contrary evidence (something that we saw people do in the expert-novice mapping study).

We also found that one of the challenges in working with students (related to lines 182-200) is their reluctance to form models or hypotheses during mapping. Whereas experts made and tested geological interpretations and hypotheses as they collected data, most novices waited until very late in the field exercise to form any interpretations. Could the framework help to address this problem and teach students to be more expert-like in how they approach making and testing hypotheses?

The paper would greatly benefit from further information about the group of experts who used the framework. What was their expertise and other demographic characteristics? 

Not only is there a quantitative range of spatial uncertainly for mappable features (lines 231-239) during geological mapping, there is also an element of locational uncertainty. Especially with students, they often struggle to accurately identify their physical location on a map. So it is very possible that they have presumptive or compelling data (or interpretations/models) but that they have entirely misplaced the location of key geological features. I wonder if there is space in the framework to recognize this additional form of uncertainty (e.g., where am I?)?

I found the order of the paper a bit odd. I wonder if the flow might be better if the expert development and testing of the framework was introduced earlier. So, moving sections 4 and 5 of the paper ahead of section 3. This ordering could address my prior comment and one of R1’s that an example of the framework would help readers understand it. Examples from the expert use of the framework could be used to construct this figure/table and its validity could be established if this section (lines 217-240) were moved earlier to when the framework is introduced. The paper could then focus on the applications of the framework to student use both in the field and in virtual instruction (presently section 3).

I am curious to learn more about how challenging it was to norm students to the uncertainty framework scale (Lines 241-249). I am also curious how students compared with experts – I agree with the point that students will likely express a higher degree of uncertainty; what may be presumptive evidence to an expert may be only suggestive to a student. [On the other hand, I know some experts who would never use “certain,” no matter how unmistakable the evidence.]

I was able to access the Sage Hen Pluton teaching materials located on the SERC website. It is not clear to me whether this teaching activity has been empirically evaluated for effectiveness. If not, I suggest softening the conclusions (specifically lines 260-266) to say that these are the potential or proposed benefits to students. As written, it sounds like these benefits are empirically tested.

Technical comments:

There may well by typos in this manuscript but none leapt out at me, thus I have no technical comments.