Analysis of cognitive skill in a water discharge activity for firefighting robots

Abstract

Teleoperated robots are expected to perform tasks in unexpected disaster sites, which have become increasingly common in recent years. However, only few teleoperated robots are used for such purposes owing to their poor usability compared to the handling of such tasks by humans. Further, human-machine interfaces (HMI) used in teleoperated robots are closely related to human skills. Firefighting activities require cognitive skill to make appropriate decisions. In extinguishing a fire or attenuating a leaked combustible gas, firefighting activities consist of two tasks: reconnaissance and water discharge to stop the fire or attenuate the leaked combustible gas, and these tasks possess different properties. In this study, we compared the cognitive skill of ten firefighters (five novices and five experts) in attenuating the leaked combustible gas, the latter of the said two tasks, using a teleoperated robot at a fire department. The experimental results indicated that experts were more multifaceted in their evaluation of the attenuation of combustible gas as compared to novices. In addition, as this result related to firefighter decision-making mechanisms, we were motivated to improve the HMI of firefighting teleoperated robots.

Introduction

Globally, recent large-scale disasters have occurred frequently. As the tasks at disaster sites are dangerous and the workload is heavy, a means to perform the tasks indirectly with teleoperated robots is an ideal quick and safe solution to avoid the various risks encountered. Many projects have been embarked on to realize this solution [1,2,3,4]. However, it is difficult to say that teleoperated robots are used in many disaster areas. Casper et al. [5] suggest that human-machine interaction (HMI) should be a priority in designing rescue robots. Hence, one of the reasons why the teleoperated robot is not used is that it has an inadequate HMI. In addition, as teleoperated robots are simply work tools, we suppose that the work effectiveness of the teleoperated disaster response robot depends on the operator’s rescue skills (for example, the cognitive skill at the disaster sites). A guideline for HMI of rescue robots has been developed [6]. This guideline describes how to display camera images for the operator to recognize the situation, how to display an environment map that shows the current robot’s position, devices that control the robot’s movement, and communication between the robot and the device, but it does not mention the display of information specific to rescue activities. There are various types of disasters, and fire is one of the common disasters for citizens. Hence, firefighting activities are crucial, and citizens often demand continuous improvement. Therefore, the development of teleoperated firefighting robots has been a hot topic [7], and the operator’s firefighting skills are important to design of the HMI of these robots.

The handbook for fire science and engineering describes firefighting activities as follows [8].

• Firefighting is the artificial and forcible stopping of fire to prevent flame spread and minimize damage.

• It is particularly important for firefighting activities to secure an activity base and water utilization.

From the above, we categorize firefighting activities into two tasks: reconnaissance to secure bases and water utilization, and water discharge to stop of fire or attenuate a leaked combustible gas. In this study, we define water discharge as the attenuation of combustible gas. This is because in places such as large plants where teleoperated robots are expected to play an active role, the number of combustible gas leaks is higher than the number of fires. These two tasks possess different properties. Reconnaissance is the task of information gathering while water discharge is the task of making decisions based on the gathered information. In a previous study, we analyzed the differences in firefighting skills between novices and experts in reconnaissance activity with teleoperated robot [9]. However, the difference in firefighting skill between novices and experts with teleoperated robots in water discharge has not been clarified. Humans make decisions based on various pieces of information in their daily life. For example, humans make decisions about which product to buy based on information such as the performance, price, and manufacturer of the product. Thus, we suppose that analyzing water discharge activity is related to the firefighter decision-making mechanism in terms of premising an action on gathered information by reconnaissance. Jonassen [10] compared normative and naturalistic approaches to decision making and mentioned that factors such as experience were considered in making decision. Hutton et al. [11] also mentions that experts possess superior memory and take the situation on a deeper level. Our hypothesis is that, in water discharge activities, experts have a more multifaceted perspective on the situation than novices, i.e., their cognitive skill are higher. The higher cognitive skill, the more likely it is that the firefighters will be able to respond flexibly to changes in various disaster areas.

In this study, to verify our hypothesis, we analyzed the cognitive skill of experts by comparing the behavior of novices and experts in water discharge activity to attenuate a combustible gas represented by hydrogen gas with the teleoperated firefighting robot.

We created evaluation items to measure the water discharge activity according to the basic tactics of firefighting, and we compared the behavior of novices and experts based on these evaluation items.

The water discharge process is as follows.

• (1) Set the water discharge position.

• (2) Set the hose route to water discharge position.

• (3) Operate the fire hose nozzle.

Here, the water discharge position refers to the position of the fire hose nozzle. These actions are the result of decision-making in water discharge activities.

In the case of teleoperated firefighting robots, the action of (1) is to set the robot’s position and the action of (2) is to set the robot’s route to the water discharge position, and the action of (3) is to operate the water cannon and camera of the robot.

In this study, the comparison actions are as follows.

• (1′) Set the robot position.

• (2′) Set the robot route.

• (3′) Operate the water cannon and camera of robot.

We confirm the difference between novices and experts by the evaluation items for these actions. We use the questionnaire and interview to measure the evaluation items.

If experts make decisions from a multifaceted perspective in water discharge activities, as our hypothesis suggests, we consider designing the HMI of teleoperated firefighting robots that enables novices to perform firefighting activities as experts. It is, for example, the HMI for teleoperated firefighting robots that displays evaluation items on the operation screen. Particularly in the disaster areas, unexpected events are likely to occur; hence, several items require greater attention than usual. However, to simply add the information on the operation screen does not necessarily improve the HMI. The importance of the HMI of teleoperated robots that filters input information effectively has been described in several studies [12,13,14,15]. However, these studies do not describe the HMI for firefighting (including water discharge activities) teleoperated robots. The mission of firefighting is to protect citizens’ lives and property from fire or disasters, and mistakes by firefighters may cause immediate loss of many lives and property. Therefore, in the HMI of teleoperated firefighting robots, ease of operation is a priority.

The remainder of this paper is structured as follows: “Evaluation checklist items” section details the evaluation checklist items to be compared between novices and experts for water discharge. “Experiment” section describes the experimental setup. “Result” section outlines the results, and “Discussion” section discusses the results in detail. Finally, “Conclusion” section provides the conclusions.

Evaluation checklist items

We create a set of evaluation items for the attenuation of combustible gas based on the basic tactics of firefighting [8] and the advice of experienced firefighters. The evaluation items are shown in Table 1. In this study, all activities related to the attenuation of combustible gas started after people were evacuated from the area. Hence, the ‘injured person’ is not included as an item in Table 1.

Table 1 Evaluation items

Experiment

As described in the previous section, the attenuation of combustible gas is one of the important activities of fire extinguishing. Therefore, we chose to conduct a comparative experiment examining the attenuation of hydrogen gas with a teleoperated robot to confirm the differences in firefighting skills between novices and experts. Since hydrogen gas is colorless and odorless, its existence can only be detected by gas detectors, for example. Therefore, in this experiment, without discharging water, the attenuation of hydrogen gas was substituted by changes in the value of the gas detector.

Experiment location

A diagram and photograph of the experimental field, which measures 80 m by 120 m, is shown in Fig. 1. Facilities A, B, and C were simulated as the oil-refining facilities. The leak area of hydrogen gas was simulated as the first floor of facility A. Facilities A, B, and C comprised the three buildings in Fig. 1b while the other facilities in Fig. 1a ware fictitious for the experimental setup. The experimental field was prepared by diverting a firefighter training facility.

Fig. 1

Experimental field

To enable realistic participation, the participants were shown a photograph of the actual oil-refining facility before the experiment was performed. However, this photograph is not included in this paper for confidential reasons.

Because the facilities in this experiment were used in actual firefighter trainings, and because the firefighting robot in this experiment has already been equipped in the firefighting headquarter, we considered that this experimental environment would be familiar to firefighters, would be less likely to cause misidentification of the this environment than a computer simulation or vertical reality, and would reduce the variation in the level of understanding of this experiment. Therefore, we did not use a computer simulation but a real robot and environment in this experiment.

Experimental configuration

Experimental configurations are shown below.

• Disaster situation: An epicentral earthquake has caused fires to break out simultaneously in many places in the prefecture. The valve of the oil refinery was loosened by the earthquake, and hydrogen gas leaked.

• Purpose of the attenuation: To allow the office staff and fire brigade members to enter the leak site and perform the work to stop the leak.

• Activity timing: To make the experiment easier, participants start the attenuation of hydrogen gas after the evacuation of citizens or office staff was completed. Only hydrogen gas was leaking, no fire had yet occurred, and all facilities were operating.

• Teleoperated robot: The firefighting robot, WATER CANNON, which was developed by the National Research Institute of Fire and Disaster is shown in Fig. 2. WATER CANNON is the member of SCRUM FORCE, the firefighting robot system [16]. In this paper, the name of the robot is capital letters as WATER CANNON, and the name of the water cannon is small letters as ‘water cannon’.

  Fig. 2

  

  ‘WATER CANNON’, the member of SCRUM FORCE

This robot has the world’s highest heat resistance and is equipped with a newly developed nozzle of water cannon that realizes wide-angle water discharge, semi-aspirate bubble radiation, and straight water discharge by switching one nozzle. The basic specifications of WATER CANNON are shown in Table 2.

Table 2 Basic specifications of WATER CANNON

In this experiment, participants operated WATER CANNON remotely in the operator station. Figure 3 shows the operator station: the laptop and the touch screen monitor operate WATER CANNON. WATER CANNON can also be operated with a PC game pad installed on the laptop. The camera and water cannon of WATER CANNON can be operated separately. Based on the results of the National Health and Nutrition Survey in Japan of the Ministry of Health, Labour and Welfare, the average height of Japanese males aged 20–49 is about 171.5 cm [17]. In order to reduce the difference between the field of view when the robot is used and when it is not used, we used this robot where the mounting height of the camera is at almost the same height as the firefighter’s eye level (168.3 cm).

Fig. 3

Operator station for this experiment

• Experimental participants: We prepared ten firefighters (five novices and five experts) to participate in this experiment. It is difficult for young firefighters to acquire firefighting skills because many depend on practical experience. Therefore, many years of firefighting experiences are required to become an expert. In this experiment, we defined the threshold of an expert as 10 years of firefighting experience as most firefighters, who focus on protection and rescue, become sergeants after 10 years of service.” (This means the firefighters start out by focusing on protection and rescue, then after 10 years of service, they become sergeants.) As our aim was not to compare the firefighters’ skill in operating WATER CANNON in this experiment, all participants only uttered verbal instructions, and the robot’s operation was consistently performed by a skilled operator.

• Leak area: The participants were informed that the leak area of hydrogen gas was inside the first floor of the central facility, as preliminary information. The participants were also notified of the % Lower Explosive Limit (% LEL) data of the leak area. The % LEL is the degree to reach the Lower Explosive Limit, and when this percentage reaches 100 %, it indicates an environment in which an explosion can occur. Hydrogen gas concentrations can be obtained from gas detection sensors installed at various locations in the facilities.

• Water discharge: We evaluated the water discharge amount of 4000 L/min, the direction of the water cannon, and the type of water discharge, assuming the following.

Straight (width: approx. 2.5 m, range: approx. 75 m, height: approx. 30 m).

Wide-range (width: approx. 7 m, range: approx. 30 m, height: approx. 15 m).

• Reconnaissance activities: The reconnaissance activities were completed at the Land Eye (driving-type reconnaissance and surveillance robot) and Sky Eye (flying-type reconnaissance and surveillance robot), which are members of the Scrum Force.

• Firefighting vehicle: It was assumed that there are two firefighting vehicles equipped with a water pump required for WATER CANNON discharge at the site.

• Weather conditions: The weather conditions in this experiment are shown in Table 3. We assumed the weather conditions for a typical summer day, which is similar to the season when this experiment was performed.

  Table 3 Weather condition

• Data acquisition: To compare the water discharge behaviors of novices and experts, we obtained several types of data in this experiment, which are listed in Table 4. We believe that the operator’s gaze data is one of the useful information for the design of HMI. However, since WATER CANNON used in this study is not designed to move after the start of attenuation activity. Therefore, during the attenuation activity, the operator can only monitor the attenuation status by camera images and move the water cannon and camera as necessary. In the case of a camera that can control the angle of elevation and rotation, the center position of the camera image and the gaze position are considered to be equivalent, so we treated the camera movement as gaze data.

  Table 4 Data acquired

Experimental procedures

Experimental procedure is shown below.

• (1) Explain the leak area of hydrogen gas and the purpose of the attenuation to the participant, after presenting him or her a recording sheet (as shown in Fig. 4), which is a typical Japanese recording sheet. The recording sheet is the plant facilities layout with information on the location of the leak, the weather conditions, and the command headquarter.

  Fig. 4

  

  Recording sheet

• (2) The participant set the water discharge position and route of WATER CANNON (concretely, this information is written on a recording sheet).

• (3) The operator (not participant) moves the water cannon to the position written on the recording sheet.

• (4) After the movement of WATER CANNON, the participant informs the operator to move the water cannon (up/down/left/right and straight/wide-range) and camera (up/down/left/right). The initial values of the water cannon and camera are shown in Table 5.

  Table 5 Initial values of water cannon and camera

• (5) During water discharge, the participant is notified of the concentration information at the leak area every 30 s. The experiment is terminated when the participant determines that the office staff and firefighters have entered the leak area and are able to stop the leakage. In this experiment, if the instruction of discharging water was to the leak area is given and the range is sufficient, %LEL was reduced by 10 % every 30 s.

• (6) The participant completes a questionnaire and interview regarding the evaluation items.

In the questionnaire, the following items were confirmed regarding the setting of the robot position, the robot route, and the operation of the water cannon and the camera.

• Which of the following evaluation items was considered?

• What are the priorities of the items considered? Please make sure to prioritize them.

• What has changed with the activities of the robot?

Result

Participants freely wrote the position and route of the robot on their recording sheets, and we divided their results into different groups as the results followed different patterns. Specifically, the patterns were that the robot positions were the up (Green diamond in Fig. 5), down (Blue diamond in Fig. 5), and left (Red diamond in Fig. 5) of the facility A at the leak area and the robot routes were on the roads described in the recording sheet. The groups of the robot positions and routes are shown in Fig. 5. The results of grouping and the results of the movement of the water cannon and camera are shown in Table 6. In Table 6, the circled numbers indicate the robot position and the roman numeral indicate the robot route (Fig. 5).

Fig. 5

Groups of robot position and route

Table 6 Results from participants’ recording sheets

We used the Chi-squared test to confirm the presence or absence of a difference between the novices’ and experts’ operation. The null hypothesis is that there is no difference between novices and experts.

First, the robot position was tested, and position (3) was the most popular answer for both novices and experts. The cross-tabulation table of Table 6 is shown in Table 7. The p value calculated from Table 7 was 1. Therefore, the null hypothesis was rejected. In other words, there is no significant difference between novice and expert populations.

Table 7 Cross-tabulation table of setting robot position

Second, the robot route was tested, and route III was the most popular answer for both novices and experts. The cross-tabulation table of Table 6 is shown in Table 8. The p value calculated from Table 6 was 0.519. Therefore, the null hypothesis was rejected. In other words, there is no significant difference between novice and expert populations.

Table 8 Cross-tabulation table of setting robot route

Third, we compared the movement of the water cannon and camera of the robot between novices and experts. The operating monitor during the attenuation of hydrogen gas is shown in Fig. 6. Participants could remotely confirm the situation of the leak area and the position of the water cannon from the camera’s image (Fig. 6a) and check the pan or tilt angle of the water cannon and camera on this monitor (Fig. 6b). The cross-tabulation table of Table 6 is shown in Table 9. The p value calculated from Table 9 was 0.519. Therefore, the null hypothesis was rejected. That is, there is no significant difference between novice and expert populations.

Fig. 6

Operating monitor during attenuation

Table 9 Cross-tabulation table of operating the water cannon and camera of robot

The results of the evaluation items (Table 1) of setting the robot position, setting the robot route, and operating the water cannon and camera of the robot are shown in Tables 10, 11, and 12. These tables were created by integrating the interview and questionnaire results. The ‘Total score’ in Tables 10 and 11, and 12 is the total value with a score of 2 if there was the intendedly answer, 1 if there was the non-conscious answer, and 0 if it there was no answer. In this study, “the conscious answer” means that the information is used consciously, “the non-conscious answer” means that the information is used non-consciously, and “no answer” means that the information is not used.

Table 10 Result of the evaluation items for setting the robot position

Table 11 Result of the evaluation items for setting the robot route

Table 12 Result of the evaluation items for operating the water cannon and camera of robot

We used the Wilcoxon rank sum test to determine whether there was a significant difference between total scores of novices and experts in Tables 10, 11, and 12 as the result of evaluation items did not follow the normal distribution.

First, from Table 10, the p value was 0.011(effect size: r = 0.83), and it was found that there was a significant difference between novices’ and experts’ groups. Since the effect size is higher than 0.5, the substantial difference was confirmed [18]. The means of the total scores in setting the robot position are shown in Fig. 7. The error bars indicate the standard error of the mean (SEM) of each group. * represent p < 0.05.

Fig. 7

Mean and SEM of the total score for setting robot position (*p < 0.05)

Second, from Table 11, the p value was 0.027 (effect size: r = 0.69), and it was found that there was a significant difference between novices’ and experts’ groups. The means of the total score in setting the robot route are shown in Fig. 8. The error bars indicate the standard error of the mean (SEM) of each group. * represent p < 0.05.

Fig. 8

Mean and SEM of the total score for setting robot route (*p < 0.05)

Third, from Table 12, the p value was 0.011(effect size: r = 0.83), and it was found that there was a significant difference between novices’ and experts’ groups. The means of the total scores for operating the water cannon and camera of the robot are shown in Fig. 9. The error bars indicate the standard error of the mean (SEM) of each group. * represent p < 0.05.

Fig. 9

Mean and SEM of the total score for operating the water cannon and camera of robot (*p < 0.05)

According to these results, experts evaluate the attenuation of hydrogen gas with a more multifaceted perspective than novices.

Figures 10, 11, and 12 show graphs of the evaluation items from Tables 10, 11, and 12, respectively.

Fig. 10

Comparison of checklist items evaluated by novices and experts at setting the robot position

Fig. 11

Comparison of checklist items between novices and experts at setting the robot route

Fig. 12

Comparison of checklist items between novices and experts at operating the water cannon and camera of robot

Table 13 shows the result of questionnaire on priority of checked evaluation items. The numbers in Table 13 indicate the numbers of the evaluation items listed in Table 1. Regarding the priority of the checked evaluation items, we confirmed that all the checked items were equally important, and there was no clear priority from the interview and questionnaire. Table 14 shows the result of questionnaire on changes in activities by the robot. To ensure safety, water discharge by humans is carried out upwind and at a distance from the target. We confirmed that many novices and experts placed robot in the location where it was at higher risk than humans, due to the water discharge by robots.

Table 13 Results from participants’ questionnaire (Priority of checked evaluation items)

Table 14 Results from participants’ questionnaire (changes in activities by the robot)

Discussion

To validate a hypothesis, this study aimed to investigate the differences between novice and expert firefighters using the attenuation of hydrogen gas. Our hypothesis was that experts, compared to novices, can consider more information based on their experience to make decisions in water discharge activity. We investigated and confirmed the hypothesis. Thus, we suppose that the knowledge of various types of the facilities, water discharge, and leak accidents are the cause of the significant difference between novices and experts in the evaluation items for the attenuation of hydrogen gas. Novices have insufficient experience and bias against water discharge to facilities; thus, it is difficult for them to organize what to check for water discharge. However, experts can organize the items to check for water discharge from their vast experience. We suppose that this is because experienced firefighters interpret fire situations as familiar patterns from their own experiences for efficient decision-making [19]. Humans make similarly efficient real-time decisions by applying typical patterns obtained from their daily life experiences. Therefore, in water discharge, ‘novices’ corresponds to the ‘competent’ skill level and ‘experts’ corresponds to the ‘expert’ skill level. Here, ‘competent’ and ‘experts’ indicate the five stages of skill acquisition proposed by Dreyfus [20].

We discuss a comparison between the results of our previous study [9] and the results of this experiment. It is consistent with the results of previous studies that the difference between novices and experts in the water discharge activity is multifaceted perspective. Thus, we clarified that improving cognitive skill is important for novices to perform firefighting activities like experts. On the other hand, our previous study confirmed that there was no significant difference between novices and experts in the evaluation items for setting the firefighting vehicle position in urban fires, but there was a significant difference between novices and experts in the evaluation items for setting the robot position in the hydrogen gas leak. This is because the characteristics of colorless and odorless hydrogen gas are difficult to recognize compared to visible flames, and the hydrogen gas leak is considered to be a difficult disaster site for novices to deal with. We think that the experts have a deep understanding of the disaster site through the multifaceted perspective built on their experience and can handle the difficult disaster site easily.

In this experiment, no difference was found between novices and experts in superficial action results such as the robot position, robot route, and movement of the water cannon and camera. This may be because the movement distance of the robot was limited to 300 m from the start position, and the angle of view and range of the water cannon were large (Fig. 6a). First, the movement distance of the robot was short relative to the field size. If the movement distance of the robot was longer or the field size was smaller, there might have been differences in the setting of the robot position and route. Second, nearly the entire leak area could be confirmed without moving the camera and water cannon. If the angle of view and the range of the water cannon was smaller, there might have been a difference in the operation of the water cannon and camera between novices and experts.

The interviews confirmed that if novices are directed by others to mark checklist items, they can perform the attenuation of combustible gas in multifaceted perspective as experts. This may present a method to bridge the gap between novices’ and experts’ cognitive skill in the attenuation of combustible gas. Therefore, we consider applying a notice that offers novices multifaceted perspectives to the HMI of firefighting robots in future works. In this case, the keyword of attention is important. This is because there is a limit to human information processing ability at the level of the sensory organs and memory system; thus, it is necessary to restrict the objects that require attention to be executed per time [21]. If information processing capacity is not properly used, important items may be missed. This is closely linked to the understandability that Norman [22] describes as important in the design of cognitive interfaces. Examples are seen in the following prior studies on attention.

Wolfe et al. [23] verified the prevalence effect that the oversight rate increases in inverse proportion to the frequency of appearance. He et al. [24] also reported that the attention resolution was higher in the lower visual field than in the upper visual field. Therefore, we conclude that the notices for evaluation items should be additional emphasis depending on the frequency of appearance, and they are better displayed in the lower area of the operation screen.

In the following paragraphs, we discuss individual evaluation items to reflect this result in the HMI design of teleoperated firefighting robots. However, the discussion qualitatively presents the difference between novices and experts. To confirm the significant difference in the results for individual evaluation items, we used a small sample size; thus, a more detailed experiment is necessary for future works.

First, Fig. 10 indicates the differences in the items between novices and experts in setting the robot position. The items, type of material, leak concentration, leak area, environmental conditions of leak area, type of water discharge, building entrance, and activities after concentration reduction, are difficult to ascertain, and they depend on several years of firefighting experiences. Particularly, setting the robot position is the first action in water discharge activity; however, experts make decisions by considering at an early stage the evaluation item and activities after concentration reduction. In addition, hydrogen gas is invisible owing to its colorlessness, and it occurs less frequently than fires. Hence, it is expected that novices would not check these evaluation items.

Second, Fig. 11 indicates the differences in the items between novices and experts in setting the robot route. The items, wind direction and activities after concentration reduction, are difficult to ascertain, and they depend on many years of firefighting experience. As the item, wind direction, changes dynamically in time series, it may change while the robot moves. However, novices do not check it again after the first check. Based on the results of this study, novices tend to feel satisfied after checking the information once. The item, environmental conditions of leak area, is not checked by experts but by novices. This may be because novices are slower than experts in checking the items. The robot’s route is set after the robot’s position is determined. All experts checked the item, environmental conditions of leak area, in setting the robot position; however, novices A and C checked this item last in setting the robot route.

Third, Fig. 12 indicates the differences in the items between novices and experts in operating the water cannon and camera of the robot. The items, wind direction, type of material, leak concentration, leak area, environmental conditions of leak area, type of water discharge, building entrance, and activities after concentration reduction, are difficult to ascertain, and they depend on many years of firefighting experience. This result is similar to the result of setting the robot route.

Considering these differences in the checklist items, we conclude that the HMI that displays these evaluation items as messages or marks on the operation screen is useful in operating the water cannon and camera of the robot.

Conclusion

In this study, we analyzed experts’ cognitive skill in water discharge by comparing the behavior of novices and experts in a firefighting activity with teleoperated robots. We adopted the attenuation of a combustible gas as the water discharge activity. By comparing the behaviors of novices and experts during the water discharge activity, we confirmed that the difference between novices and experts is the presence or absence of multifaceted perspectives in decision-making. The experts evaluated the attenuation of hydrogen gas in a multifaceted way and acted based on that result. In contrast, the novices evaluated a limited range of the attenuation of hydrogen gas.

In this experiment, interviews confirmed that if checklist items are pointed out to novices by others, the novices can perform a multifaceted evaluation as experts. Based on the results of this experiment and our previous study, we believe that by creating the HMI of teleoperated firefighting robots that presents evaluation items as a message or mark on the operation screen, firefighting activities (especially fire extinguishing) from a multifaceted perspective similar to that of an experienced person is possible even for novices. In addition, by reflecting the test results of this experiment in the education system for firefighters, we expect an early acquisition of firefighting skills by novices.