1 Introduction

Beyond efforts to steadily increase the degree of automation for individual transportation (i.e., cars), automated buses are increasingly in the focus of research, development, and deployment efforts [1]. Today, automated shuttles are available for purchase from several manufacturers—most predominantly from several France-based companies [2, 3] but there are others as well [4]. Be it the Navya Arma, the EasyMile EZ10, or the Local Motors Olli—all of them aim to eventually provide a completely driverless vehicle that is fully capable of safely and efficiently completing all driving tasks in the relevant (urban or rural) traffic environments.

Achieving full automation (i.e., SAE level 5 [5]) in on-road public transport is considered a vital step to achieve Vision Zero, which refers to the goal of eliminating casualties and reducing serious injuries caused by on-road accidents—and both connected and automated vehicles are expected to play an essential part on the road to this goal [6]. Vehicle automation is expected to help achieve vision zero by reducing—and eventually eliminating—human errors. This means that the vehicle does not merely replace the human driver role, it must surpass it in order to perform better than a human driver would. In public transport, the driver’s role is often not limited to getting the vehicle from point A to point B. The driver can also be responsible for ticketing and capacity management, interventions in case of incidents (be they of technical nature or between passengers), or act as a source of information regarding itinerary and possible connections. Thus, it stands to reason that, beyond the driving performance, a fully automated or driverless means of public transport should be able to address and provide these functions or it would not be a fully realized replacement for the (formerly) human driver role.

In public transportation today, especially as far as accessibility is concerned, there are many people who still rely on a human driver to receive help when needed [1]. Furthermore, emergencies in driverless vehicles can be perceived to be worse or more severe than in conventional buses [7]. This suggests that substantial additional effort is required in order to elevate automated shuttles to the same level as conventional ones as far as passenger needs in safety critical situations are concerned.

In this paper, we present an investigation into potential incident situations in a driverless shuttle bus together with an attempt to guide and support passengers in the incident resolution via auditory announcements. We begin with an outline of the specific research questions and goals, followed by an overview of related work in the area. We then describe the technical setups and procedures for both studies that were conducted. After that, we present the quantitative results across both studies as well as the more in-depth qualitative results from both studies separately. We conclude with a reflection in the discussion.

2 Study goals and research questions

In order to address the aforementioned challenge of autonomous shuttle behavior in incident situations, we set out to define the scope of investigation and corresponding research goals. As technology matures and automated vehicles are more and more integrated into the infrastructure (and in line with previous research [7, 8]), we expect the following cases to occur and/or be relevant in a variety of different contexts:

  • The vehicle stops outside of a designated stop, either in the middle of the road or on the roadside without a discernible cause. This could be due to any number of reasons ranging from technical defects to missing or incorrect routing information from the vehicle’s sensors or infrastructure provider.

  • The vehicle stops outside of a designated stop, either in the middle of the road or on the roadside with a (potentially) discernible cause. This could occur due to congestion, roadwork, road damage, accidents involving other traffic participants, objects on the road that cannot be easily evaded, and similar situations.

  • Passengers feel uneasy, threatened, or otherwise unsafe before boarding or while riding an automated bus. Such situations could include dirty interiors, intoxicated individuals being present in the bus, or riding at night alone or with only complete strangers present in the bus.

We defined unplanned vehicle stops with discernible versus not discernible causes separately for the reason that the latter might, depending on the situation, provide the opportunity for the passengers to interfere and resolve the situation themselves. Due to any number of possible quirks or faults, it is not inconceivable for the bus to stop due to something comparably trivial (e.g., a light object on the road, misdetected as a larger one). At the same time, passengers cannot be expected to act in the way a professional driver would, so it is interesting to investigate both the capabilities and willingness of passengers to interfere in such situations. To this end, we decided to focus on the following research questions:

  • RQ1: Which are passengers’ needs for intervention capabilities and information provision in a highly automated shuttle bus regarding

    1. Scenario 1:

      unplanned stops without discernible cause,

    2. Scenario 2:

      unplanned stops with discernible cause, or

    3. Scenario 3:

      potentially threatening or otherwise unsafe situations?

  • RQ2: Can passengers be expected to intervene and resolve certain unplanned stop situations and, if so, is it possible to provide assistance via standard communication means inside a bus?

For providing incident-related information and assistance, the auditory channel was chosen because it is the primary channel for communication of information related to unexpected events in public transportation (e.g., planes, buses, trains). Visual information mostly communicates itinerary information (and related deviations), general rules of conduct in emergencies, and information that accompanies the auditory announcements whenever the event was among the expected risks.

A lot of work in automotive HCI focuses on general acceptance of driverless means of transport and operation under normal conditions. The volume of user-centered research on incident management in automated public transport is still rather low. Therefore, we opted to start on a level that focuses on user needs in specific incident situations (RQ1), together with an investigation of potential solution strategies (RQ2). We expect these results to inform research and development related to unexpected situations or nonstandard behavior in automated public transport means—both in relation to potentially occurring situations and the impact these have on the passengers, as well as what can to be implemented in the vehicle or provided to the passenger in order to resolve the situation. In addition, we have documented our study setup, which enables wizarding of auditory announcements and in-vehicle passenger displays without requiring access to the bus’ internal systems.

3 Related work

Existing work and literature specific to incident management in automated public transport is still rather limited, with most work focusing on incident management in cars in relation to take-over-requests (TORs), acceptance of automated buses in general, or communication between automated buses and passengers or the driving environment in mostly non-emergency conditions (e.g., [9,10,11,12,13,14,15]).

Faltaous et al. [16] investigated how to communicate to the driver of a SAE level 2–3 vehicle when the system failed or an unexpected situation for the vehicle has occurred. Their results are five design guidelines, which are targeted toward driver space design for level 2–3 vehicles and, thus, are of limited suitability for fully automated public transportation scenarios.

Verma et al. [17] presented a co-design study with the goal of communicating intent or operational state of an automated shuttle to other road users. Their work focused primarily on the co-design approach and aimed at gathering overall user requirements for successful interactions. As a result, the design implications are similarly general with no specific focus on incident management. Wintersberger et al. [18] investigated the acceptance of passengers in an autonomous bus compared with a taxi. The results showed that only the slow speed of the bus reduces the usefulness.

In public transport, constant information presentation is required [19]. This includes pre-trip and on-board/wayside information targeted at different user groups such as age groups (children vs. older passengers) or type of travel (e.g., long-distance commuters vs. tourists) [9]. Millonig and Fröhlich [20] identified four passenger needs in automated shuttles: availability, affordability, accessibility, and acceptability. They state that it is difficult to transfer findings from the automotive domain to automated buses and stress the need for transparency and efficiency when the bus is communicating with its passengers.

Brown [21] and Brown and Laurier [22] raised the issue of automated vehicles having to respond to a number of social challenges, where “correct” behavior from a legal and technical standpoint can be interpreted incorrectly from a social point of view (e.g., in relation to gaps between vehicles). Eden et al. [23] further stressed that there are not only technical but also social challenges when designing for level 4 automated shuttles.

Passengers’ safety perceptions are known to have an impact on public transport ridership, i.e., on people’s willingness to use public transportation. Yet, the effects that feelings of personal safety have on transit ridership are not widely researched (Delbosc and Currie [8]). In a study within the CityMobil2-project, Salonen et al. [7] found that 54% of passengers in a small driverless shuttle bus found the emergency management (fire, vehicle failure, etc.) of the bus either worse or even much worse than that of a conventional bus. Mahmoud and Currie [24] identified measures to address personal safety issues while travelling on public transport vehicles, with 55% of people (n = 239) ranking roaming security guards on public transport as their preferred measure to take. Another 16% would want to refuse entry to intoxicated persons, and 12% would feel safer with security cameras on board. Alarms or panic buttons to alert guards were ranked first by 10% of people, while another 4% would feel safer if the lighting on public transport would be increased. Also, Stradling et al. [25] investigated people’s reasons to not take public transportation, and again drunken passengers at a nightly hour discouraged people the most to get onto the next bus (45%, n = 1.012).

While there is some valuable work available related to management of incidents and unexpected situations in automated vehicles, there is still a gap of identifying specific relevant situations and stakeholders’ requirements within these situations, as well as strategies for effective incident management, especially in driverless public transport. We contribute to closing this gap by presenting the results from an in-depth qualitative investigation for incident management in an automated shuttle bus.

4 Study setup

We conducted two studies to address our two research questions. Study 1 was intended to address RQ1 and gather passengers’ requirements for the three defined scenarios. Study 2 was a follow-up study with a focus on scenario 2 (unplanned stop with discernible cause) addressing RQ2. We implemented standardized auditory messages in the bus to assist passengers to address RQ2. Both studies were conducted in the field—study 1 on a closed test track and study 2 in a real road environment (see Fig. 3). For both studies, ethical approval and data privacy measures were obtained.

Study 1 lasted for 3 days and took place at a closed test track for driver training. The shuttle bus was an EasyMile EZ10 first generation model. Study 2 was realized on a road in a small village in a real road environment and lasted for 1 day. The shuttle bus was an EasyMile EZ10 second-generation model. Both models target operation on SAE level 4. For safety and legal reasons, however, a trained operator had to be present during all rides. The operator interacted with a control unit, which was inactive unless input was required. The study tracks at both locations were circular and had approximate lengths of 800 (study 1) and 3000 (study 2) meters.

The track of study 1 included a stop at a traffic light, a roundabout section, and two bus stops (see Fig. 1). The track was even and the average lap time was 5 min. The bus drove with a speed between 8 and 12 km/h, faster on the straight segments and slower during turns and similar maneuvers. The only other road users occasionally present at the track during the study were student drivers and their attendants.

Fig. 1
figure 1

The two tracks used for study 1 (left) and study 2 (right)

Full size image

The track of study 2 was a regular road with six bus stops, two of them serving as turning points. The track had ascents and descents and the average lap time was 24 min. The bus drove at a speed of 5 to 15 km/h depending on the current traffic situation and gradient. Different types of road users (e.g., cars, bikes, tractors, buses, pedestrians) were present during the study.

The buses used in both studies provided seating for six passengers. Both studies required two wizards: one in the bus to control announcements (wizard B) and a second one outside the bus to simulate an intercom control center (wizard I). Wizard B had a tablet for control of the auditory announcements in the bus and was seated left of the entrance on the seat furthest to the back. Passengers were not informed about the role of the wizard but were told that this person was a researcher logging technical data. Wizard I was equipped with a mobile phone to simulate the chatbot communication in case the passengers used the intercom for support. The operator stood in the bus at all times to be able to reach all emergency controls in case of an unexpected incident. Passengers were told not to communicate with the wizard and operator but behave as if they were not present. In study 2, another seat was taken by an observer, who was there to observe the incident situation but was introduced as being a fellow passenger (see Fig. 2).

Fig. 2
figure 2

Technical setups and positions of passengers and researchers inside the shuttle

Full size image

In both studies, an intercom equipped with a touch display was installed in the bus next to the seat on the right, furthest to the back. The intercom interface was used to call the control station in case of an emergency. When the participants pressed either and SOS or info button on the intercom, a connection to an intercom chatbot was established. This chatbot was operated by wizard I who was outside the bus remotely via a smartphone app. The bus was equipped with two GoPro cameras at the back and in the front to record all rides. In order to simulate an obstacle on the road (required for scenario 2), a luggage trolley on wheels was placed in the buses’ path, which could be removed easily by the passengers while being big enough to be visible from inside the bus.

4.1 Technical setup for audio announcements

The bus announcements in both studies were stored on the intercom as individual audio files. Via a simple Web interface on a tablet, which sent https requests to the intercom, wizard B could play back each of these files individually via a simple tap. The announcements in study 1 contained regular on-trip information (bus stops, information of imminent departure, etc.). Information relating to incidents was limited to a general notification and guidance on how to contact the control center (see items 7, 8, and 14 in Table 1). Instructions on how to resolve the incident were entirely provided by wizard I acting as a chatbot based on a pre-defined speech protocol.

Table 1 Overview of the speech items used to simulate the chatbot inside the bus
Full size table

In study 2, the aim was to resolve the situation by using only the pre-recorded bus announcements. Thus, an extended list of items for incident-related announcements (see full list in Table 1) was used. Wizard I, who simulated the control center, was still present but only as a fallback option. For study 1, we used a machine voice to record the bus announcements. Due to feedback from the participants that the announcements were perceived to be unpleasant and difficult to understand at times, we used a different solution for study 2. For study 2, all speech items (see Table 1) were recorded by a researcher taking care to use proper pronunciation and intonation as it is usually used in public announcements. The samples were then processed with the audio software Logic Pro [26]. In order to achieve a better quality of the recordings the very low (< 20Hz) and the very high (> 20 kHz) frequencies of the files were cut and voice clarity was improved. At the end, the files were compressed to − 20 RMS and − 8.0 peak to further improve voice volume. There was a limit to the possible number of files as the space on the intercom’s internal storage was very limited (about 18 MB).

4.2 Specifics study 1

Study 1 aimed at covering all three kinds of scenarios defined in the study goal section in order to identify passengers’ requirements. As it was situated at a test track, there were fewer restrictions regarding traffic regulations and contextual constraints as in a real road environment.

4.2.1 Scenario 1: interruption of operations

Scenario 1 was set to occur during the second round of a three-round ride on the test track. Bus announcements activated by wizard B informed the passengers about an interruption of operations due to unknown reasons. Then, after approx. 2 more minutes without any further information, participants were informed that the bus would be able to continue its journey. Scenario 1 did not require the participants to take any action as the situation was resolved automatically after some time had passed. Scenario 1 focused on the lack of the bus driver as a person to speak to in the case of an irregular stop and what amount of information is sufficient enough to feel safe in automated public transport in the case of a service interruption.

4.2.2 Scenario 2: obstacle

Scenario 2 was a variation on scenario 1. Once again, the bus had an unplanned stop but this time due to an obstacle, which obstructed the buses’ path. Initially, this was not revealed to the participants. Instead, they were first informed about the bus not being able to proceed and that they should call the control station with the help of the intercom in the bus. These instructions were also activated by wizard B via bus announcements. Throughout the scripted conversation with wizard I via the intercom, participants were then offered two choices, either to remove the obstacle themselves or to call an emergency vehicle to remove the obstacle for them. The scenario concluded if the obstacle could be successfully removed and the bus had completed its ride or immediately, in case the participants chose to call an emergency vehicle instead. Scenario 2 focused on the participants’ willingness to take over responsibility and act as well as the usefulness of the chatbot conversation for resolving the situation.

4.2.3 Scenario 3: threats

Scenario 3 comprised three hypothetical threat scenarios, which were discussed with the participants while they rode in the automated shuttle bus. The focus of scenario 3 was set on vandalism, harassment, and being late to catch the shuttle bus in time and the question whether or not the bus should stop in a situation like this. Due to their nature, these situations were not simulated like scenarios 1 and 2 had been. The three scenarios were read to the participants aloud, after which they were asked to give their opinion on each of these situations separately.

4.3 Specifics study 2

Study 2 focused entirely on scenario 2, since we wanted to further explore passengers’ problem solving potential in a more focused setup. Also, we used feedback from study 1 to implement an improved auditory setup for study 2

After study 1, we found this scenario to be the most relevant and interesting one with regard to finding out about the needs of passengers in automated public transport in the case of an unexpected incident. As we had already experienced the phenomenon of diffusion of responsibility during the initial study, we wanted to pursue it further.

Also, some of the feedback collected during scenario 2 in study 1 got already implemented in study 2. For example, the information provided by the intercom chatbot had been experienced as over-lengthy and complex. Therefore, the auditory information was reduced to concise auditory recordings/in-bus announcements, which were played automatically after the incident occurred without any further passenger interaction necessary. The intercom chatbot was only the secondary means of interaction and set up as a fallback, in case the participants needed more guidance. Hence, it was no longer necessary to rely on the intercom chatbot to resolve the situation but instead the in-bus announcements supposedly provided sufficient information to the passengers to resolve the situation efficiently and in a satisfying manner (see Table 2 for overview on in-bus announcements).

Table 2 Overview of the quantitative measures used for each study
Full size table

5 Study procedure

Participants were recruited via various channels (e.g., mailing lists, bulletins at municipal office, local associations). Exclusion criteria were unaccompanied children under the age of 14, wheelchair users, and people with baby carriages due to legal reasons. None of the subjects participated in both studies.

Studies 1 and 2 proceeded very similar. Participants were welcomed at an appointed meeting place and time. They were introduced to the study procedure and signed an informed consent. Legal guardians signed for their underage children. All participants filled in a pre-ride questionnaire (see Table 2). The two children taking part in the real road study were also provided with age-adjusted versions of all questionnaires that were handed out during the study.

In study 1, one seat was occupied by wizard B. In study 2, two seats were occupied by wizard B and the observer. Thus, participants were split into groups of a maximum of five (study 1) and four (study 2) people. Each group was led to the bus stop where the bus initially departed from. Before getting into the bus, the participants received behavior instructions (e.g., to stay seated at all times during the ride) and were advised to not interact with the operator and the researcher (wizard B). During the second study on the real road, they were allowed to talk with the observer, who was introduced as being a fellow passenger. All participants had the task to take a ride until their designated bus stop.

The setup for scenario 2 varied a bit between the two studies. While on the test track (study 1), the obstacle occurred immediately when the bus was about to leave the bus stop; on the real road (study 2), the obstacle occurred at the bus stop, which was the turning point on the route after a 12-min ride.

During the ride as well as the incident, wizard B played the bus announcements in accordance with what was actually happening on the route (e.g., “Next stop: [stop name],” or “An obstacle has been detected, which appears to be blocking the bus’ path.”). The obstacle for scenario 2 was placed in the bus—after the first bus stop in study 1 and at the turning point bus stop (after a 12-min ride) in study 2. After the incident was resolved either way (participants removed obstacle) or the other (control station sent for emergency vehicle to remove obstacle), the bus continued its ride back to the first bus stop. Participants, then, got out of the bus and filled in the post-ride questionnaire (see Table 2).

The study design slightly varied here again. Participants in study 1 only had to fill in the post-ride questionnaire, while participants in study 2 had to fill in two additional questionnaires. One addressed the perceived quality of the bus announcements and the other the perceived safety during the ride (see Table 2). After filling in the questionnaires, the incidents were discussed in the group with respect to the participants’ experiences. Suggestions for improvement with regard to the announcements and emergency measures were collected. The final discussion was recorded with a voice recorder.

One run with one group of participants took about 1 to 2 h in total, depending on the number of incidents the participants experienced and how quickly they resolved scenario 2. If participants were to take part in study 1, scenario 1 always happened prior to scenario 2 because the procedure was similar to the one in scenario 2 but with one major difference: participants were not held responsible for solving the problem, but were just experiencing it.

Scenario 3 was always set up to conclude the study as it was an extension of the already ongoing discussion started after scenario 1 and/or 2. It was, in fact, an in situ discussion group, with participants discussing threat scenarios, while riding in an autonomous shuttle bus.

5.1 Participants

Overall, 24 participants went on a ride with the automated shuttle bus and experienced an incident in study 1 (13 participants in three groups) and study 2 (11 participants in three groups). In study 1, four participants experienced the obstacle scenario (2) only, while four other participants only experienced the interruption-of-operations incident (scenario 1) and discussed in-vehicle security based on the three threat scenarios (scenario 3). Five participants were involved in all incidents during the test track study (Fig. 3).

Fig. 3
figure 3

Study: Participants (right) exiting the bus to remove the trolley (top left)

Full size image

Of the 24 participants, 16 passengers were female (66.7%) and eight passengers male (33.3%). Five participants belonged to the age group of 18-to-25-year-olds (20.9%), 13 to the age group of 26-to-50-year-olds (54.1%), and four participants were over 50 years old (16.7%). Also, 2 children took part in study 2, who were 8 and 10 years old. Over 90% of the participants (n = 22) have had no pre-experiences with self-driving vehicles before taking part in the study. One has been riding in a Tesla and one in a different kind of automated shuttle bus. One-fifth of them reported to use public transport on a daily basis (20.8%), another fourth several times a week (25.0%). In total, 16.7% use public transport at least several times a month, one-fourth one time a month or less often (25%), and 8.3% reported to never use public transport at all. Two participants indicated to be near-sighted and one participant reported to have a mild form of eye cataract.

6 Results and general reflection

The pre-ride and the post-ride questionnaires were completed by all participants in both studies. These were intended to provide a general insight into the potential effect of incident situations with automated buses on passengers’ perceived safety, reliability, and convenience. Thus, these results are reported for the whole sample. Where relevant, we inserted quotes from the discussions or video observations (primarily from study 1) for the purpose of illustration. The qualitative results are reported afterwards and separately for each study. In general, the results from the quantitative scales are, due to the participant numbers, primarily to be considered a basis to be supplemented with the qualitative results.

Before and after the ride, participants judged the bus on a 4-point scale (fully agree, rather agree, rather not agree, not agree at all) with regard to the characteristics: reliability, safety, and convenience. There was also a “don’t know” option available.

Overall, participants seemed to be able to better form an opinion on the shuttle bus, which can be seen in the decline in “do not know” answers after the ride. Regarding safety, before the ride, 40% of participants were not sure whether being in an autonomous shuttle bus would be safe. After the ride, nearly 55% of them fully agreed or rather agreed to the ride being safe, but also over 40% did rather not or not agree at all to the ride being safe. As one participant mentioned: “I’m rather disillusioned about trusting the bus to be safe after today’s ride, because without an operator nothing works.” Another one stated: “The audio system failed, so one is at the mercy of the bus. I don’t find this trustworthy. I thought my trust towards the bus would be higher than it was after this ride.” Another participant questioned the bus’s safety with increasing speed: “If the bus has a speed of 10 km/h it’s fine but if it’s speeding up to 50 km/h, I would feel safer with a safety belt.” (study 1). Since all participants had experienced incidents during the ride, a change in perception toward more concrete opinions (both positive and negative) was to be expected. Interestingly, however, these changes in perception of the bus’ safety turned out to be non-significant after performing a Wilcoxon signed-rank test (z = − 1.490,p < 0.136). This suggests that the participants might not actually have considered all the situations they experienced as safety critical but rather associated them with the reliability of the bus.

With regard to reliability, the “do not know” ratings before and after the ride were rather similar to the ones for safety. Before the ride, over 45% of the participants were not sure whether the bus would be reliable or not. After the ride, two-thirds of them fully agreed or rather agreed to the bus being reliable, while one-third considered the bus to be rather not reliable, which is an increase of slightly more than 20% in this category compared with before the ride, which supports the assumption that people not only assessed the incidents as safety relevant but also as critical with regard to the reliability of the bus. These results were significant (z = − 2.250,p < 0.024). Especially, when the actions of the bus were not transparent for the participants, they felt left alone: “We stood on the street for 3 minutes, without knowing why. We also pushed the button but the door did not open.” (study 2). Another one stated: “You think: ‘Why is the bus stopping now?’ but you get no answer. That is a weird feeling.” (study 2).

It was a bit different with the estimated convenience because over 40% of the participants were rather sure initially that a ride in an autonomous shuttle bus would be a rather convenient experience, but afterwards nearly as many participants rather did not agree to that. A fifth found it to be fully convenient, though, and no one found it to be not convenient at all. These results were also significant (z = − 2.390,p < 0.017). One participant criticized: “I sat against the driving direction, which I found unpleasant. I would like to keep an eye on the surroundings.” (study 2). One participant underlined the comfort of a bus driver being present in the bus. who passengers can talk with: “Frankly speaking, it’s just more convenient to have a bus driver in the bus, who tells you what is happening right now.” (study 1). Overall, the participants appear to consider the issues and failures presented to them during the study as potentially inconvenient but less as indicators for the bus’ performance under safety critical (i.e., with potential for bodily harm) situations (Fig. 4).

Fig. 4
figure 4

Study 2 results regarding perceived reliability, safety, and convenience

Full size image

After getting off the bus, participants were asked how much they liked the ride on a 4-point scale ranging from very much to not at all. In total, 30% liked the ride very much and 70% liked it. Participants were also asked how safe they felt during the ride on a 4-point scale (very safe, safe, less safe, not safe at all). Nearly 80% of the participants felt either safe (58%) or very safe (30%). The remaining 12% felt less safe, but no one felt not safe at all. The adult participants (n = 22) were also asked if they would let their (potential) children take a ride on the autonomous shuttle bus. All participants, who were parents agreed that they would let their children take a ride on the self-driving shuttle bus (36.4%). Of the participants without children, one-half agreed to allow their children to get on board of the autonomous shuttle bus (31.8%), while the other half would not allow that (31.8%).

7 Qualitative results

In the following, at first, the qualitative findings from the discussions, video observation, and additional scales used are presented. Once again, we first report the results from study 1 and afterwards the results from study 2 separately. Since study 1 focused on requirements in three case scenarios (RQ1), the findings consist of a number of participant requirements for each case. In addition, potential supportive and hindering conditions as well as solution strategies are presented. For study 2 (primarily addressing RQ2), we report applied solution strategies and supportive and hindering conditions as well as participants’ perceived safety and perceived voice quality [27]. The latter was added to further supplement RQ2 and to detect potential influences caused by the study setup (specifically the quality of the recorded announcements).

7.1 Study 1

7.1.1 Scenario 1: interruption of operations

The bus suddenly stopped after it left the bus station and the audio information that an interruption of operation occurred was given. After a few seconds, the information was given that the ride will continue.

In general, the participants experienced the situation as non-hazardous. The participants stated the following requirements:

  • Audio information The provided audio information was assessed as helpful and sufficient. Repetition of information was considered necessary, if the interruption lasts longer than 5 min.

  • Textual information Textual information should be provided additionally on displays to inform the passengers about the duration of the interruption, if they could set any action or have any possibilities to get more information about the situation, e.g., use of intercom in the bus.

  • Interaction via intercom In case of longer interruptions, the participants expected be able to interact with a real person via an intercom.

  • Getting off the bus A further requirement was to include a function that passengers can get off the bus in case of interruption of operations.

  • Video surveillance (CCTV) Although video surveillance as an additional feature in the bus was discussed controversially, it was considered important in more threatening emergency cases. (“If it is necessary it would be good that the cameras are active, but one is observed all the time.”)

  • Interaction with environment The interaction of the bus with the environment to inform other road users that there is an interruption of operation was also an important aspect for the participants.

7.1.2 Scenario 2: obstacle on the road

As outlined, the second incident was a situation in which an obstacle in front of the bus hinders the bus from continuing the ride. The participants were informed that there is interruption of operation and that the control center should be contacted.

Based on the video recordings of the rides of the three test groups, we coded which strategies the participants applied to manage the incident. In the final discussion, the participants were asked how they experienced the incident, if the acoustic information was helpful to manage the incident, and which improvements they would suggest. To structure the findings, in the following, the applied strategies, hindering conditions as well as the suggestions for improvement are presented.

The applied strategy can be characterized as active and cooperative. Both test groups reacted immediately after the audio information, that the control center should be contacted, was given. The coordination of action was experienced as easily: the person next to the intercom contacted the control center and a volunteer got off the bus and removed the obstacle.

The following conditions were regarded as hindering:

  • Explanation of functions of buttons There was a short confusion if the emergency button or the info button should be pressed and what possible consequences might be. Here, a clear information is requested. Does pressing the emergency button alert the police or ambulance or just the control center? (”I pressed the info button, but it is was not clear if the info or the emergency button should be pressed. It was not an emergency, thus, I pressed the info button.”)

  • Chatbot vs. human The conversation with the chatbot was assessed as too long, especially for life-threatening situations immediate help is expected, preferably by interacting with a human instead of a chatbot. (“The communication took quite long. It should be possible to get help quicker in case of emergency.”)

  • Diffusion of responsibility Although both groups resolved the situation by active involvement, the phenomenon of diffusion of responsibility was discussed, which describes the phenomenon that a person is less likely to take responsibility to act accordingly when others are present. The setting on the test track was experienced as familial and non-hazardous, but the participants doubt that it would be that easy to cooperate with strangers in such a situation. Thus, a suggestion was to nominate a person in the bus, who should take the responsibility in such a situation, e.g., the person next to the intercom.

  • Shift in responsibility The participants also provided an explanation why passengers of autonomous shuttles might react reserved, as it is a fundamental shift in the common hierarchy of responsibility. The participants were asked to overtake responsibility for the undisturbed operation of the shuttle and are asked to act, which is currently very unusual when using public transport. (“If nobody in the bus has the feeling to be competent, then more passengers won’t feel addressed.”)

The following suggestions for improvement were given:

  • Audio information The requirements referring to the audio information resp. conversation with the chat bot were that the volume should be higher, and the provided information should be more precise, e.g., “Please press the emergency button on the intercom to contact the control centre.”, instead of “Please contact the control centre via the intercom.”)

  • Textual information In general, the willingness to follow the instructions of the control centre was high, but as already stated for incident 1 the combination of audio information and textual information was suggested as an improvement.

7.1.3 Scenario 3: potential threats

In order to collect requirements for exceptional conditions, three different situations were presented to and discussed with the participants (harassment, dirt, and catch the bus).

The findings of the discussions show that in general participants had the same requirements and expectations as they would have in current public transport, especially the underground, where the driver also is not present.

With respect to harassment, the following information was given to participants: “You are sitting alone in the bus; another person enters bus and sits down next to you and comes very close. You feel uncomfortable.” Participants’ feedback was given to the following aspects:

  • CCTV In case of harassment, the use of CCTV is considered helpful, as it strengthens the individual’s feeling of safety. Some of the participants attribute a deterrent effect to CCTV and expect that CCTV would immediately support them if one is confronted with harassment.

  • Mobile phones or emergency buttons These were considered helpful devices resp. functions in an exceptional situation, as their handling is well known.

  • Getting off the bus The wish for a possibility to get off the bus was a common request in all test groups.

With respect to the use case dirt, participants were told: “You want to enter the bus, but it is very dirty. A sticky fluid is on the floor and it has a strong smell.” Participants gave the following responses:

  • Interaction via intercom The participants agreed on the requirement that in case of a dirty bus it should be possible to contact the operating company already at the bus station, e.g., via intercom, and notify them that the bus is dirty.

  • Features of an autonomous bus station The requirements are quite similar to requirements for common public transport stations, e.g., weather protection, info displays, rubbish bin, but additional an intercom is suggested and plug sockets, as the supply with infrastructure for charging mobile phones is considered as getting more and more important.

For the use case catch the bus, participants were told: “You are in the bus, which is already leaving the station, when you see a person running after the bus trying to catch it.” Participants responded with the following:

  • Regular line traffic vs. on demand In regular line operation, the same procedure should be applied as it is common in Austria now. If the doors are closed and the bus drives on, the bus should not stop anymore. An exception would be on demand traffic, where further options for prolonging the stop at a station could be integrated. The arguments of the participants for not stopping the bus are that the bus must follow a schedule and that further buses are available.

  • Exceptional conditions Depending on the situation, the participants agreed on some exceptions. Late at night, if this would be the last bus or if there are very long intervals, the option to stop the bus would be helpful. As a possible solution, one of the participants suggested an application, where the waiting time can be prolonged for money.

7.2 Study 2

In study 2 in a real traffic environment, only the incident management for the second scenario (obstacle hinders the bus to drive on) was tested. The provided audio information was adapted based on the findings of the test at the test track.

Based on the video recordings of the rides of the three test groups, we coded which strategies the participants applied to manage the incident. In the final discussion, the participants were asked how they experienced the incident, if the acoustic information was helpful to manage the incident and which improvements they would suggest. To structure the findings, in the following: the applied strategies, the supportive, and the hindering conditions as well as the suggestions for improvement are presented.

All 3 test groups did react on the acoustic information, but the applied strategies to manage the incident as well as the range of actions set differed. Two of the three test groups applied a strategy which can be characterized as active and cooperative. The participants agreed on the definition that they are confronted with a test task and coordinated their further actions. The applied strategy of the third test group can be characterized as passive awaiting. The test persons were uncertain about the situation and they did not jointly define the incident as a test situation.

The following conditions were regarded as supportive:

  • Relationship between participants Some of the participants know each other, so cooperative action was supported. As stated before, the initiation of cooperation with strangers was assessed as difficult.

  • Audio information The provided audio information not only triggered the participants to define the incident situation as a test situation, but also was assessed as credible and helpful. The audio information was experienced as clear and accepted as a guidance for action.

  • Attention level and readiness to overtake responsibility The participants of two groups were attentive to the surroundings and the bus during the ride (e.g., looked out of the window and observed overtaking cars, searched for information on the display). These participants felt responsible for resolving the incident. The participants of the third group, on the contrary, paid hardly to no attention to the surroundings, talked busily with each other, and during the incident did not take any action and did not feel responsible.

The following conditions were regarded as hindering:

  • Ambiguity of audio information The instruction for action should be clearer: “Press the green button to open the doors,” instead of “open the doors.” Some participants stated that the wording was irritating, as they associated big troubles with the term “incident” and felt rather discouraged at the first moment to set any action to resolve the incident.

  • Seriousness of audio information unclear Related to the aspect of ambiguity is the fact that the provided information was not clearly discernible as a relevant and serious information for all passengers. A kind of sound signature or marker for important audio information was recommended, e.g., “Attention please, or beeping.”

  • Additional textual information There were no further information, besides the audio information in the bus how to manage the incident. The repetition of the audio information was not considered helpful, but as even more overstraining.

  • Technical failure The connection to the control center was disturbed during one of the test groups, which hindered the participants to develop an alternative strategy to manage the incident.

  • Missing agreement among participants Group dynamics are also an important aspect when analyzing incident management. In one of the test groups, the participant, which tried to set any action to manage the situation, was too uncertain to take any action without the affirmation of the other persons.

  • Shift of responsibility As already stated, the request to intervene was irritating for some of the participants, they did not expect to be confronted with such a task and considered this as the duty of the operator: Further some of the participants felt unable to cope with the situation, as they had the feeling not to have the competences to deal with a problem in a high-tech bus (“What if everybody presses another button, this is certainly adverse.” “I thought, that I am the passenger and I won’t get out of the bus.”).

  • Rigid compliance to instructions The participants were instructed before the test and asked not to press the emergency buttons for the operator. Thus, the audio information to contact the control center was not accepted as a task from all participants, as they wanted to comply with the rules and not touch any of the buttons in the bus.

The following suggestions for improvement were given:

  • Nomination of person in response As already stated in the discussion on the test track, the discussion in the real environment test confirmed that a crucial aspect is the decision who should set the first action. The phenomenon of diffusion of responsibility in the view of the participants could be to nominate a specific person, e.g., “The person on the left by the door.”

  • Additional features in the bus requested were (1) a central, more visible position of the intercom, so that each passenger can see it easily; (2) the integration of the function, that in emergency cases the passengers can directly contact the police or the ambulance; (3) additional cameras outside the vehicle, so that the control center also can see what is going on around the bus; (4) additional touchscreen for additional information retrieval; (5) additional light signal on the bus to attract the attention of persons, who are distracted or wear headphones.

7.2.1 Perceived safety questionnaire

Six participants out of nine agreed or fully agreed to the statement that they felt relaxed during the ride on a 6-point scale ranging from fully agree to not agree at all. Two participants felt rather relaxed and one person was not relaxed at all. Four participants, also, fully agreed that they felt safe during the bus ride, while two agreed and three rather agreed to that statement. Only, one participant did not feel safe while in the bus. Also, three out of nine participants rather agreed to not feeling in control during the ride. One person fully agreed, one rather not agreed, and two not agreed or not agreed at all to that statement. To summarize, half of the participants felt rather in control, while the other half did rather not. Seven participants felt not nervous (at all) during the ride. One participant felt totally nervous and another at least nervous. Eight participants did not agree to the statement, that they wanted to get out of the bus during the ride. One rather agreed, though. Also, one participant out of nine would not want to take a ride in an autonomous bus again, while seven definitely would and one rather would.

When asked if the the pre-recorded bus announcements assured them of having received all necessary information, although no bus driver was present during the ride, five out of ten participants fully agreed to that, three agreed, and two rather agreed. No participant did not agree (at all). When asked if the the pre-recorded bus announcements assured them of knowing what to do in the case of an incident with no bus driver being present, four participants fully agreed, three agreed, two rather agreed, and one rather not agreed.

7.2.2 Perceived quality of voice experience questionnaire

Participants were also asked via two items on a 6-point scale regarding the quality of the pre-recorded in-bus announcements. Item 1 asked whether the voice was clearly audible; item 2 whether the voice sounded as if a real person was talking to them. Eight out of nine participants fully agreed with item 1, with only one participant not agreeing at all. Thus, perceived sound quality was rated very good overall. However, for item 2, three participants out of nine did not agree at all and three did not agree. Only two rather agreed and one fully agreed. These results suggest that the voice interaction alone might not sufficiently compensate for no human driver being present.

8 Discussion

In the following, we discuss the findings with respect to identified passenger needs and expectations regarding information provision and intervention possibilities in the defined scenarios (RQ1) as well as passengers’ experiences with resolving incidents in specific situations (RQ2).

8.1 Diffusion of responsibility and precision of information

While scenario 2 was resolved successfully in every case but one, it should not be expected that this quota is representative for all such situations in a real deployment. Diffusion of responsibility and a resulting unresponsiveness to related messages are potential issues requiring appropriate design solutions. Responsibility needs to be first appointed and then also accepted by the individual(s) potentially feeling responsible. In order for this to work, instructions can be directed better, the shorter and more concise they are. This is also something that worked better in study 2, where only very few and concise instructions were communicated. Once one person felt responsible and started to move, others would start to move as well. Thus, as long as one individual in the bus can be reached, diffusion of responsibility can be countered.

It is likely that an in-time solution alone, which apportions responsibility once an incident has occurred, is not sufficient. Adequate information regarding responsibility and expected behavior in case of an incident should also be provided before an incident can occur, not dissimilar to emergency instructions in an airplane. After all, a train passenger can expect to never be asked to do an engine or rail track inspection after having boarded a train. This should be the same for automated buses. In addition, not every passenger is a healthy, middle-aged individual fluent in the local language. Thus, different problem solution capabilities on the parts of the passengers are to be expected.

It should be kept in mind at this point that the goal of automation should not be to put the burden of compensating for automation failures on the passenger. This is not the point of this investigation and would arguably miss the point of automation technology altogether. However, there is a difference between, e.g., a bus stopping due to engine failure versus an inattentive teenager immersed in her/his smartphone blocking the bus’ path. The former requires professional intervention, the latter not necessarily.

8.2 Appropriate cueing

Directly related to how diffusion of responsibility might be resolved is the issue of appropriate cueing. As it turned out, the fact that, from their auditory qualities alone, incident-related messages were indistinguishable from regular announcements was a contributing factor to passengers paying less attention in case of an incident than they might have otherwise. Auditory cueing before announcements is not a novel concept and is nowadays standard, e.g., in trains, supermarkets, airports, and similar contexts. However, such cues are usually uniform, which means there is one cue with the sole function of signifying that information content is to follow but without specifying the type of content.

Brown and Laurier [22] suggest using initial motions to cue the driver of a level 3+ vehicle toward the intended maneuver before it is executed. While this cannot be directly applied to the automated shuttle context, as there is no driver and neither are most situations related to specific driving maneuvers, the same channel that is already being used for passenger interaction. Thus, if the primary information channel for non-emergency information used in the vehicle is the auditory one, then cueing should be done via the same channel.

While this may sound trivial in itself, there is further differentiation necessary for the individual cues and the situations they relate to. Different cues should relate to different types of content and be consistent with standard communication (i.e., not incident-related) inside the bus. Regular announcements should be clearly distinguishable from nonregular announcements before they occur, so the passengers can appropriately adjust their attention beforehand. These need not be limited to auditory cues either. Ambient lighting to visualize the bus status (driving, planned stop, unplanned stop, approaching stop, about to start driving, etc.) can similarly be used to direct passengers’ attention to incoming situation-relevant information.

8.3 Open door policy?

An interesting point, which came up several times across all studies and scenarios, was related to the bus doors. Quite understandably, the participants expressed a desire to be able to exit the bus at any time when confronted with incident situations. While it would be difficult to argue for why passengers would want to be trapped inside an automated vehicle, there is the valid question of whether it is really the best solution to have the doors freely able to be opened at all times.

With intended use comes unintended use and a door that can be opened at all times can be opened while the bus is driving as well. This might not be that big of an issue if the bus has just departed or if it is an urban shuttle gently driving along in an otherwise pedestrian zone. However, things look different when the vehicle is driving on a country road going 60 km/h, where both boarding and departure would be neither safe nor sensible. Thus, an automated vehicle will need to be able to detect the driving environment also with regard to passenger safety and not just what is necessary to execute driving maneuvers. At the very least, the vehicle should provide recommendations on whether it is safe to leave the vehicle and which potential hazards could be expected if they do so (or whether they should stay in the vehicle and that responders are on the way). Even then, there is no driver who would, in case of an accident, take safety precautions so no further accidents occur. As one participant in study 1 mentioned: “A [human] bus driver would simply put up a warning triangle but here you just have this bus standing in the middle of the road. Maybe it should have a display, just like the police cars do sometimes, that says ‘Warning, accident ahead!’ or something similar.”

But then there are not just accidents but also other situations of potential danger to passengers. If someone is trying to, e.g., rob a passenger in the bus, then one of the safest measures would be to make it as easy as possible for the passenger to escape. On the other hand, imagine a situation where a vulnerable individual inside the bus is approaching a bus stop, alone and at night, and it is clear that the unsavory-looking individuals waiting at the stop are just waiting for their victim to arrive—after all, there is no human driver present and no way for the passenger to keep the doors from opening [24]. As one participant asked “Can I lock myself inside the bus, so nobody can come inside?” A valid question, which is unlikely to have a single answer that works for all contexts.

8.4 The “lazy” passenger

On a more general level, another finding of our studies is that we might need to rethink the nature of riding in a bus, as the human’s role in it might change with the introduction of automated buses. Riding in a bus with a driver ensures that I am the passenger and play only a very passive role. I can operate my smartphone or just listen to music and enjoy the environment. The driver will solve any problems and if there is anything I am required to do I am told to do so by another person (e.g., when showing my ticket). It is not required to take any other action nor to take over responsibilities save for what is necessary to get on and off the bus at my desired stops. This is how we are socialized given how public transportation has worked for decades.

However, unless the ideal goal of flawlessly working fully automated public transportation is realized, riding in an automated bus might require active involvement from passengers at key points or instances. But this is—and rightfully so—not how bus riding is seen today. A shift in expectations of what a bus ride in an automated bus is needed. And paradoxically enough, this could mean higher potential for passenger involvement when the bus is fully automated. If there is the possibility that passengers have to take over responsibilities, this has to be communicated in advance—so far in advance, in fact, that it is part of their expectations before boarding the bus, so that it is part of their informed choice of which means of transport to use. A clear transparency of what might or might not be required from passengers is needed, otherwise user experience in automated buses might could a bumpier road ahead than expected.

9 Limitations

Due to legal requirements, it is not yet possible to operate vehicles of SAE level 4 and above in Austria. Regardless of vehicle automation level, each vehicle must have a designated human driver who is capable of interventions and responsible in case of incidents (in this case, this role was taken by the operator). For the user studies, this meant that it was not possible to simulate an environment that is fully identical to a future level 4 or 5 automation scenario, as the operator was always present and could not be disguised as a passenger, due to semi-regular interaction with the operator control unit in the bus, which was visible to the passengers. In order to simulate full automation as best as possible, the passengers were instructed to ignore the operator and the operator was instructed to not communicate with the passengers until a trial had been concluded. Still, it should be expected that any observed effects might be different (and potentially stronger) in a vehicle without a human operator present in the vehicle.

With respect to external validity and experiencing scenarios in real life, we had to experience some limitations. The choice to design the threat scenarios as interactive discussions was primarily made for reasons of safety and feasibility. The more realistic such a threat is to a participant, the higher the likelihood that said participant would respond in an unexpected manner. This presents a risk to both the researchers and other participants. For the same reason, ethical approval (which we did have for the presented study) for such a procedure would have been difficult to obtain as well. Presenting these scenarios in a realistic way also requires appropriate immersion of the participants, potentially requiring conduction of the study during nighttimes, appropriate acting skills on part of the researcher, and other such factors. If, at any point during the study, the immersion is lost, then so is the external validity. Apart from this, the “missed the bus” scenario is simply very hard to time properly, so that the participants can neither leisurely stroll to the bus and easily board it nor miss it before they have even a chance to reach it. The secondary task before boarding needs to be carefully calibrated, so as to not distract the participants too much, while, at the same time, not appear to specifically set them up just to catch the bus and nothing else, or external validity is lost once again.

A contextual discussion, while starting from a lower level of external validity, does not run these risks to the same degree. By having the discussion in the bus as it is driving, we attempted to provide a safe minimal degree of immersion, as the participants would articulate their points from their passenger role in that moment. Our methodological choice has lowered the external validity in comparison with more realistic studies.

The number of participants for both studies (N = 13 and N = 11) is not sufficient to draw meaningful quantitative conclusions. Thus, quantitative means were used sparingly and intended as a basis for the qualitative results. As with any qualitative study, the focus is on identifying and attempting to explain individual phenomena, while its results might or might not hold on a larger scale and across different demographics. Investigation of perception and management of incidents in automated public transport on a larger scale is, therefore, subject to further research, which is planned to continue in early 2020.

10 Conclusion

In this paper, we presented two studies that investigated the problem of passenger information and interaction in a driverless shuttle bus when the bus faces an incident. We found a number of factors surrounding the aspects Diffusion of responsibility, proper announcement/interaction cueing, and door behavior. We found that diffusion of responsibility can be an issue and can effectively halt passenger interaction with the interface(s) unless resolved. Clear and focused instructions aimed toward the passengers are needed when passengers have to take actions. Regular announcements and information about incidents need to be clearly distinguishable from one another via appropriate cues.

In more concrete terms, we found and suggest that in case of an incident, audio information about the incident is needed. This information needs to be repeated at least every 5 min. Additionally, textual information about the duration of the interruption and the possibility to get further information need to be provided. For longer interruptions, a communication with a remote operator is needed via an intercom. Passengers need to have the possibility to exit the bus if desired, although it will need to be handled on a context-sensitive basis in order to ensure a good balance between passenger freedom and safety. CCTV is helpful to increase perceived safety but is subject to potential privacy issues.

Information on the purpose and consequences of emergency and information buttons need to be very precise. For example, does pressing the emergency button alarm the police/ambulance or the control center? The usage of a chatbot in an emergency situation is not recommended, since it might lead to delays. If a chatbot is used, then the volume needs to be such so that it is audible for all passengers. Allocation of responsibility should be done explicitly (e.g., “The person next to the intercom”). A distinction between traditional voice information (e.g., estimated time of arrival) and information form serious information can be achieved by a sound signature or auditory icons.

While it should not be expected that the human passenger in an automated bus is a reliable fallback strategy in case of incidents, incidents can and will occur and appropriate strategies need to be devised, when a human driver is no longer present.