INTRODUCTION

On September 16, 1620, a group of 135 English Puritans known today as the Pilgrims in the USA left Plymouth onboard the Mayflower galleon to reach New England after a 66-day voyage across the Atlantic, found the settlement named after their native city, and obtain the absolute freedom they sought [1].

This day after 400 years, on September 16, 2020, during a special ceremony in Plymouth, UK, the name Mayflower was inherited by a trimaran (Fig. 1) designed by marine research organization ProMare, USA, in cooperation with IBM UK and Wärtsilä shipbuilding company, Finland [2]. It was planned to follow the path of its predecessor to the American Plymouth, but the coronavirus pandemic put these plans on hold. Now the launch of the Mayflower is scheduled for April 2021.

Fig.1.
figure 1

Mayflower trimaran.

Full size image

There would be nothing special about it if it were not for one factor: the ship will have no onboard crew, neither will it be controlled remotely! That would be 3200 nm of autonomous navigation. Andy Stanford-Clark (IBM UK) compared this voyage with the major technological achievements of the mankind over the last 50 years: “IBM helped mankind land on the Moon and is excited by the challenge of using advanced technologies to cross and research oceans”. The ship is equipped with the radar, LIDAR, Automatic Identification System (AIS), six video cameras and highly effective solar batteries in addition to the standard navigation equipment. The Mayflower will be controlled by the virtual AI Captain taking the decisions based on IBM supercomputer data using artificial intelligence (AI), computer vision, cloud and edge computations [3].

To dare to perform the breakthrough transatlantic voyage, at the first testing stage the trimaran equipment was tested at the departure point by the Plymouth University team using the deep learning model developed by IBM. The main objective was to enable the integrated system to adequately assess the environment to prevent collisions with surrounding objects with the relevant route optimization. At the next stage for almost 2 years the Mayflower software had been tested for compliance with the requirements of the International Regulations for Preventing Collisions at Sea (COLREGs) and the International Convention for the Safety of Life at Sea (SOLAS) imposed by International Maritime Organization (IMO), without which actual autonomous navigation would have been impossible [4].

The unmanned ships (UMS) are often believed to become a panacea for the navigation safety, the major problem in marine shipping. Why? By the early 2000s, a paradox situation arouse. Along with AIS, the ships started to be equipped with GNSS receivers, Electronic Chart Display Information Systems (ECDIS), Automatic Radar Plotting Aids (ARPA), Integrated Bridge Systems (IBS), Vessel Tracking Management System (VTMS), and Global Maritime Distress Signal System (GMDSS), but alas! In 2000–2005 the UK Department for Transport Marine Accident Investigation Branch revealed that marine accident rate barely decreased despite these improvements [5].

Not much has changed since that time. According to Allianz Global Corporate & Speciality, the world leader in risk insurance, the human factor is the major reason of accidents [6]. As estimated by the company, the human errors affected some 75–96% of insurance events registered in marine shipping. Moreover, the analysis of about 15 000 accidents occurring in 2011–2016 and leading to almost $1.5 bln of losses has revealed that 75% of accidents were caused by the human factor.

Finally, the problem of ship manning is quite acute. BIMCO, a nongovernmental association focused on shipping policy and uniting ship owners and brokers, predicts a shortage of 147500 seafarers by 2025, caused by ageing of the ship crews and youngsters’ reluctance to go to sea.

These facts, though not exhausting the reasons for using autonomous ships, have led to a forecast by ReportLinker predicting the growth of UMS market from $13.2 bln in 2019 to 21.7 bln in 2025 [1].

It should be noted that here and below the paper does not focus on autonomous underwater vehicle technology, which deserves a special survey due to its specific features.

DEVELOPMENT OF UMS TECHNOLOGY IN EUROPE

In Europe, the first notable results in UMS technology development have been received in MUNIN project (Maritime Unmanned Navigation through Intelligence in Networks) with a budget of EUR 3.8 mln. It was realized in 2012–2015 by nine companies headed by Fraunhofer Center for Maritime Logistics and Services and supported by the European Commission [7].

This pioneering project has developed a concept for the operation of unmanned ships and assessed its technical, economic and legal feasibility on the assumption that a ship would be completely unmanned at least for parts of the voyage. In those years a constant autonomous mode was a remote prospect. According to the research results, the following systems should be developed to operate the autonomous ships [8]:

(1) a navigation sensor module fusing sensor data including radar and AIS, combined with daylight and infrared camera imagery;

(2) a navigation system for shipping in deep seas, which follows a predefined voyage plan and adjusts the route to arising encounter situations and significant changes in weather;

(3) an autonomous engine and monitoring control system, able to detect, or even predetect possible failures in a propulsion system, thus keeping its optimal efficiency;

(4) a shore control center with a team of skilled nautical officers and engineers, which continuously remotely controls the ship at the self-navigating parts of the voyage (the crew leaves the ship when it goes to the sea and returns to the ship using the helicopter when approaching an intense navigation area or a port).

For a bulk carrier on an intercontinental line mostly away from the intensive shipping routes, a 25‑year use of an autonomous ship would save $7 mln compared to a conventional manned bulker. The collision probability for such a ship is shown to reduce tenfold compared to a manned ship, firstly because the crew fatigue factor is eliminated.

Transfer of navigation and operation control to the shore team in UMS remote control mode would also mitigate the second problem mentioned above, potentially attracting an influx of young crew.

MUNIN ideas were further developed in 2015–17 in AAWA project (Advanced Autonomous Waterborne Applications Initiative), which received EUR 6.6 mln from the Finnish Funding Agency for Technology and Innovation [9]. The project led by Rolls-Royce brought together the maritime cluster of DNV GL, NAPA, Deltamarin, Inmarsat, and the number of universities (Tampere University of Technology, Aalto University, the University of Turku, and Åbo Akademi University), and VTT Technical Research Center of Finland Ltd. The ferry operator Finferries and ESL shipping, the leading carrier of dry bulk cargoes in the Baltic, also participated in the project. Finferries provided its vessel, Stella, operating between Korpo and Houtskär for the tests to confirm the suitability of the technological solutions selected by the developers.

The project developers searched for answers for the following main questions [10]:

(1) what technology is needed and how it can be best combined to allow a UMS to operate autonomously miles from ashore;

(2) how can a UMS be made at least as safe as existing ships, what new risks can it face and how can they be mitigated;

(3) what will be the incentive for ship owners and investors to invest in UMS and who is liable in the event of an accident.

Focus on the first two questions. First, we need to understand what is meant by autonomous. There exist a number of definitions of automation levels characterizing the computer ability to perform actions, with T. Sheridan’s concept being the most popular [11]. He proposed 10 levels, where at the first level the computer does not assist the human performing all the operations, whereas at the last level they change their roles. AAWA revealed that the automation levels differ depending on the type of vessel and specific performance. According to the developers, in some cases the ship can be operated almost automatically (mainly in the open sea), whereas in the others it must be remotely controlled by an operator. Operating a large container carrier and a small-sized bulker makes a great difference.

UMS navigation architecture was proposed, including [10]:

—a propulsion control system;

—dynamic positioning system (DPS);

—autonomous navigation system (ANS);

—situation awareness system (SAS) containing all the required sensors and fusing their data;

—data link systems;

—systems functioning as a remote operator.

The ANS, in its turn, comprises a route planning module (responsible for preliminary route plotting only), collision avoidance module, situation awareness module (making decisions based on SAS data), and ship state definition module. The most important is the last module, or a Virtual Captain, which combines information from different ANS subsystems as well as from DPS and other systems. This module determines the allowed ship operation mode and continuously provides the raw data for the remote operator.

When combined with GNSS, the wind sensors and inertial measurement units (IMUs), DPS would be able to keep its position even in rough navigation conditions. The DPS should also provide collision avoidance and planning the current route part.

The developers have thoroughly studied the sensors enabling autonomous navigation when combined with conventional systems such as AIS, ECDIS, GNSS, and ARPA, as well as with IMUs never been applied onboard IMO vessels. Here the key sensors are daylight and IR cameras, radars and LIDARs.

The cameras offer the advantage of separating the relevant objects from the background of sea surface using color information. A configuration of two cameras gives a limited 3D sensing. This would require extensive processing performance and high-bandwidth data links, however, this is not a problem already. Among IR cameras, the most practical are those operating in 8–14 μm range and able to work in total darkness. Thermal imaging can be beneficial even in daylight conditions in heavy fog or rain, when daylight cameras fail.

The cameras’ disadvantages include their insufficient weather tolerance and inability to extract object distance information, which is done by radar and LIDAR. The traditional marine radars using S- and X-bands have been found to feature insufficient resolution for UMS operation in near-port area, so Ka- and W-band automotive radars should be applied. As to the LIDAR technology, it was demonstrated to be effective in marine applications, despite being weather tolerant, by Jimenez as early as in 2009.

The optimal use of existing communication technologies for data transfer between the ship and the shore was studied with a focus on cybersecurity, which is critical for UMS [12]. A simulated UMS control system was designed to analyze the data exchange via satellite communication link with account for video data compression and selection depending on the system mode.

Since 2014, while participating in AAWA, Rolls-Royce has simultaneously been working on designing cargo unmanned ships [26]. It was planned to remove the bridge, crew cabins, water supply, sewage, conditioning systems, and water supply systems from the container carriers. It would allow a 5% reduction in weight and a 15% reduction in fuel consumption. Unmanned ships would also reduce the crew costs during the voyage (about $3300 per day for a large container ship, which is 44% of a total of operational costs of marine cargo shipping).

AAWA ideas were practically confirmed by Rolls-Royce and Finferries in SVAN project (Safer Vessel with Autonomous Navigation) completed in 2018 [14, 15]. The developers have tested “the first fully autonomous ferry in the world”, a 54 m-long car ferry Falco constructed in 1998 (Fig 2).

Fig. 2.
figure 2

Ferry Falco.

Full size image

A completely automatic test voyage was conducted for almost 400 h in the archipelago south of the city of Turku, Finland. The vessel successfully conducted collision avoidance and demonstrated automatic berthing. The ferry is equipped with a range of sensors which allows it to build a picture of its surroundings with a level of accuracy beyond that of the human eye. The situational awareness picture was relayed to Finferries’ remote operating centre on land, some 50 kilometres away in Turku, where the operator could take control of the vessel if necessary.

When asked about the time perspective of full automation of ferry navigation in the test area, Rolls-Royce and Finferries answered that it would require from 5 to 15 years.

If Rolls-Royce strives to become the world pioneer in fully autonomous ferries, Wärtsilä wins the championship in the first automatic docking [15]. The tests were conducted onboard a ferry Folgefonn owned by Norled, Norway, in January−April 2018. The ship positioning was performed by the Wärtsilä’s DPS with differential GPS used as the primary sensor. In case of GPS signal outage, a Wärtsilä Guidance Marine CyScan AS based on LIDAR data is being used.

Simultaneously with AAWA, the AUTOSEA project focusing on UMS collision avoidance was developed. It was realized by the Department of Engineering Cybernetics, Norwegian University of Science and Technology, and Norway-based companies Kongsberg Maritime, Maritime Robotic and DNV GL, and sponsored by the Norway Research Council [16–18]. Collision avoidance solution in compliance with the COLREGs utilized data from the radars, AIS, LIDAR, IMUs, daylight and IR cameras, and GPS. The collision avoidance system (CAS) was initially developed based on state of the art in target tracking and guidance/control.

CAS sensor fusion was implemented based on PDA (Probabilistic Data Association) tracking, where the probability of adequacy of data on the meeting ship is calculated. The CAS solution was based on model-predictive control, where the collision risk assessments are included in the relevant cost function. Uncertainties in the ships’ relative positioning, sensor data, CAS actuators, and intention of the meeting ship were accounted for. On completion of the development, CAS tests in the waters of Norway and Netherlands have confirmed its high performance.

The AUTOSEA project was further developed by the Hull-to-Hull (H2H) project financed by the European Union since November 2017. The project is run by Kongsberg Seatex, Sintef Ocean and Sintef Digital (Norway), Mampaey Offshore Industries (the Nertherlands), and KU Lueven (Belgium) [19].

The overall objective of the project is to address the need to safely navigate in close proximity to other stationary or moving objects, thus the project outcomes can be applied to support UMS technology. The problem is solved by fusion of various navigation sensors, mainly GNSS Galileo and the regional satellite-based augmentation system EGNOS and using 3D models of certain unmanned ships with the vessel main dimensions.

This approach allows precision estimation of distance to the surrounding objects as well as the approach speed. According to the estimates, for UMS the object distance error should not exceed several dm, which is going to be achieved by fusing data from multisystem dual-frequency GNSS receiver, IMU, AIS, LIDAR, radar, and video cameras. Moreover, sensor data and 3D ship model will be exchanged between the ships.

The project was generally led by Kongsberg Seatex. Sintef Ocean and Sintef Digital appraised the decisions, the university KU Lueven estimated them in terms of inland navigation, and Mampaey Offshore Industries focused on tugging, mooring, and docking. The project coordinator Per Erik Kvam supposes that on completion of H2H nonstandard maneuvers and procedures, however, complying with COLREGs will become feasible.

The project developers pay special attention to autonomous navigation. If UMS were recommended for commercial use, extremely strict reliability and safety requirements would be imposed on their equipment and technology. This would require constant data exchange between the interacting ships via a fault-free high-speed data link supporting relative positioning and exchange of 3D-models. The link is also being developed within the project H2H.

Once the project is completed, demonstration testing is planned in Norway (interaction with a conventional ship), in the Netherlands (docking), and in Belgium (ship positioning in inland waters by Galileo and EGNOS under various external conditions).

Realization of the above initiatives has lastly motivated IMO to focus on UMS development problems. A MASS (Maritime Autonomous Surface Ship) has been defined as a ship which, to a varying degree, can operate independently of human interaction [20].

The significant IMO’s Maritime Safety Committee’s (MSC) 100th session in December 2018 was focused on UMS [21]. For each navigation instrument provisions were identified which

—apply to MASS or have no application to MASS operations;

—apply to MASS but may need to be amended or clarified;

—apply to MASS and require no actions.

Four degrees of autonomy have been identified, which solves the problem faced by AAWA developers:

—Degree one relates to a ship with automated processes and decision support, where seafarers are on board to operate and control shipboard systems and functions. Some operations may be automated.

—Degree two covers a remotely controlled ship with seafarers on board. Seafarers are available to take control if required.

—Degree three is a remotely controlled ship without seafarers on board.

—Degree four is a fully autonomous ship.

However, an opinion was expressed that degrees three and four would require the AI advancement that is unreachable so far.

The same problem was addressed at the 101st session of the MSC in June 2019. It considered the MASS trials and decided that they should be conducted in a manner that provides at least the same level of required navigation characteristics as those relating to manned vessel operations [22].

In 2019 the European Union launched two new initiatives focused on UMS coastwise navigation and cargo transportation in inland water ways, which is surely more demanding than open sea navigation. The first, AUTOSHIP (Autonomous Shipping Initiative for European Waters) is scheduled to be completed in 2022 with a total project cost EUR 29.5 mln. The project is led by Kongsberg accompanied by Ciaotech Srl (Italy), Blue Line Logistics (Belgium), Sintef Ocean AS (Norway), Upm-Kymmeue Oyj (Finland) and University of Strathclyde (UK) [23, 24].

The project will build two different vessels and conduct trials to demonstrate their operative capabilities, including reduction of costs and delivery terms, making them more competitive to replace road transport. It is also planned to reduce fuel consumption and simplify logistic procedures, and to provide the cybersecurity of navigation.

The second project is AEGIS (Advanced, Efficient and Green Intermodal Systems) with a EUR 7.5 mln budget run for three years by a consortium of 12 companies from Norway, Denmark, Finland, and Germany led by Sintef Ocean AS, which is also involved in AUTOSHIP [25]. The project will also design autonomous ship technologies for coastwise and intermodal navigation while reducing both noise and dust pollution in river navigation, particularly in the ports. The goal is actually a shift in cargo handling from road and rail transport to waterborne transport using small ships preferably operated by electric propulsion, and automation of port and terminal operations.

The latest achievements of Kongsberg that swallowed Rolls-Royce Commercial Marine in April 2019 and became the European (if not the worldwide) leader in unmanned navigation are as follows. In February 2020, Kongsberg Maritime, the Norwegian Maritime Authority and the shipping company Bastø Fosen announced that the first fully automatic adaptive ferry Bastø Fosen VI was ready for Horten-Moss line service. The vessel fully loaded with passengers and vehicles first demonstrated the automatic voyage from dock to dock [27]. However, the crew is still carried onboard. The ferry control system enables timely arrival at the destination point, which is a critical problem posed recently by IMO. During trials in December 2019, the Bastø Fosen VI arrived within two seconds of the scheduled time. The only one navigation function is not automated yet: if vessels are detected on a collision course, captain will take control and select the avoidance maneuver.

Another interesting initiative is an autonomous container ship Yara Birkeland (Fig. 3) with electric propulsion developed by Kongsberg with Yara as the customer, designer Marin Teknikk (both, Norway), and shipbuilder VARD (Romania) [28, 29].

Fig. 3.
figure 3

Container ship Yara Birkeland.

Full size image

The ship featuring a length of 80 m, service speed 6–7 kt (max 15 kt) and cargo capacity 120 TEU is intended to carry cargo from Yara plant in Porsgrunn to the ports of Breivik, 7 nm, and Larvik. At the first phase of the project a detachable bridge with equipment will be implemented for the crew, and then (2022 was scheduled before the coronavirus era) the ship will be operated fully automatically. The project also develops automatic unloading equipment and loaders for cargo handling at the berth.

Despite Kongsberg expansion, new companies working for autonomous navigation are emerging in Europe. The tech start-up Captain AI, ship design company Rotortug, and maritime service provider KOTUG International demonstrated AI capabilities for automatic shipping during the Netherlands Forum for Smart Shipping in September 2020 [30, 31]. In the Rotterdam experiment the Rotortug tug Borkum sailed autonomously on the river Nieuwe Maas following the optimal route generated online by the Captain AI, demonstrating safe voyage in an extremely heavy-traffic port.

In all the initiatives above, the software for autonomous navigation was developed as applicable to a specific ship, while the Autoplan project run by a consortium of engineering companies, a shipyard and academia from Germany and Turkey sets a bit different objective [32]. The project launched in September 2020 is funded by the German Federal Ministry for Economic Affairs and Energy and the Scientific and Technological Research Council of Turkey. Autoplan aims to develop a unified intelligent navigation assistance system that could be used for autonomous operations of high-speed vessels (firstly, hovercraft ships).

AUTONOMOUS NAVIGATION IN ASIA AND PACIFIC

Let us now consider the development of autonomous navigation technology outside Europe, without an intention to cover all the initiatives in this field. We’ll focus on milestones only.

China is one of the leaders in developing the autonomous navigation technology. The most prominent recent event was the completion of trials of autonomous cargo ship Jin Dou Yun 0 Hao developed by Yunzhou Tech, Wuhan University of Technology, and China Classification Society in December 2019 [33].

This 13-m electrically driven vessel (Fig. 4) is designed to run at 8 kt and was 20% cheaper to construct than manned ships. During the trials it automatically carried a cargo from Dong Ao island, Guandong province to a pier of the Hong Kong-Zhuhai-Macao bridge.

Fig.4.
figure 4

Cargo ship Jin Dou Yun O Hao.

Full size image

In December 2019, Navigation Brilliance, a Qingdao-based company, placed an order at Yangfan Shipbuilding’s Qingdao Shipyard for the construction of an autonomous small-sized container ship Zhi Fei. The ship was developed by Navigation Brilliance in partnership with Dalian Maritime University and China Waterborne Transport Research Institute [34]. Earlier the company had conducted the comprehensive trials of a small-sized ship Zhi Teng thus receiving the required experience in UMS design. If Zhi Fei trials are completed successively, then Navigation Brilliance is intending to develop an autonomous container ship carrying up to 10 000 TEU.

Not only Rotterdam port authorities are working towards an autonomous tug. This direction is followed by the similar Singapore authorities: PSA Marine, a major tug owner (Singapore), has initiated testing of its harbour tug PSA Polaris [35]. The trials are conducted within the IntellTug project led by Wärtsilä. Along with the traditional navigation solutions, Wärtsilä has developed a new DPS enabling intuitive control of the tug motion and passing navigation unit data to the vessel’s propulsion system.

Japan also undertakes many efforts in the development of autonomous navigation technology, assuming that by 2040 50% of domestic ships will be unmanned, with the economic effect of unmanned ships estimated at approximately ¥1 trillion ($10 bln). The developers are also motivated by the fact that 56% of sailors in Japanese merchant fleet are 50 plus aged, and young people are reluctant to go on watch, according to data published in 2015 [36].

At the first stage the UMS initiative was run by the shipbuilding cluster [37]. In 2016 NYK Line, a bulk cargo shipping company, begun to develop automatic collision avoidance systems and the equipment fusing the navigation sensor data and elements of augmented reality for autonomous coastal ships. In 2018 NYK Line initiated the development of AI navigation assistance system optimizing the ship route for navigation in Japanese coastal waters, with the system tests scheduled for 2021.

Mitsui O.S.K. Lines, Ltd. (cargo shipping) announced in May 2017 that its joint project with Mitsui Engineering & Shipbuilding Co., Ltd. on the technological concept for autonomous ocean transport system was selected by Japan’s Ministry of Land, Infrastructure, Transportation and Tourism for its FY2017 Transportation Research and Technology Promotion Program. The research consortium of the project is comprised of MOL, the National Institute of Maritime, Port and Aviation Technology, Tokyo University of Marine Science and Technology, and a ship classification society Nippon Kaiji Kyokai [38]. The consortium members will first develop the technological concept setting a course toward development of the technology needed to realize UMS.

In 2018 the industry initiative was supported by the Ministry of Land, Infrastructure, Transport and Tourism that launched three pilot projects to demonstrate [37]

—a remotely controlled ship (led by NYK Line);

—an autonomously controlled ship (Oshima Shipbuilding));

—automatic docking (Mitsui Engineering & Shipbuilding Co).

The results appeared very soon. In April 2019, Mitsui & Co. began the development of an automatic navigation system for large merchant ships in collaboration with ST Engineering, a major engineering company in Singapore, and Lloyd’s Register of Shipping, the UK classification society. A test run was carried out in Singapore, one of the most congested ports in the world.

Even more impressive are the results obtained by Japan’s NYK Line, which has completed a two-day trial of a 70826-tonne (!) unmanned ship Iris Leader, sailing from Nagoya to Yokohama under the supervision of the crew [39]. This is claimed to be the first UMS trial conducted in full compliance with IMO’s MSC requirements adopted at its 101st session. During the trial, the performance of Sherpa System for Real ship (SSR) in actual sea conditions was monitored as it collected information on environmental conditions around the ship from existing navigational devices, calculated the collision risk, automatically determined the optimal routes and speeds that were safe and economical, and then automatically navigated the ship.

Lastly, Kansai Electric Power Co has partnered with e5 Lab Inc to develop the passenger pilotless zero-emission electric vessels to operate in the Kansai bay area in Japan. The vessels will be engineered in preparation for the Osaka/Kansai Expo, which is scheduled to take place in 2025 [40]. e5 Lab is a part of an e5 Consortium uniting 7 Japanese companies (including Mitsui OSK Lines and Mitsubishi Corporation) and planning to launch an all-electric zero-emission tanker with all systems powered by li-ion batteries by 2022.

It would be surprising if South Korea, the world shipbuiling leader, would have stayed on the sidelines of this trend. Indeed, in October 2020 Samsung Heavy Industries started the trials of an autonomous navigation technology (at first with a remote monitoring) in Geoje area [41]. In a test voyage, Samsung T-8 tug was navigated using Samsung Autonomous Ship technology 240 km away from the shore center to demonstrate automatic collision avoidance with ships and remote keeping the tug to predefined route. Samsung Autonomous Ship analyzes real-time transmissions from cameras, radar, AIS and GPS receiver. This test confirmed the technology’s capabilities for avoiding all mobile and static objects in a radius of 1 km from the Samsung T-8 using AI to optimize the predefined route without violating COLREGs.

FUNDAMENTAL ISSUES OF AUTONOMOUS NAVIGATION

All the above presented did not account for the fact that first, the class of unmanned ships has not yet been legalized in the world law, therefore it is not clear who would cover the losses in case of an accident and according to which norms (and since it is beyond the scope of Gyroscopy and Navigation it will not be discussed further). Second, UMS will navigate in waterways filled with conventional ships following COLREGs and operated by the navigators prone to the human factor.

This problem is discussed in [42], which poses the following obvious questions. Should then COLREGs be modified, and if no, are they sufficient to ensure safe autonomous navigation? Should the ships be marked to inform that they are unmanned?

The first question is caused by the fact that these regulations are written in a rather general manner to cover as many sea situations as possible, and can be interpreted quite differently by a software engineer writing a code for an autonomous navigation system.

Rule 2 regulating the responsibility for noncompliance with COLREGs requires that one needs to comply with the “ordinary practice of seamen” and allows noncompliance with the COLREGs “to avoid immediate danger”. In other words, if necessary, all regulations may be violated. And, which is principal, instead of specifying the ship avoidance conditions with figures and measures, the seafarers are proposed to apply “the ordinary practice of seamen” undefined in COLREGs.

The rule 15 reads: “When two … vessels are crossing so as to involve risk of collision, the vessel which has the other on her own starboard side shall keep out of the way and shall, if the circumstances of the case admit, avoid crossing ahead of the other vessel”. Naturally, the risk of collision can be estimated assuming constant heading and speed of the meeting ship, but we can never know its intentions. We only know that the risk of collision increases if its bearing stays constant for a long time.

Rule 16 defines how the way should be given to the other ship. “Every vessel … shall, as far as possible, take early and substantial action to keep well clear”. Obviously, for a software engineer the notion of early without nautical miles or time indication or substantial without the magnitude of heading change or speed change do not make any sense. Moreover, avoidance actions significantly depend on the circumstances. It is one thing to keep clear of a single ship in open sea, and a quite different to avoid the ships in the narrow congested Malacca Strait.

The next problem in autonomous navigation technology emerges because unmanned ships need to interact with conventional ships. While the former will analyze all the sensor data and statistically estimate the situation, the captain of the latter will utilize his cognitive capabilities relying rather on visual observations than on radar or chart data.

It follows from the above said that from the software viewpoint COLREGs should be specified with miles, angular seconds, and clock minutes. This would surely simplify the programming but would critically extend the regulations which however would fail to embrace all possible avoidance scenarios.

A candidate solution to the problem is to learn the system to efficiently maneuver during avoidance based on AIS data and deep modeling so that to comply with COLREGs. This machine learning based on AI might be rational right on board, thus improving the avoidance solution from year to year. However, IMO is unlikely to approve such a non-certified procedure onboard.

There is one more thing that matters. Obviously, the actions of UMS should be predictable and transparent for the crews of manned ships. People are prone to be anthropomorphic, i.e., attribute human traits, emotions, and intentions to non-human entities. Therefore the author of [42] supposes that UMS must have a special symbol displayed by AIS on ECNIS or radar. However, this is not a panacea. One could also suppose that if the autonomous navigation system ensures compliance with COLREGs, its actions would be completely predictable. Nevertheless, the paper shows that it is not the case in poor visibility conditions.

Here the key factor in avoidance is the need to know the intentions of the meeting ship, which applies both to manned and unmanned ships. Otherwise, COLREGs will hardly save the situation. It is the knowledge of the avoidance intentions of the meeting ship that allows correct interpretation of the avoidance regulations. What can help?

To start with, all the ships complying with SOLAS should transmit their position and motion parameters via AIS. Then they are easily tracked by the radar and follow a predefined route. A number of completed and ongoing IMO projects on e-navigation [43] such as EfficienSea, ACCSEAS, MonaLisa, SMART Navigation, SESAME, and STM analyze these routes and optimize their efficiency and safety. The route exchange helps to estimate the ship intentions long before it enters the zone regulated by COLREGs. Particularly, all the surrounding ships will be aware of the future turning points of the traffic participants.

To conclude, the following can be said. The key factor in interaction between the manned and unmanned ships is the predictable maneuvering of the latter. An AI system onboard a UMS will act smarter than a conventional navigator, will be able to look ahead, and therefore to behave in a manner unusual for a conventional navigator. Instead, it is desirable that the system behave in a human-like style.

This transparency of the system performance assumes that

—it informs the surrounding ships on its autonomous operation (using a special symbol);

—it exchanges the estimated situation (which ships are tracked and which are not);

—it transmits data on the maneuvers it is planning.

Only if the other navigators understand the actions of UMS, their peaceful coexistence will be possible.

Moreover, apart from COLREGs modification, at least the following documents should be revised [37]:

—The UN Convention on the Law of the Sea, determining the rights and responsibilities with respect to the ship navigation (as mentioned above) and not covering the notion of UMS ;

—The International Convention for the Safety of Life at Sea (SOLAS), setting safety standards in the construction, equipment and operation of ships, which should be significantly modified as applied to UMS;

—The International Convention on Standards of Training, Certification, and Watchkeeping for Seafarers, which seems not to cover UMS, however, it should be remembered that humans—and not robots – will commission and hand over the ships to the customers;

—The International Convention for the Prevention of Pollution from Ships, which should provide relevant limitations regarding sea pollution by UMS.

Interaction between the unmanned and conventional ships is discussed in [44]. First, it offers to use the proposal of IMO’s e-navigation concept for AIS to transmit the ship intended route to the surrounding vessels or their shore command centers, which would surely be a great benefit to the ship avoidance capabilities. Second, similar to Traffic Separation Scheme that had been designated in congested traffic water areas, the IMO should consider to designate ocean routes for UMS.

Concepts are a great thing, but studies discussing their practical implementation are also of great interest. It has been repeated above that AI is applied in a number of products, however, the developers do not specify the results they obtained. The paper by Glenn Wright [44], on the contrary, presents a sensor fusion solution providing the ship autonomous navigation based on the machine learning and deep learning artificial intelligence.

At the first step the paper considers the requirements to the set of navigation sensors providing the ship navigation without any human intervention and ensuring safe navigation regulated by COLREGs since 1972.

According to Rule 5, “Every vessel shall at all times maintain a proper lookout by sight and hearing as well as by all available means appropriate to the prevailing circumstances and conditions so as to make a full appraisal of the situation and of the risk of collision”. Simultaneously, proper use shall be made of radar equipment and radar plotting or equivalent systematic observations of detected objects, and assumptions shall not be made on the basis of scanty information (Rule 7).

Surely, these rules have been written for the ships operated by the crews who took decisions based on their perception and experience, and used data from the echo sounders, radars, ARPA, AIS, ECDIS, and GNSS. This is not sufficient for crewless ships.

For automated shipping the whole range of sensors should be utilized to characterize the condition of surface and underwater area, which greatly outperform the visual and hearing monitoring capabilities of the seafarers. It is also needed to observe the underwater situation at least ahead of the ship to avoid collisions with the objects not indicated on maps that might have been constructed dozens of years ago.

The paper proposes to apply the following equipment for autonomous shipping in addition to the conventional aids (gyro- and magnetic compass, log, GNSS receiver, etc.):

—IMUs;

—LIDARs;

—millimeter wave radars;

—daylight and IR cameras;

—microphones;

—scanning sonars.

Apart from the use of AI, the author of [44] distinguishes three classes of data available from maritime sensors comprising the pixel, time and frequency domains. The first domain naturally includes the visual sensors. Radar, LIDAR and sonar data are analyzed in the frequency and time domains and processed to create a pixel domain representation.

Another feature of the technology is the fusion of shipboard sensor data with information contained within navigation charts, local notices to mariners, tide and currents.

The developed technology was tested onboard a 10-meter research vessel within a route 11 nm long on the east coast of the United States within the Chesapeake Bay. The ship was additionally equipped with the following sensors:

—Furuno GP-37 WAAS/DGPS receiver,

—Furuno 1954C 4ft. 48 rpm radar with ARPA,

—Furuno GD-1920C color video plotter,

—ICOM MA-500T class B AIS,

—EchoPilot 3D forward looking sonar,

—Lowrance HDS5 echosounder/fish finder,

—FLIR MD-625 thermal imaging camera, and

—Hikivision 8MP ultra low light imaging camera.

Data communications were accomplished under the NMEA 0183 data bus architecture with direct image capture to video data recorder, all under the control of a Dell Inspiron quad core laptop.

The following features of the implemented procedures are covered in [44]. The neural network to detect and classify various types of vessels and aids to navigation (ATON) was trained using unsupervised learning. Optical character recognition (OCR) was used to positively identify individual buoys and fixed marks from video and infrared imagery with a 60° field of view that may be zoomed to 17° for precise identification. Radar data were analyzed using deep learning AI to help discern information beyond that available in the pixel domain radar image. This included analyzing changes in frequency, amplitude, and phase.

The experiment results are as follows. With initial neural network configuration consisting of the ResNet50 architecture, ATON identification and classification rates of min 93% were achieved, and subsequent use of a Convolutional Neural Network (CNN) resulted in enhanced results (98%). Further improvements were achieved through adding new layers tailored specifically to features and attributes associated with vessels and ATON. The primary CNN architecture for the pixel domain was an AlexNet design consisting of 27 different layers, and 29 different layers for the time and frequency domains.

All ATON along the route were identified as being of the appropriate type (church, lighthouse, building, etc.) and having proper characteristics (red, green, numbers, letters, etc.). Positive identification of specific buoy occurred for 18 of the 27 occurrences, and for three vessels, with correct positioning made using radar.

Route selection considering vessel draft and route directness was confirmed by forward looking navigation sonar bottom topography, with temporary deviation from planned course necessary for vessel avoidance on three occasions.

When discussing the experiment results, the author remarks that the primary factor in the failure of this method was in cases where the buoy was not viewable due to its rotation. The system demonstrated decreased efficiency in cases of small vessels showing erratic behavior along with unpredictable changes in course and speed.

AUTONOMOUS NAVIGATION TECHNOLOGY IN THE NAVY

Autonomous navigation technologies could not but attract the attention of the navy. Among the sea ships, first they were applied onboard the cutters. The unmanned cutter projects are being run by a number of countries, first of all the USA, Germany, Israel, Singapore, UK, and France.

At the first stage they developed small-sized remotely controlled cutters to assist small ships in coast guard, which are cheaper than coast guard ships, and the absence of the crew critically reduces the cost [45]. In the USA, the first Fleet type cutter (Fig. 5) was handed over to the Navy by General Dynamics in 2008.

Fig. 5.
figure 5

Coast guard cutter Fleet.

Full size image

In 2010 the Defense Advanced Research Projects Agency (DARPA) launched the project ACTUV (Antisubmarine Warfare Continuous Trail Unmanned Vessel), for which the trimaran Sea Hunter was developed by Leidos [46]. The 40 m long self-piloting ship with a speed of 27 kt and cruising range of 10 000 nm has autonomously navigated from San Diego to Pearl Harbor, Hawaii, and back (a total of 5200 nm), it was announced in January 2019.

Recently, China has started to join the first players [47]. A group of 56 small-sized unmanned cutters were tested in 2018 in the South China Sea, where they demonstrated different figures and avoided the obstacles successfully. In the nearest future JARI-USV, a 15 m long unmanned combat boat with 42 kt speed and 500 nm cruising range, is expected to be taken into service. The boat can intercept underwater, surface and air targets depending on the equipment installed.

Having mastered the cutter building technology, in 2020 the USA proceeded to the larger combat ships. First the U.S. Navy asked L3Harris Technologies Inc. to build a prototype Medium Unmanned Surface Vehicle (MUSV) under terms of a $35 million contract [48]. The MUSV prototype for intelligence gathering having a 60 m length and 500 tons displacement will be tested in the early 2022. It is planned to construct a total of 40 ships.

Almost simultaneously in September 2020 the USA Navy awarded $7 million contracts to each of six companies including Huntington Ingalls (the only builder of USA atomic aircraft carriers), Lockheed Martin, Fincantieri Marinette Marine (both building the littoral combat ships) for Large Unmanned Surface Vessel (LUSV) studies [49]. Simultaneously a $59 million option is provided for the construction of the best prototype. The ships are expected to become high-performance combatants with a displacement of 1000–2000 tons.

In another project initiated by DARPA, a $14 million contract was awarded to Applied Physical Sciences Corp. for the technology of unmanned transportation of military forces to remote naval bases [50]. It is planned to build medium-sized UMS connected in the so-called sea trains. The ships will move together in groups formed closely front and back without connection, to reduce the forces of waves and fuel consumption. A similar $9.5 million tender contract for the first development stage was signed between DARPA and Gibbs&Cox, a shipbuilding company with an experience in this sphere [51].

Unmanned cutters are also being developed in Russia. The cutter Iskatel (Fig. 6) by Air and Marine Electronics (NPP AME) JSC, St. Petersburg, tested in 2016 was presented at the 8th International Maritime Defense Show-2017 [52]. Actually NPP AME designed the electronic equipment and run the project, the hull was built by Neptun company, and radiolink, by Luch design bureau. The cutter has a 8.4 m length and a 25 kt full speed, its optoelectronic monitoring system is capable of detecting and tracking the surface objects at 5 km range.

Fig. 6.
figure 6

Cutter Iskatel.

Full size image

UNMANNED SHIPS IN RUSSIA

Russia started the large-scale development of autonomous navigation technology for the merchant fleet in 2016, when within the National Technology Initiative the working group was formed and MARINET roadmap was developed, focusing, among the others, on unmanned shipping [52]. The development is run by Kronshtadt technologies, assisted by Krylov State Research Center and NPP AME. Alexander Pinskiy, the Director General of Marinet Industrial Center, noted in November 2018 that by “unmanned shipping” we mean the automated remote control of the ship motion.

Unlike the international projects, where UMS are built from the ground up, in Russia the developers follow a different principle: equipping the existing ships with the required systems. In September 2019 the Ministry of Industry and Trade of Russia signed a RUR 310 million contract with Kronshtadt technologies for the development of a uniform technological platform for unmanned control of merchant sea ships [53].

The core of the platform is the simulation environment for the ship navigation with account for all affecting factors and their characteristics, including the avoidance in narrows. The platform also includes the following key components [54]:

–3D visualization environment to visualize the ship trials;

–module displaying the images and data from the ship sensors with account for the equipment errors and noise;

–control software module for the ship and its systems. The commands generated by the algorithms are transmitted to the ship model ensuring its virtual navigation adequate to the reality. This technology is used to develop the unmanned ships and their systems and to critically reduce the development terms.

In March 2019 it was announced that the National Technology Initiative experiment was going to enter its practical phase [55]. The project participants include:

—Sovcomflot, which provided the shuttle tanker Mikhail Ulyanov sailing between Murmansk and the Prirazlomnaya platform;

—Pola Rise, which provided the bulker Pola Anfisa sailing in the Black Sea;

—Rosmorport, which provided the self-propelled hopper barge Rabochaya with the dredger Redut performing dredging in Port Kavkaz.

Interestingly, the ships of three different types operating in different regions were selected. This would allow testing the performance of the Kronshtadt Technologies platform. In November 2020, an update on Rosmorport activities was released [56]. A remote control panel for the barge Rabochaya is installed onboard the dredger Redut, while the barge also houses the special components of the platform. It is planned to test the barge voyage from the dredging area to the offshore spoil bank 30 km away first in remote control mode, and later, completely automatically.

Realizing the importance of transition to unmanned navigation, on December 2020 the Russian Government approved a decree on the introduction of a national-wide experiment in UMS operations till 31.12.2025 [57]. The trials are to be conducted in 11 regions of Russia. Based on the results, the Ministry of Industry and Trade will organize the production of the ship remote and automatic control systems.

The test water areas for the ship trials are being prepared already in St. Petersburg and Leningrad oblast [58]. They are organized jointly by Admiral Makarov State University of Maritime and Inland Shipping and the Volga-Baltic State Basin Board of Waterways and Navigation. The first polygon is organized in the Ladoga lake, the second one, in the Neva river. The 70 nm2 Ladoga polygon is the most important. The region features the sufficient depth, required navigation equipment, mapping with updated electronic navigation charts, and satellite communication capabilities.

CONCLUSIONS

The following can be noted to conclude the paper:

—the state of the art in computation, navigation, and communication technologies is sufficient for developing the unmanned ships;

—a considerable number of remotely controlled unmanned ships are likely to appear in the next 10–15 years, and completely automatic ships are expected in 15–20 years;

—completely autonomous ships will prove effective only if based on machine learning and deep learning artificial intelligence;

—IMO should develop/revise the documents legalizing the existence of unmanned ships and ensuring their safe coexistence with the crewed ships as soon as possible.

It remains to be seen whether the author’s opinions presented in the paper will be confirmed or denied over time.