The Smart Cities Robotics Challenge (SciRoc) competition was organized in ‘episodes’, where each episode tested a robot on a combination of skills that are relevant for a specific context, combining object recognition, manipulation and social interaction. The procedures for each episode were published in advance to give teams time to develop the software needed for the task. However, some aspects remained intentionally vague or were randomized for each run. Ten teams from five countries participated in the challenges.

Credit: Enrico Motta

A running theme of robotics competitions is that tasks that to us seem trivial can create hard problems for fully autonomous robots. An approach that might work well in the lab when interacting with the engineers who programmed the robot usually does not translate easily to interacting with random volunteers in an open and noisy environment.

One such surprisingly hard task for robots is opening a door and walking through it. Famously, the 2015 DARPA robotics challenge showed how hard the problem can be, with more than one humanoid robot toppling over or crumpling before they could open the door. In the SciRoc challenge, the robots had much less trouble in keeping their balance, as they had a rolling base, but still the motion of opening any door is a challenging one.

The participants also had to compete against the door itself. The MADROB project, led by researchers from the Polytechnic University of Milan, developed an active door for Eurobench, a Europe-wide project to establish benchmarking procedures for humanoid robots1, which can simulate weight and resistance in the door and measure the forces the robots apply during a test. The appearance and handle can also be adapted during a competition, increasing the difficulty for autonomous door openers.

The second episode in the ‘manipulation’ category was a ‘pick and pack’ shopping challenge, in which a robot had to verify inventory in shelves and grasp common products that each require different grasping and holding strategies (pictured). Again, while this task seems simple, grasping any possible objects from a selection of many is generally a difficult task for robots2.

In two other tasks, the robots had to closely interact with volunteers who had no robot handling experience. In an experimental coffee shop, they had to take and fulfil orders after scanning each table and determining whether it was already served, required service or was empty. It was then tasked to take orders and bring new guests to empty tables. In a mock elevator, the robots had to navigate close to moving humans in a confined space and ask them verbally which floor they were on.

After each of the interactive episodes, the volunteers were asked to fill in a questionnaire on how comfortable they felt around the robot. The questionnaire also resulted in a conference presentation at Ro-Man 20193. In addition to selecting a winner for each separate task, the robot that received the highest score on its social abilities overall was named the ‘most social robot’.

The integration into the smart city infrastructure of Milton Keynes, the SmartMK data hub, allowed real-time monitoring of what each robot was doing during its task. Some tasks required interaction with the data hub to update inventory or retrieve orders. In the pick and pack challenge, for example, the robot received its order and the available inventory through the data hub, and had to report each change it made. Points were deducted when a robot, after dropping an item on the way, still claimed to have successfully delivered the item. For coordinating robots in real-world tasks, communication is essential to respond to errors and unforeseen events.

In the emergency response category, medication had to be delivered via an autonomous drone through obstacles. This task requires very precise localization, which was quite challenging under changing light conditions and without GPS.

Earlier this year, PAL Robotics offered teams a TIAGo Robot to rent for preparing and taking part in the competition. Three teams made use of that offer, while two others competed with their own TIAGo robots. They competed against a Monarch from IDMind, a KUKA youBot and a Softbank Pepper.

Providing a unified data hub hopefully catches on in other competitions, as this might be a good place to store logs and debug information for evaluation by other groups. While over 200,000 data entries were stored during this competition, logging the full sensory data of a robot for post-hoc evaluation by all competing teams would still be technically challenging, both in terms of wireless connectivity and finding a common representation, but also potentially very valuable to make robots more robust to the noisy and changing environment that led more than one robot in the competition to lose their orientation seemingly at random.

Milton Keynes seems uniquely positioned to hold an event like this as they are already trialling prototypes for autonomous delivery robots and smart city infrastructure. Bringing robot competitions into a public space, such as a shopping centre, can help to maintain a dialogue between researchers and the public. While many shoppers were excited to see the robots competing, some also expressed concerns about the impact robots can have on the labour market and society in general. At the same time, some were surprised to see that robots still struggle to recognize humans, to use doors reliably, or simply grasp and carry an object. Public events like SciRoc will help to get a better understanding of the public’s expectations and concerns.

The second Smart Cities Robotics Challenge will take place in 2021.