Siarhei Herasiuta

United Institute of Informatics Problems, Minsk, Belarus


Uladzislau Sychou

Belarusian State University of Informatics and Radioelectronics, Minsk, Belarus


Ryhor Prakapovich

United Institute of Informatics Problems, Minsk, Belarus


Abstract. In this paper we describe our experiments with a group of collaborative robots, IoT devices connected to robots based on WebSockets RFC 6455 Protocol using Google protocol buffers for message handling, distributed information system for it and a human interaction problem with these systems.

Keywords: Internet of Things   Robotics as a service   WebSockets   Collaborative robots   Google protocol buffers   Robo-hockey competition   Distributed information systemt

1 Introduction

In the IoT age, producing almost any machine going forward means giving it an IP address on the Internet. With robots we no longer just retrieve information from somewhere remote, we no longer just become tele-present somewhere else, we can now act on the physical world in a location other than where our physical body exists. This aspect of robotics is rarely discussed outside of a few existing examples such as military drones or healthcare. Yes, there are and will be many cases where fully autonomous robots will be designed to perform particular tasks for us. It’s not necessary for us to be involved in controlling the operation of our vacuum cleaning robots. However, there will be just as many cases $($and perhaps many more$)$ where we will want to act, in a general sense, in a remote location, and where we’ll share decision making and control of the robot, with the robot.

In Fig. 1, we demonstrate a very low size and low power new IoT device in comparison with “old” hardware, but they can control regular human oriented devices $($refrigerators, washing machines and etc.$)$ and communicate with robots. So there we see a very quick technology revolution for such type of devices.

Completely new IoT hardware ESP8266 ESP-05 with antenna (green small square) and old IoT hardware WiFly RN-XV 171 (red big square) placed in low level robot control system (Color figure online)

Fig. 1. Completely new IoT hardware ESP8266 ESP-05 with antenna $($green small square$)$ and old IoT hardware WiFly RN-XV 171 $($red big square$)$ placed in low level robot control system $($Color figure online$)$

2 “Robots as a Service” Approach in Building Distributed Information Systems

The basic idea is that run web service infrastructure support on a robot control system board. One of the solution is to use Web Sockets technology $[1]$. The WebSocket Protocol enables two-way communication between a client running untrusted code in a controlled environment to a remote host that has opted-in to communications from that code. The security model used for this is the origin-based security model commonly used by web browsers. The protocol consists of an opening handshake followed by basic message framing, layered over TCP. The goal of this technology is to provide a mechanism for browser-based applications that need two-way communication with servers that does not rely on opening multiple HTTP connections $($e.g., using XMLHttpRequest or <iframes> and long polling$)$.

Other idea is to decrease maximum data exchange between robots using well optimizing binary data protocol for objects handling over network channels. And the best solution there is using Google Protocol buffers. Google Protocol buffers are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. We can define how we want our data to be structured once, then we can use special generated source code to easily write and read our structured data to and from a variety of data streams and using a variety of languages.

3 Robot to Robot and Robot to Human Communication Problem

Social science literature indicates many types of human communication behavior used during collaborative tasks, including attentional cues to indicate an area of focus, staging actions to maximize shared visual information, gestural and speech cues indicating intentional goals or instructions, and coaching actions such as feedback, encouragement, and empathetic displays to build team rapport. Effectively producing all of these communication actions on robots, in real-world task environments, is not currently feasible and would be difficult to generalize across different robot embodiments; for example, indicating attentional focus is different with humanoid and non-humanoid robots. Based on a review of the relevant social science literature covering human collaborations, and on observations of person-person task collaboration in our experimental setting, we focus in this work on using speech. Speech works well across different embodiments and is effective in communicating intent. On the other hand, speech has obvious limitations in noisy environments, with users with hearing or linguistic limitations, and in certain scenarios, such as disambiguating many similar objects. Nonetheless, speech is a natural human communication modality that addresses a range of use cases in home and work environments. Other methods of group robot control is not suitable if robots are different $($Fig. 2$)$.

Group of different types mobile robots controlled by Android application using Wi-Fi protocol

Fig. 2. Group of different types mobile robots controlled by Android application using Wi-Fi protocol

4 Language Modelling for Robots-Human Interaction Using NooJ Approach Experiment

The goal of this experiment $[3]$ is to interact with some number of robots in order to make them perform commands. With NooJ approach $[2]$ this model can be designed in much easier way compared to other tools. The idea is to design the language that would be common and close to every-day language of the humans and that it would be able for machines to ‘understand’ it. Further design will be dedicated to replaying the model that has already been designed and the new data which is the new possible language constructions, phrases, linguistic units, etc. that can be expected from humans in order to interact with machines in their natural way.

At the start stage of the work we use deep syntactic analysis to get the model that is as simple as possible and yet far from underfitting the real model. We will use such concepts as ‘Subject’, ‘Action’, ‘Object’ and ‘Features’. Using NooJ Syntactic Grammar we design graph model for combining all these concepts and linguistic units that will refer to them. Eventually we perform play-out routine to generate dictionary for robots using NooJ Dictionary. Some units from this dictionary for Belarusian language will look like:

Poбaт _ Iнфapмaтap cкaжы надвор'е назаўтра, GUID = R3 + Action = tell + Object = wheather + Features = for tommorow
Poбaт _ З _ Кaмepaй зpaбi фoтaздымaк злeвa, GUID = R2 + Action = make + Object = photo + Features = from left
Poбaт _ Tpaнcпapтнiк прывязi напоi для мяне, GUID = R1 + Action = bring + Object = drinks + Features = for me

‘Subject’ $($GUID in our example$)$ refers to a robot’s name. ‘Action’ refers to an action to be performed by robots that is usually represented by a verb. ‘Object’ represents a target of the action. And ‘Features’ is an add-on to specify ‘Object’ or ‘Action’.

Using such kind of concepts, which are natural for humans, and NooJ tools we can generate Language Model for robots-human interaction and it will be high level stand of robot control system $($Fig. 3$)$.

Ontology semantic based high level standard of control system for three robots

Fig. 3. Ontology semantic based high level standard of control system for three robots

5 Experiment of Controlling Two Groups of Robots Functioning Opposing Each Other

Experimental studies were carried out in two stages (Fig. 4). In the first phase conducted field tests using the experimental group of mobile robots. The group consists of eight robots and the central computer. Each robot is equipped with a wireless modem XBee Pro, working in the broadcast mode, the microcontroller the low-level control system and tracked chassis with dimensions 150 × 190 × 100 mm.

Hockey with robots competition

Fig. 4. Hockey with robots competition

Low-level control system implements a targeted reception of data packets from the central computer, and execute control commands, and send to the central computer of sensory data.

The central computer is equipped with a radio modem of the same type as the robots. Its software allows you to communicate with robots, data storage and the formation of management teams with interviews scrap-hundred milliseconds.

For the interaction of robots and the central computer network protocol is designed based on the physical and data link layers XBee Pro and implements the application layer of the OSI model, as well as elements of the network and transport levels to the extent necessary to control a group of robots.

Using existing robots possible to determine the basic relationships that characterize the information transfer processes in a group of robots. The dependence of the time the data packet on the packet size, the effect of the removal robot the signal strength $($RSS$)$ and the related transmission errors, the maximum number of simultaneously operating the transmission in the broadcast mode radio modem $[4–7]$.

Table 1. Serilization/deserialization performance comparison for small and large objects in different common used data object formats $($Size in bytes, time in milliseconds$)$

XMLBinaryJSON-service stackProtocol buffer
Size $($large$)$323.981204.793141.863104.191
Deserialize $($large$)$7.88938419.397635.7314723.82069
Serialize $($large$)$5.50809113.7000643.5596881.447036
Size $($small$)$2986698662
Deserialize $($small$)$0.0159770.0194050.001740.003883
Serialize $($small$)$0.0218970.0210230.0036450.000989

6 Conclusion

Our experiments give us vision that we need to create a new programming algorithms for software development of distributed information system for collaborative robots and IoT. This algorithms should adopt well known approaches to a new class of Intellectual robots with complex control system and IoT enabled devices. We recommend build software stack using Web Sockets, Google Protocol Buffers $($Table 1$)$ for serialization/ deserialization over TCP Wi-Fi high speed connection. Also we need to develop an Operating system for robot or adopt current used $($ROS$)$ for collaborative robots.