The NAO Robot in Slippery Scenarios: A Strategy

The Robot ABSTRACT In this paper, a strategy for adapting the NAO robot to different floors with different slipperiness degrees is presented while following a desired human-like Zero Moment Point trajectory. The robot’s gait is generated based on the HRSP software package and it aims to be as human-like as possible. The gait parameters such as the step length and walking speed are optimized in order to generate gaits with adequate RCoF for the floor’s ACoF, which minimizes the slipping probability, and as such avoiding undesired falls. This choice of gait parameters is based on the analysis of the ground reaction forces and human behavior. Also, gait adaptation is further improved in order to follow a desired Zero Moment Point trajectory, through the use of a controller that offsets the hip and ankle joint angles. The novelty of this work lies on the fact that machine learning techniques are used to adapt the gait parameters and joint corrections to make the robot more resistant to both slipping and external disturbances.


INTRODUCTION
Humanoid robot development has been a field of study that has seen a lot of research in the last few years and is becoming more and more relevant as human-machine interaction becomes omnipresent, and robots fill increasingly important roles in society, such as rescue, industrial automation, therapy, and rehabilitation. Though many wheeled robots perform these tasks adequately, many are the situations which require bipedal robot intervention, namely through terrain, large obstacles to traverse, or stairs to climb. The disadvantage bipedal/humanoid robots have compared to wheeled ones is that they are more complex and difficult to control due to their high center of gravity and multiple degrees of freedom, and despite various efforts to come up with efficient and robust locomotion strategies, there are still no definite solutions for humanoid gait and balance control. The complexity of the problem increases when in the presence of low friction scenarios, where any minor slip can result in an unpredictable response, causing the robot to fall. It is then fundamental to have a control strategy that enables walking on environments with different foot-floor friction coefficients without compromising the robot's balance. Therefore, this project aims to develop a system capable of adapting the robot's locomotion according to the available friction coefficient, as to prevent possible undesired slips, improving stability in slippery floors.
The proposed approach can be divided in four parts: 1) Use HRSP, a previously developed humanoid robot simulator at Mihajlo Pupin Institute that uses machine learning techniques, to generate human-like gait trajectories based on human motion capture.
2) Dynamically choose gait parameters to be used on the gait generation that minimize the risk of slipping, based on human behavior and ground reaction forces analysis obtained from simulations. The simulation results will be used to train neural networks that automatically choose the gait parameters.
3) Generate adequate Zero Moment Point (ZMP) trajectories based on human gait. 4) Develop a controller that offsets the hip and ankle joint angles in order to follow a ZMP reference trajectory, correcting also sudden balance disturbances. Both traditional and machine learning methods will be developed.
This paper is an update of the previous paper published in the 2019 14th Iberian Conference on Information Systems and Technologies (CISTI) (Franco et al., 2019), where a more detailed explanation of the controller is given, as well as additional preliminary results. The simulation output is explained in more detail as well as a comparison between the factory default gait and the human-like gait that is generated by the HRSP system. It is organized in the following manner: Chapter Introduction provides a brief introduction of the research project and justification for it. Chapter Related Work focuses more on the state of the art and related research. In Chapter Proposed Strategy, the strategy for the development of the control system for the robot is described in more detail. Chapter Conclusion and Future Steps discusses the preliminary results and plans the next steps to take for the implementation of the final system.

RELATED WORK
The aim of the presented research is to prevent undesired external perturbations (slippage), by changing the robot's gait when the surface´s friction coefficient is known. As such, it is pertinent to establish an introductory definition of what friction and friction coefficient is. The friction coefficient is described by where μ is the static friction coefficient, is the perpendicular force applied to the motion plane, and is the maximum reaction force on the motion plane that is contrary to an applied force on the same plane. We assume that the friction coefficient does not depend on the direction of .
This means that for a given μ, the necessary force to overcome is directly proportional to . When the robot is in its single support phase, has its maximum value. However, a high enough momentum (caused by the robot's movement) can cause the friction force to be overcome if the friction coefficient is low enough. Now that the definition of friction coefficient has been established, the next logical step is to explore how can a robot maintain its balance in the presence of different friction coefficients.
Balance and disturbance adaptation in bipedal/humanoid robots are challenges that have been tackled since the first generations of humanoid robots (Abrams, 2005;Akhtaruzzaman and Shafie, 2010). Starting in 1968, the concept of ZMP was introduced by Miomir Vukobratović (Sardain and Bessonnet, 2004;Vukobratovic and Borovac, 2004;Vukobratović and Juričić, 1968), which paved the way to the basics of robot balance, leading to balance recovery techniques, like inverted pendulum and center of mass balancing, ankle compensation, hip compensation, hip and ankle compensation, capture point, or a combination of them (Elhasairi and Pechev, 2015;Ferreira et al., 2010;Gouaillier et al., 2010;Griffin et al., 2017;Pratt et al., 2006;Semwal and Nandi, 2013;Wei et al., 2009). Despite these advancements, many bipedal robots still have problems maintaining balance when walking on slippery surfaces due to sudden disturbances caused by an unexpected slip of the supporting foot. The ZMP definition assumes that the contact between the floor and the supporting foot is slip-free, as if the foot was 'glued' to the ground. Therefore, stabilizing the robot by monitoring the ZMP position alone does not suffice when in the presence of a low friction floor. Some research has been made (Brandão et al., 2016;Khadiv et al., 2017) regarding this situation, but the emphasis seems to be on the response to external pushes (Kasaei et al., 2017;Semwal et al., 2017) or ground inclination changes, and not on slipping prevention. A good reference for prior adaptation to slippery floors is the adaptations and precautions human beings take when traversing slippery floors. Studies have shown that prior visual inspection or knowledge of slippery floors cause humans to adapt their gait before encountering the low friction surface, by altering their single-phase duration, reducing their step length and lowering their walking speed (Cham and Redfern, 2002;Heiden et al., 2006). Studies have also shown that human beings do not need to alter their gait parameters up to a threshold of available friction coefficient (Chang et al., 2017). Their gait characteristics allows them to maintain a required friction coefficient (RCoF) low enough so that even at medium-low available friction coefficients (ACoF) they can walk safely as RCoF is lower than ACoF.
Another relevant part of human-like gait is the ZMP trajectory and its relationship with the Center of Pressure (CoP). In robot locomotion, the CoP is the point in the support polygon where all the applied vertical forces in different points are concentrated. It is somewhat analogous the center of mass, but with forces instead of masses and an area instead of a volume. If the ZMP does not leave the support polygon, it is coincident with the CoP. Therefore, using the CoP instead of the ZMP can be advantageous in the way that it simplifies the calculations and reduces the computational power required to do so.
When slipping occurs, the static friction force is and the robot's ZMP quickly moves outside the support polygon, and the CoP quickly moves to its edge. This means that the robot cannot regain its balance without taking an extra step so that the ZMP and CoP return to a stable position (Pratt et al., 2006;Wei et al., 2009). Therefore, monitoring and controlling the ZMP/CoP is fundamental to assure the robot's balance during its walking cycle. Mimicking a human's ZMP trajectory on a bipedal robot has been proven to be a good strategy in order to obtain a more stable and human-like gait cycle compared to a traditional approach (Ferreira et al., 2009a(Ferreira et al., , 2009b(Ferreira et al., , 2010Griffin et al., 2018;Lima et al., 2014;. While a traditional ZMP trajectory tries to maintain it at the center of the foot during the single support phase, a human ZMP/CoP trajectory presents a gradual weight transfer from the heel to the toe during the single support phase (see Figure 1).
If a humanoid robot is to be capable of adapting to different floors with different degrees of slipperiness, it is important for it to adapt its gait not only to minimize slipping chances, but also to follow a correct ZMP trajectory to increase its stability. This project aims to reach that goal using accurate simulations in order to develop an intelligent controller to be applied afterwards in a physical NAO humanoid robot.

PROPOSED STRATEGY
Prior to implementing and testing the final controller on a physical robot, it is important to do research and development in a simulator in order to avoid possible damage to the robot and to better control the environment variables and ensure repeatability.
The chosen platforms are MATLAB alongside with V-REP (Coppelia Robotics, 2014). MATLAB provides fast and practical algorithm implementation and data analysis, making it extremely easy to perform matrix operations, and easy graph plotting. V REP provides a powerful open-source simulator that comes from factory with the used humanoid NAO robot (Gouaillier et al., 2008). V-REP is an open-source robotics simulation platform that has several physics engines to choose from (Bullet (Coumans, n.d.), ODE (Smith, n.d.), Vortex (CM Labs, n.d.), and Newton), and it is possible to communicate with external applications using ROS nodes, Blue Zero Nodes, or an external API.

Simulation Method
MATLAB's communication with V-REP will be made using V REP's provided API. A MATLAB script initializes all the necessary variables and opens a new V-REP simulation. Then, reads and writes necessary data from sensors and actuators in V-REP, and triggers another simulation step in blocking mode, meaning that the script does not continue its execution until another simulation step is taken in V-REP. After one simulation step, the MATLAB script continues to run, and this process repeats until the final iteration is reached, and MATLAB closes the communication channel with V-REP. This method, despite suffering from communication lag, allows for several simulations in different virtual scenarios, with different ACoF values, since it is not possible to select a different ACoF during a simulation, from within V-REP. The generated data will then be easily saved and organized. Figure 2 shows the overall strategy for the robot control. The physics engine to be used in V-REP is Vortex, which can accurately simulate rigid body dynamics and collisions, and is used mostly for robotics and heavy equipment training for both commercial and military applications. The simulated foot-floor friction coefficients will range from 0.1 to 0.5, the gait speed ranged from 1.8 cm/s to 7.8 cm/s, and the step lengths from 3 cm to 8 cm. These values were chosen based on preliminary tests and are the ones that provide a stable gait range.
The foot-floor friction coefficient will be estimated using specially developed shoes that are able to measure both vertical and horizontal applied forces (Almeida et al., 2018) (see Figure 3).
These shoes are custom-made for the NAO robot, and possess horizontal force sensors placed at a 45-degree angle, which allow for the measurement of horizontal forces. It consists of an inner sole and an outer sole. The forces acting between them gives an estimate of the ACoF. Since the shoes are considerably thick (18 mm) when compared to the total height of the NAO robot (574 mm), it is important to measure the effects they have on the robot's overall balance, which is why all experiments and simulations will be made with and without the shoes on (see Figure 4).  The proposed methodology to tackle the problem can be divided in two distinct sub-problems: gait adaptation and robot balance.

Gait adaptation
For the purposes of model simulation, the specialized modeling and simulation software entitled Humanoid Robot Simulation Platform (HRSP) (Rodić, 2007) has been applied. This software has been designed for research in biomechanics and humanoid robotics, especially for analysis of bipedal locomotion (Rodić et al., 2008;Vukobratović and Rodić, 2007;Vukobratovic et al., , 2012. The NAO robot parameters are acquired from the manufacturer's catalogue (Gouaillier et al., 2008) and used in the software package HRSP. It is assumed that the robot moves on a flat surface (fully slippery, moderate slippery or rough surface with a friction coefficient between the foot and the supporting surface in the range 0.1 (slippery), 0.3 (moderate) and 0.5 (rough)).
The NAO humanoid robot has factory pre-defined gait parameters which enable stable walking on a flat surface. The user can choose and interactively change the length of robot step within its range of 0.04 to 0.08 m. The time period of a single step can be varied between 0.46 s and 0.60 s. In this way, one can define the robot walking speed. The gai t of the NAO robot is not anthropomorphic (human-like) but consists of regular geometric cycloids. NAO robot gait is influenced primarily by driving forces of actuators, geometric characteristics of the mechanism and shape of the foot. Although the NAO robot manufacturer offers the possibility to vary the parameters of stable walk in the user menu, it was proven in practice that this is not always the case and that the stability of the robot significantly depends on the type of ground surface on which this motion is performed. In practice with the NAO robot, problems were encountered, especially when it was required for the robot to walk on very smooth and slippery surfaces. It often happened that the robot became unstable and tipped over. In this work, we tackled the problem of solving the robot gait stability by synthesizing a knowledge-based gait controller associating the dynamic indicators of gait stability (ZMP position, dynamic ground reaction force), the friction coefficient between the foot of the robot and the substrate and the set of gait parameters -step length and walking speed. Several gaits will be generated, spanning the range of step lengths and velocities (3 cm to 8 cm, and 1.8 cm/s to 7.8 cm/s, respectively) and simulated in a slip-free environment. Then, the ground reaction forces will be obtained in order to calculate the Required Coefficient of Friction (RCoF) as shown in Figure 5, which is the minimum friction coefficient between the foot and the floor, that prevents slipping. Another possible definition of RCoF is the ratio between the shear and normal forces when no slipping occurs. If the floor's friction coefficient is smaller than the RCoF, the robot will slip causing instability and a consequent fall.
It is important to note that RCoF depends solely on the gait characteristics, and not on the floor's friction coefficient. An example of a RCoF measurement in a simulation can be seen in Figure 5 with and without shoes: Although similar, the curves show that with shoes, the robot has a lower peak RCoF as compared without shoes. This can be explained by the fact that a higher normal force (caused by the added weight of the shoes) requires a higher tangential force to be applied for the foot to slip.
Other data is collected, such as the ground reaction forces and moments as well as the ZMP in the x and y axis. The graphs presented in Figures 6-10 show some characteristic simulation results that will be used to build the intelligent gait controller.  Such data from simulations will be used to develop the controller. The target foot cycloids and joint angles are inputted to the simulation environment, from which the ground reaction forces, momentum and ZMP trajectories will be used to train the intelligent controller that enables the robot to follow a target ZMP trajectory and monitor/manipulate the momentum and ground reaction forces in order to minimize slipping.
From the simulations made, the optimal gait parameters are identified, choosing those that maximize the walking speed and minimize slipping probability (see Figure 11). It was observed in preliminary tests (Ferreira et al., 2020) that this criterion leads to a similar adaptation that human beings have to slippery floors (Cham and Redfern, 2002;Chang et al., 2017).
It can be seen that the HRSP trajectory is more stable than the factory default one in the sense that it does not reach the edge of the foot/shoe as much. This stability gain can also be further improved by implementing a balance controller that 'forces' the CoP to follow a desired trajectory. . Adapted gait parameters that maximize walking speed and minimize slipping probability. Above an ACoF threshold of 0.3 the robot can walk safely at maximum speed and step length. The dotted lines represent the linear approximation between the maximum slip-free values obtained from simulation and gait step length and speed equal to zero, when the ACoF is zero as well

Balance controller
The balance control strategy consists of offsetting the hip and ankle joint angles in order to correct any ZMP deviations from the desired ZMP trajectory. In order to achieve that, several hip/ankle offsets will be applied to the robot both in a static position during single phase and mid-gait. The offset angles range from -16º to +16º in the coronal and sagittal planes. Then, the ZMP trajectory responses are obtained and machine learning techniques (neural networks, support vector machines and extreme learning machines) are used in order to perform the inverse operation, that is, to obtain the necessary offset angles for a desired ZMP movement. The parameters to take into account are not only the current joint angles, but also the past sampled joint angles (as much as the hardware will allow it), the present and past CoP values, and the readings from the gyroscope that is built into the NAO robot. Also, the robot's cameras will be used to train a neural network that will predict possible floor material changes/contaminants as to anticipate any friction coefficient change, and not rely solely on the instrumented shoes. These angle offsets may be applied as a step disturbance, a ramp ranging from 20 ms to 100 ms, or a third-degree polynomial with 80 ms or 100 ms duration. Another control strategy to be implemented is to obtain the ZMP values given a known robot's joint configuration. A range of possible robot positions will be simulated, and the ZMP values obtained. Then, an artificial neural network will be trained in order to predict the ZMP values from the robot's joint values, and a PD controller is used to correct for any deviations. The final system for the balance controller is represented in Figure 13.
The green rectangles refer to the controller part, responsible for the adaptation of the gait parameters and ZMP correction. The blue rectangle is concerned with the measurement of the foot-floor friction coefficient. The black rectangles are responsible for low-level robot manipulation techniques.

CONLUSION AND FUTURE STEPS
Preliminary results shown in III (b) show that the proposed goal for the controller is plausible i.e., it is possible for the robot to adapt to slippery floor by following a human-like strategy, and that strategy is optimal. This motivates the authors and supports the next steps which are to develop a controller that allows for the robot to follow a ZMP trajectory and recover from sudden slips to an extent that it does not have to take an extra step, and then validate the final control system on a real robot. The controller will use machine learning techniques such as Convolutional Neural Networks to infer the friction coefficient based on vision and reinforcement learning to adapt to various ACoF, where classical control would probably not be sufficient.