Universidade de Aveiro 2023

Luís Miguel Martins Almeida Melhoramento da marcha humanóide através da análise de uma aprendizagem baseada em um novo sistema de força 3D

Improved humanoid gait using learning-based analysis of a new wearable 3D force system Luís Miguel Martins Almeida Melhoramento da marcha humanóide através da análise de uma aprendizagem baseada em um novo sistema de força 3D

Improved humanoid gait using learning-based analysis of a new wearable 3D force system

Dissertação/Tese apresentada à Universidade de Aveiro para cumprimento dos requisitos necessários à obtenção do grau de Doutor em Eng. Mecânica, realizada sob a orientação científica do Doutor Vítor Manuel Ferreira dos Santos, Professor Associado com Agregação do Departamento de Engenharia Mecânica da Universidade de Aveiro e do Doutor João Paulo Morais Ferreira, Professor Coordenador do Instituto Politécnico de Coimbra.

This work was supported by FCT through the doctoral grant SFRH/BD/136680/2018 and Project PTDC/EEI-AUT/5141/2014. MISS-ING FCT AND RELEVANT LOGOS HERE Dedicated to my parents and sister. o júri / the jury

presidente / president ABC

Professor Catedrático da Universidade de Aveiro (por delegação da Reitora da Universidade de Aveiro)

vogais / examiners committee DEF

Professor Catedrático da Universidade de Aveiro (orientador)

GHI

Professor associado da Universidade J (co-orientador)

KLM

Professor Catedrático da Universidade N

acknowledgements I am deeply grateful for the invaluable contributions of my supervisor, Professor Doctor Vítor Santos. His insightful discussions, unwavering guidance, and constant encouragement were crucial in shaping the research presented in this thesis. Furthermore, I extend my appreciation to my co-supervisor, Professor Doctor João Ferreira, whose expertise and readiness to assist have greatly enriched the depth and breadth of this study. Their combined mentorship has been instrumental in the successful completion of this research work. My sincere gratitude to the Fundação para a Ciência e a Tecnologia (FCT) for funding the work presented in this thesis throught the PhD grant SFRH/BD/136680/2018. This research was also partially supported by FCT project A2HR with reference PTDC/EEIAUT/5141/2014. I am grateful to the Institute of Electronics and Informatics Engineering of Aveiro (IEETA), the Department of Mechanical Engineering at Aveiro University, and the Institute of Systems and Robotics (ISR) at Coimbra University for their generous accommodation and provision of essential space and materials during my time of research. I would like to thank the Serbian Institute Mihajlo Pupin and Professor Doctor Aleksandar Rodić for graciously welcoming me into their laboratories at the inception of my research journey. I would also like to extend my appreciation to UC3M and Professor Doctor Santiago Martínez for granting me access to their humanoid robot and facilitating the commencement of my research into humanoid control. Finally, but by no means the least, I want to wholeheartedly express my gratitude to my parents and sister for their devoted support.

Palavras-chave Robô Humanóide, Locomoção, Sensores de Força, Inteligência Artificial, Controlo, Melhoramento

Resumo A locomoção de robôs humanoides em superfícies escorregadias apresenta desafios significativos, exigindo soluções inovadoras para estabilidade e adaptabilidade. Este trabalho de doutoramento é composto por três partes integrantes que abordam coletivamente esses desafios, com cada parte construindo sobre a anterior. Parte I explora a medição crítica das forças de reação do solo em robôs humanoides, fundamentais para a análise biomecânica e aplicações potenciais na reabilitação da marcha humana. Um sistema inovador de sapato instrumentado (ITshoe), económico e leve, é introduzido para a medição em tempo real das forças de reação, melhorando a nossa compreensão da locomoção do robô em diversas superfícies escorregadias. Parte II aprofunda a tarefa vital do reconhecimento de superfícies para robôs humanoides que navegam em ambientes do mundo real. Um conjunto abrangente de dados, obtido através de sensores táteis de força em várias superfícies, serve como base. Técnicas de inteligência computacional, incluindo Artificial Neural Networks (ANNs), Extreme Learning Machines (ELMs) e Long Short-Term Memory (LSTM) recurrent neural network, são aplicadas para classificar diferentes superfícies. Experiências em tempo real são apresentadas para demonstrar a adequação das abordagens propostas para o problema de classificação múltipla abordado. Parte III apresenta uma estratégia abrangente para melhorar a locomoção de robôs humanoides em superfícies escorregadias. Esta estratégia incorpora uma arquitetura de controlo baseada no Divergent-Component-of-Motion (DCM) e um Embbed Yaw Controller (EYC) baseado num algoritmo PID. Esta estratégia não aborda apenas o comportamento de deslize em superfícies de baixa fricção, mas também enfrenta padrões de locomoção nãolineares, mesmo em pisos não escorregadios. Experiências extensivas de locomoção demonstram melhorias significativas na estabilidade, redução do consumo de energia e duração da tarefa. As descobertas apresentadas nesta tese fornecem evidências convincentes de uma locomoção adaptável de humanoides em superfícies escorregadias. Estas experiências destacam o potencial de um sistema wearable para aprimorar a interpretação das forças de reação do solo em conjunto com técnicas de inteligência computacional. Além disso, demonstram a capacidade do sistema de adaptar o controlador do robô, em conjunto com um controlador PID, minimizando eficazmente o escorregamento em superfícies classificadas.

Keywords Humanoid Robot, Locomotion, Force Sensors, Artificial Intelligence, Control, Improvement

Abstract Humanoid robot locomotion on slippery surfaces poses significant challenges, demanding innovative solutions for stability and adaptability. This doctoral research comprises three integral parts that collectively address these challenges, with each part building upon the last. Part I explores the critical measurement of Ground Reaction Forces (GRFs) in humanoid robots, crucial for biomechanical analysis and potential applications in human walking rehabilitation. A novel, cost-effective, and lightweight instrumented shoe (ITshoe) system is introduced for real-time GRF measurement, enhancing our understanding of robot locomotion across diverse slippery surfaces. Part II delves into the vital task of surface recognition for humanoid robots navigating real-world environments. A comprehensive dataset, acquired through force tactile sensors on varius floors, serves as the foundation. Computational intelligence techniques, including artificial neural networks (ANNs), extreme learning machines (ELMs) and Long short-term memory (LSTM) recurrent neural network are applied to classify different surfaces. Real-time experiments are presented to demonstrate the suitability of the proposed approaches for the multi-classification problem addressed. Part III presents a comprehensive strategy to enhance humanoid robot locomotion on slippery surfaces. It incorporates a Divergent-Component-of-Motion (DCM) based control architecture and an Embedded Yaw Controller (EYC) based on a PID algorithm. This strategy not only addresses slip behavior on low-friction surfaces but also tackles non-straight walking patterns, even on non-slippery floors. Extensive locomotion experiments demonstrate significant improvements in stability, reduced energy consumption, and task duration. The findings presented in this thesis offer compelling evidence of adaptable humanoid locomotion on slippery surfaces. They underscore the potential of a wearable system to enhance the interpretation of ground reaction forces (GRFs) in conjunction with computational intelligence techniques. Moreover, they demonstrate the system’s ability to adapt robot controller, coupled with a well-established PID controller, effectively minimizing slipperiness on classified surfaces.

Table of contents

Table of contents i

List of figures v

List of tables ix

List of abbreviations xi

1 General Introduction 1 1.1 Hypotheses and Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Fundamental concepts of humanoid stability and locomotion . . . . . . . . . 5 1.3 Instrumented systems used in the field of humanoid locomotion . . . . . . . 7 1.4 The Significance of Computational Intelligent Techniques in Humanoid

Robotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.5 Advancements in Humanoid Control . . . . . . . . . . . . . . . . . . . . . . 10 1.6 Humanoid Locomotion in Slippery Environments . . . . . . . . . . . . . . . 11 1.7 Brief Research Journey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.8 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

I 3D FORCE SYSTEM FOR GROUND REACTION FORCES ANALYSIS IN HUMANOIDS 21

2 A Novel Wireless Instrumented System for Ground Reaction Forces Analysis in Humanoids 23 2.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.4 ITshoe Prototype Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.4.1 ITshoe structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.4.2 Sensor calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.5 System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.6 Experiments and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.6.1 Data upload rate percentage loss . . . . . . . . . . . . . . . . . . . . 30 2.6.2 Detection of the vGRFs for a static robot position . . . . . . . . . . 31 2.6.3 Detection of the total GRFs during robot locomotion on two different

floors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.7 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . 34

II LEARNING-BASED ANALYSIS TO CLASSIFY DIFFERENT SLIPPERY FLOORS 35

3 Learning-based Analysis of a New Wearable 3D force System Data to Classify the Underlying Surface of a Walking Robot 37 3.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.3 Materials and methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.3.1 Floor classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.3.2 Data manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.3.3 Data for CIT training, validation and testing . . . . . . . . . . . . . 43

3.4 CIT experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.4.1 ANN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.4.2 ELM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 3.4.3 ANN vs ELM: Classification time . . . . . . . . . . . . . . . . . . . . 45 3.4.4 Step percentage reduction . . . . . . . . . . . . . . . . . . . . . . . . 46

3.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

4 Real-time LSTM-RNN Classification of Floors with Different Friction Coefficients for a Walking Humanoid Robot Wearing a 3D Force System 57 4.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.3 Materials and methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

4.3.1 Floor multi-classification problem . . . . . . . . . . . . . . . . . . . . 63 4.3.2 Data manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 4.3.3 Data for LSTM training, validation and testing . . . . . . . . . . . . 64

4.4 LSTM-RNN experimental results . . . . . . . . . . . . . . . . . . . . . . . . 65 4.5 Online experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

III Adaptive humanoid controller to Improve Humanoid Locomotion on Slippery floors 75

5 Enhancement of Humanoid Robot Locomotion on Slippery Floors using an Adaptive Controller 77 5.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 5.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 5.3 Humanoid Whole-body Controller . . . . . . . . . . . . . . . . . . . . . . . 81

5.3.1 Trajectory optimization . . . . . . . . . . . . . . . . . . . . . . . . . 82 5.3.2 Simplified Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 5.3.3 Whole Body QP Control . . . . . . . . . . . . . . . . . . . . . . . . 83 5.3.4 Implementation in the NAO robot . . . . . . . . . . . . . . . . . . . 83

5.4 Locomotion experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 5.4.1 Step frequency experiments . . . . . . . . . . . . . . . . . . . . . . . 90 5.4.2 EYC adaptive controller experiments . . . . . . . . . . . . . . . . . . 92

5.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

IV GENERAL DISCUSSION AND CONCLUSIONS 101

List of figures

6 Discussion and Concluding Remarks 103 6.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.2 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 6.3 Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 6.4 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 1.1 Illustration depicting a humanoid robot’s locomotion on various surfaces

with progressively reduced coefficients of friction . . . . . . . . . . . . . . . 3 1.2 Schematic illustration of the research framework for enhancing humanoid

robot locomotion on slippery surfaces. . . . . . . . . . . . . . . . . . . . . . 4 1.3 Illustration of the linear inverted pendulum model and ideal representation

of the Zero Moment Point (ZMP) inside the support polygon . . . . . . . . 6 1.4 Schematic representation of strategic placement of FlexiForce sensors …. 8 1.5 Illustration of common sensing systems employed in humanoid locomotion

for accurate measurement and control. . . . . . . . . . . . . . . . . . . . . . 9 1.6 Schematic of the TMLIP model, a bipedal robotic walker and a 5-link full

body model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.7 Control diagram for walking. . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.8 Humanoid Transformer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.9 Variable stiffness feet for humanoid walking. . . . . . . . . . . . . . . . . . . 17

2.1 ITshoe model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.2 ITshoes schematic structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.3 Force sensor calibration line. . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.4 Flexiforce sensor dynamic response. . . . . . . . . . . . . . . . . . . . . . . . 29 2.5 System basic architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.6 Acquisition process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.7 Graphical representation of the force values (example using the LITshoe). . 30 2.8 Robot in a static position. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.9 vGRFs for a robot static position. . . . . . . . . . . . . . . . . . . . . . . . 31 2.10 vGRFs for a walking humanoid robot on two different grounds. . . . . . . . 32 2.11 hGRFs axis detail. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.12 hGRFs in the sagittal plane for a walking humanoid robot on two different

grounds. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.13 hGRFs in the transverse plane for a walking humanoid robot on two different

grounds. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.1 ITshoe schematic structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.2 GRFs for a walking humanoid robot. . . . . . . . . . . . . . . . . . . . . . . 49 3.3 Shoe soles materials. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.4 Layout for the data acquisition and CoF measurement. . . . . . . . . . . . . 51 3.5 Example of the input dataset F h_{st} normalized. . . . . . . . . . . . . . . . . 52 3.6 Neural Network architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.7 ANN overall performance for the diverse input variables. . . . . . . . . . . . 54 3.8 Best ANN testing confusion matrix using the tangential forces in the trans-

verse plane as input for the network. . . . . . . . . . . . . . . . . . . . . . . 54 3.9 Best ANN testing confusion matrix using both tangential forces as input

for the network. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 3.10 ELM training and testing classification performance. . . . . . . . . . . . . . 55 3.11 ELM testing confusion matrix using both tangential forces as input for the

network. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 3.12 ANN testing performance for different percentage of step data points. … 56

4.1 ITshoe schematic structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 4.2 Layout for data collection with four different floors. . . . . . . . . . . . . . . 63 4.3 Illustration of the 50 chosen samples of the normal force of a robot step to

be used on the classification of the robot walking floor. . . . . . . . . . . . . 66 4.4 LSTM cell structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 4.5 Dropout behaviour. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 4.6 Optimized LSTM model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.7 LSTM model, Accuracy vs Epochs. . . . . . . . . . . . . . . . . . . . . . . . 68 4.8 Confusion matrix of the testing dataset. . . . . . . . . . . . . . . . . . . . . 69 4.9 Layout for the real-time on-line experiment. . . . . . . . . . . . . . . . . . . 69 4.10 Individual results on floor classification of NAO walking experiment in real-

time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 4.11 Confusion matrix of the real-time experiment. . . . . . . . . . . . . . . . . . 71 4.12 LSTM network testing accuracy using different sizes of timesteps. . . . . . 71 4.13 Confusion matrix of the testing dataset using 40 timesteps. . . . . . . . . . 72 4.14 Confusion matrix of the testing dataset using 30 timesteps. . . . . . . . . . 72 4.15 LSTM average test accuracy. . . . . . . . . . . . . . . . . . . . . . . . . . . 73

5.1 Modified humanoid 3-layer control architecture with an additional EYC block. 81 5.2 Illustration of the NAO robot’s inability to maintain a straight path. . . . . 83 5.3 Visualizing humanoid locomotion behavior: trajectories illustrating straight

walking command execution . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 5.4 ITshoe schematic structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 5.5 Layout of the simulated environment featuring the NAO robot and various

types of slippery floors in the Gazebo simulator. . . . . . . . . . . . . . . . 87 5.6 Absolute centre of mass position in the Y-axis for three exemplary walking

trials on a flat surface resulting in the least amount of slip . . . . . . . . . . 91 5.7 Absolute centre of mass position in the Y-axis for three exemplary walking

trials on a flat surface resulting in the highest amount of slip . . . . . . . . 91 5.8 Slip and threshold visualization for the top three experiences with the least

amount of slip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 5.9 Slip and threshold visualization for the top three experiences with the high-

est amount of slip. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 5.10 Comparison of slip sum for different step frequencies and slippery floors… 93 5.11 Experiment proportional gain impact on the task performance. . . . . . . . 95 5.12 Impact of integral gain on task performance. . . . . . . . . . . . . . . . . . 95 5.13 Experiment derivative gain impact on the task performance. . . . . . . . . . 96 5.14 Comparison of absolute centre of mass position in the Y-axis using an adap-

tive PD controller vs non-variable PD . . . . . . . . . . . . . . . . . . . . . 97

List of tables

2.1 Data upload rate percentage loss for both ITshoes. . . . . . . . . . . . . . . 30

3.1 Shoe soles coefficient of friction. . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.2 Dimensions of the input and target matrices . . . . . . . . . . . . . . . . . . 44

4.1 Floor-Shoe coefficient of friction . . . . . . . . . . . . . . . . . . . . . . . . . 64 4.2 Dimensions of the input and target matrices . . . . . . . . . . . . . . . . . . 65 4.3 Range of network parameters evaluated. . . . . . . . . . . . . . . . . . . . . 67 4.4 LSTM model Classification report. . . . . . . . . . . . . . . . . . . . . . . . 68

5.1 Points adapted in the original walking controller to fit the NAO requirements. 85 5.2 Best PID gains for each floor . . . . . . . . . . . . . . . . . . . . . . . . . . 96 5.3 Experiment results by floor and overall experience, comparing the perfor-

mance of the robot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

List of abbreviations

ANN . . . . . . . . . . . Artificial Neural Network CITs . . . . . . . . . . . Computational Intelligence Techniques CoF . . . . . . . . . . . Coefficient of Friction CoM . . . . . . . . . . . Center of Mass CoP . . . . . . . . . . . Center of Pressure DCM . . . . . . . . . . Divergent Component of Motion ELM . . . . . . . . . . . Extreme Learning Machine EYC . . . . . . . . . . . Embedded Yaw Controller F/T . . . . . . . . . . . Force Torque FSRs . . . . . . . . . . Force Sensitive Resistors hGRFs . . . . . . . . . horizontal Ground Reaction Forces IMUs . . . . . . . . . . Inertial Measurement Units ITshoes . . . . . . . . . Instrumented Shoes LIPM . . . . . . . . . . Linear Inverted Pendulum Model LSTM . . . . . . . . . . Long Short-Term Memory MPC . . . . . . . . . . Model Predictive Control PID . . . . . . . . . . . Proportional-Integral-Derivative RNNs . . . . . . . . . . Recurrent Neural Networks SLIP . . . . . . . . . . . Spring-loaded Inverted Pendulum SVM . . . . . . . . . . . Support Vector Machine TMLIP . . . . . . . . . Two-Mass Linear Inverted Pendulum vGRFs . . . . . . . . . . vertical Ground Reaction Forces WBC . . . . . . . . . . Whole-Body Control ZMP . . . . . . . . . . . Zero Moment Point

Chapter 1

General Introduction

In the multifaceted world of robotics, engineers and researchers face a crucial challenge:

To develop a robust and efficient locomotion system for humanoid robots,

enabling them to seamlessly navigate and adapt to the intricate and ever-evolving nature of complex environments.

These robots, designed to mimic the human form, strive to replicate our remarkable abilities for movement and adaptability. However, achieving a robotic equivalent of human locomotion is a complex task, requiring advanced technical knowledge, innovative design approaches, and the application of robust mathematical models kajita2014introduction ; vukobratovic2004zero .

The ability for a robot to adapt dynamically to a variety of terrains and situations is a critical capability. Just as humans do, humanoid robots need to adjust their locomotion patterns to navigate different ground surfaces and overcome obstacles. This challenge becomes even more significant considering the diverse tasks humanoid robots are assigned to, ranging from assisting the elderly in their homes to aiding in disaster recovery missions or exploring uncharted environments on Earth or even beyond Xie_2020 .

At the heart of addressing this challenge lies the understanding and leverage of groundfoot reaction forces, the forces exerted by the ground on the robot’s foot during locomotion. As humans, we instinctively adjust our walking pattern based on these forces, enabling us to traverse surfaces such as sand or ice with equal ease. By obtaining a detailed understanding of these forces within the context of humanoid robots, it is possible to design advanced control algorithms that can dynamically adjust the robot’s walking pattern in real-time.

Chapter 1. General Introduction

In the pursuit of this understanding, machine learning plays a significant role. By employing machine learning algorithms, robots extract meaningful information from sensory data, including visual images, depth maps, and force/torque measurements. This data-driven approach allows robots to perceive and interpret subtle nuances in their environment, enabling them to uncover patterns and correlations that contribute to stable walking patterns. By harnessing this knowledge, it becomes possible to design or refine advanced control algorithms that can dynamically adapt the robot’s walking pattern in real-time, utilizing the insights gained from the ground-foot reaction forces.

In the context of the broader challenge, this PhD thesis research undertakes a specific challenge, namely:

Enhancing Bipedal Locomotion on Slippery Environments.

Navigating and adapting to slippery surfaces present unique difficulties for humanoid robots, requiring specialized control strategies and mechanisms to maintain stability and ensure efficient locomotion. The objective of this research is to explore innovative approaches and techniques that can enhance the ability of humanoid robots to traverse and maneuver on slippery terrains, contributing to advancements in the field of bipedal locomotion. Figure 1.1 visually encapsulates the essence of this research objective. It illustrates the challenge faced by the humanoid robot as it struggles to maintain a straight path on increasingly slippery surfaces. Furthermore, this slipperiness poses a significant risk of causing the robot to fall, underscoring the critical importance of addressing this issue.

1.1 Hypotheses and Approach

To address this challenge, this research proposes three interconnected hypotheses, each posing a specific question:

• Hypothesis 1: Can we create a wearable, sensor-equipped shoe for humanoid robots that unobtrusively measures ground reaction forces, exhibits universal adaptability across different robot designs, and can be seamlessly integrated without compromising the robot’s locomotion?

• Hypothesis 2: Can we utilize machine learning techniques to effectively distinguish between various types of slippery surfaces, leveraging the normal and tangential ground reaction forces data obtained from the wearable, sensor-equipped shoe proposed in Hypothesis 1?

• Hypothesis 3: Can we enhance the robot’s control system by utilizing the floor-type classification obtained through machine learning techniques, bolstering the walking performance and ensuring the robot adheres to a specified trajectory, regardless of the slipperiness of the ground?

Each hypothesis builds upon the previous one, forming a comprehensive research framework (as depicted in Figure 1.2) that addresses the intricacies of humanoid locomotion and Figure 1.1: Illustration depicting a humanoid robot’s locomotion on various surfaces with progressively reduced coefficients of friction, without slip control and with control. The performance can be assessed by measuring the deviation from the path’s mid-line across the four experiments. In the first three without control, noticeable path deviations are evident, while in the fourth, where control is incorporated, deviations are negligible.

adaptability on slippery floors. Through the exploration of these hypotheses, this research endeavours to deepen our understanding and equip humanoid robots with the capability to attain stable walking patterns while maximizing efficiency.

To achieve these objectives, the research will involve a rigorous analysis of scientific papers to identify existing knowledge and gaps in the field. The construction of a wearable system will be undertaken to unobtrusively measure total ground reaction forces and ensure adaptability across different robot designs. Furthermore, machine learning algo-Figure 1.2: Schematic illustration of the research framework for enhancing humanoid robot locomotion on slippery surfaces.

rithms will be explored to classify various categories of slippery surfaces based on the ground reaction force data captured by the wearable system. Finally, the control system of the robot will be enhanced using the floor-type classification obtained through machine learning techniques to improve walking performance and ensure adherence to a specified trajectory.

By addressing these hypotheses and employing a multidisciplinary approach that combines theoretical analysis, experimental evaluations, and technological advancements, this work aims to unlock new possibilities in humanoid robotics. The outcomes have the potential to significantly contribute to the adaptability of humanoid robots on challenging floors, specifically slippery surfaces, enabling them to navigate such environments with enhanced stability, efficiency, and adaptability.

1.2 Fundamental concepts of humanoid stability and locomotion

Humanoid stability and locomotion play a foundational role in robotics, driven by several critical factors. First and foremost, stability is of utmost importance to ensure the safety and integrity of humanoid robots, preventing falls and potential damage in complex environments. Efficient locomotion enhances overall performance, enabling robots to conserve energy, increase endurance, and minimize power consumption. Additionally, humanoid stability and locomotion contribute to adaptability, empowering robots to navigate diverse terrains, overcome obstacles, and engage with objects effectively. This is pivotal if we aspire for robots to effectively and safely coexist alongside humans.

At the core of these principles lie key concepts that shape our understanding of humanoid stability and locomotion. The ZMP is a critical parameter representing the point on the ground where the net moment of inertial and gravity forces is zero, essential for achieving balance control. By carefully controlling the motion of the ZMP and ensuring it stays within the support polygon, humanoid robots can effectively resist external disturbances and maintain a stable base of support. The support polygon refers to the convex polygon formed by the contact points of the robot’s feet with the ground. It defines the region on the ground that provides support to the robot’s body. By ensuring that the ZMP remains within this polygon, the robot can maintain its stability and balance vukobratovic2004zero . The Center of Mass (CoM) represents the average position of the robot’s mass, playing a significant role in locomotion dynamics. The Center of Pressure (CoP), on the other hand, represents the distribution of forces exerted by the robot’s contact points with the ground. It provides information about the stability and balance of the robot during locomotion and can be used to estimate the ZMP. Miomir Vukobratović and Davor Juričić introduced the concept of ZMP in their pioneering work on biped gait synthesis and mathematical modeling. Their research laid the foundation for ZMP-based control strategies, shaping the development of stable walking patterns for humanoid robots vukobratovic1969contribution ; juricic1972mathematical . The Linear Inverted Pendulum Model (LIPM) simplifies CoM dynamics, providing valuable insights into balance control and facilitating the development of stable locomotion algorithms kajita20013d .

Figure 1.3 provides a visual representation of a humanoid robot in a single-support phase of its walking cycle. The figure shows the robot balancing on one foot, with the other foot suspended in air. The LIPM is superimposed on the robot’s structure, highlighting how the robot’s CoM projects down to the ground, forming the base of the pendulum. In an ideal scenario, the ZMP aligns with the center of the foot in contact with the ground, effectively serving as the point about which the entire system balances (note that foot slip is not considered).

Figure 1.3: Illustration of the linear inverted pendulum model and ideal representation of the ZMP inside the support polygon during a single-support phase of a walking humanoid.

Balance control emerges as a critical aspect, requiring continuous adjustments in posture, foot placement, and CoM position to respond to external disturbances. The main objective of the control in a walking humanoid is to guarantee zero moment point and body posture to keep the dynamic balance ott2011posture . The key issue in the matter is to look for a fast algorithm that allows a control even if the robot is subjected to impulsive disturbances caused by external force or unknown ground.

Gait generation focuses on generating coordinated leg movements that optimize efficiency and stability during locomotion. It involves designing algorithms that determine the trajectory and timing of leg movements to achieve stable and efficient locomotion. Gait generation algorithms take into account factors such as step length, step width, walking speed, and terrain conditions to adapt the walking pattern accordingly. These algorithms may be rule-based or employ optimization techniques to find optimal gait parameters kramer_review_2010 .

The Whole-Body Control (WBC) unit coordinates multiple body parts, including arms, legs, and torso, to accomplish tasks while preserving balance and adapting to dynamic environments. Whole-body control algorithms consider the dynamics and kinematics of the robot, along with the desired task objectives and constraints. This field of research aims to develop control strategies that enable humanoid robots to manipulate objects, perform dexterous tasks, interact with humans, and navigate through complex environments. Recent advancements in whole-body control have explored the integration of force and tactile sensing, as well as compliance control, to enhance the robot’s ability to interact and collaborate with the surrounding world huang2023task .

Trajectory planning also plays a vital role by generating smooth and feasible motion paths for limbs or the entire body, considering physical limitations, task objectives, and obstacle avoidance.

Advancements in these concepts have fuelled comprehensive research, leading to improved stability, adaptability, and performance of humanoid robots. Researchers and engineers leverage these principles to develop increasingly capable and versatile humanoid systems capable of navigating challenging environments, interacting with objects, and providing assistance in human rehabilitation contexts. By embracing and refining these concepts, the field continues to push the boundaries of humanoid stability and locomotion, opening new possibilities for real-world applications and human-robot interactions.

1.3 Instrumented systems used in the field of humanoid locomotion

Much like humans who rely on their senses to adapt while walking, instrumented systems play a central and indispensable role in the field of humanoid locomotion, providing valuable data and insights into the forces and movements involved in walking and maintaining stability. These systems allow to accurately measure and analyse various parameters, contributing to a better understanding and optimization of humanoid locomotion.

One important aspect of instrumented systems used in humanoid locomotion is the measurement of ground reaction forces. These forces result from the interaction between the robot’s feet and the supporting surface. Force Torque (F/T) sensors are commonly employed to measure these forces, capturing both their magnitude and direction. By integrating F/T sensors into the robot’s feet or lower limbs, precise real-time measurements can be obtained, facilitating a comprehensive understanding of the interaction between the robot and the ground. On the other hand, flexible force sensors, such as flexiforce sensors, are also used to measure and analyse the ground reaction forces. These thin and flexible sensors not only detect pressure but also provide information on the magnitude and distribution of forces exerted on the robot’s feet. By strategically placing flexiforce sensors in various regions of the robot’s feet, researchers can gain comprehensive insights into the ground reaction forces during walking, thereby enhancing their understanding of stability and balance control in humanoid locomotion. While F/T sensors provide comprehensive data, they can often be cost-prohibitive and may pose challenges in terms of integration. In response to these challenges, an alternative approach utilizing strategically positioned flexiforce sensors emerges as a practical solution. As depicted in Fig. 1.4, the placement of these sensors at specific angles, such as at 0 degrees for vertical alignment and 45 degrees for horizontal alignment, allows for a measurement of both the vertical and horizontal components of ground reaction forces.

Figure 1.4: Schematic representation of strategic placement of FlexiForce sensors at 0 and 45 degrees for measurement of vertical and horizontal components of ground reaction forces.

The 0-degree flexiforce sensors allows for the measurement of vertical forces exerted by the robot’s feet on the ground. This information proves vital in assessing weight distribution and overall stability during locomotion. On the other hand, the 45-degree flexiforce sensor captures measurements of horizontal forces exerted on the robot’s feet. These forces are crucial for evaluating the friction and traction between the robot and the ground, especially in challenging conditions such as slippery surfaces. Precise measurements of horizontal forces enable us to assess the robot’s grip and make appropriate adjustments to ensure stable locomotion across a wide range of terrains.

Moreover, flexiforce sensors offer several advantages over F/T sensors, including costeffectiveness and ease of integration. These sensors are thin and flexible, allowing for effortless positioning on the robot’s feet without requiring significant modifications to the robot’s structure. This versatility makes them a practical solution for acquiring force measurements in humanoid locomotion research and applications. Despite their advantages, it is important to note that flexiforce sensors may require more careful use and maintenance compared to F/T sensors.

Furthermore, Inertial Measurement Units (IMUs) are employed in instrumented systems to monitor the orientation and movement of humanoid robots. IMUs consist of accelerometers, gyroscopes, and sometimes magnetometers, collectively providing information on the robot’s acceleration, angular velocity, and orientation in three-dimensional space. Integrating IMUs into the robot’s body allow to track the robot’s movements, detect deviations from the desired path, and make necessary adjustments to maintain stable locomotion.

Figure 1.5 shows an example of a F/T, flexiforce and IMUs sensor. The amalgamation of F/T sensors, flexiforce sensors, and IMUs within instrumented systems facilitates comprehensive measurements of ground reaction forces, pressure distribution, and robot orientation. The sensors mentioned above are essential for the development of humanoid

(a) Force torque sensor.  (b) Flexiforce sensor.  (c) Inertial measurement unit.

Figure 1.5: Illustration of common sensing systems employed in humanoid locomotion for accurate measurement and control.

robots, and they have been used in the development of almost every humanoid robot, including Atlas, iCub, and NAO.

While other sensors, such as cameras and LiDAR, are also important, they are not the focus of this work.

1.4 The Significance of Computational Intelligent Techniques in Humanoid Robotics

Computational intelligent techniques play a central role in extracting valuable insights from data across various domains, and their significance is particularly pronounced in the field of humanoid robotics. These techniques, driven by advanced computational algorithms and models, have the ability to analyse and interpret complex sensor data, enabling humanoid robots to make informed decisions and adapt their locomotion strategies accordingly.

In the realm of humanoid robotics, computational intelligent techniques have transformative applications. They empower robots to process sensory information from multiple sources, such as cameras, force sensors, and IMUs sensors, to gain a comprehensive understanding of their environment. By leveraging this data, humanoid robots can navigate and interact with their surroundings in a more intuitive and adaptive manner. When it comes to locomotion on different types of floors, such as surfaces with varying coefficients of friction, computational intelligent techniques become even more essential. The accurate interpretation of data related to ground reaction forces, IMUs readings, and other relevant parameters becomes crucial for maintaining stability, preventing slips and falls, and optimizing locomotion performance.

By analysing foot contact forces, computational intelligent techniques provide humanoid robots with valuable information about the interaction between their feet and the surface. This data allows them to adjust their gait, joint torques, or control actions to ensure optimal stability and balance on surfaces with different frictional properties. For example, on slippery floors with low coefficients of friction, the robot can adapt its movements to reduce the risk of slipping, while on surfaces with high coefficients of friction, it can optimize its foot placement to maximize traction and efficiency.

Moreover, computational intelligent techniques enable real-time analysis of IMUs data, which provides information about the robot’s orientation, acceleration, and angular velocity. By monitoring these parameters, humanoid robots can detect changes in the surface conditions and dynamically adjust their locomotion strategies. For instance, if the coefficient of friction suddenly decreases on a previously stable surface, the robot can react swiftly by modifying its joint torques or adopting a different walking pattern to maintain balance and prevent falls.

1.5 Advancements in Humanoid Control

Humanoid robots, designed to mimic human form and behaviour, require sophisticated control systems to achieve stable and efficient locomotion. The field of humanoid robotics has seen significant advancements in the development of controllers that enable these robots to perform complex tasks, navigate diverse environments, and adapt to changing conditions. Humanoid controllers serve as the underlying framework that orchestrates the robot’s movements, ensuring coordination, balance, and responsiveness.

At the core of humanoid controllers is the principle of dynamic stability. Unlike static stability, which focuses on maintaining balance while stationary, dynamic stability refers to the ability of a robot to maintain balance while in motion. Achieving dynamic stability in humanoid locomotion is essential for navigating challenging terrains, performing agile movements, and recovering from external disturbances. Humanoid controllers employ advanced algorithms and control strategies to enable the robot to dynamically adjust its posture, joint angles, and motor torques to maintain stability.

There are various approaches to designing humanoid controllers, each with its own strengths and limitations. One common approach is based on the concept of whole-body control, which considers the robot as a unified system rather than a collection of individual joints and limbs. Whole-body control aims to coordinate the movements of all the robot’s body parts to achieve stable and coordinated motion. This approach often incorporates dynamic modelling, optimization techniques, and feedback control to generate desired trajectories and achieve stability sentis2006whole .

Another approach is behaviour-based control, which focuses on organizing the robot’s actions into modular behaviours or skills. These behaviours can be designed individually and then combined to generate complex movements and tasks. Behaviour-based controllers provide a flexible and modular framework, allowing for easy adaptation and reconfiguration of the robot’s actions based on the task and environment katic2003survey .

Furthermore, humanoid controllers can incorporate machine learning techniques to enhance their adaptability and responsiveness. Machine learning algorithms can be used to learn from sensory data and optimize control policies, enabling the robot to adapt its movements based on real-time feedback and environmental conditions.

Several notable examples highlight the capabilities and advancements in humanoid controllers. One example is the ZMP controller, which utilizes the concept of the ZMP to achieve dynamic stability sayari2019perfect . The ZMP controller calculates the point on the ground where the total moments acting on the robot’s body are balanced, ensuring stability during walking and other locomotion tasks.

Another example is the Model Predictive Control (MPC) approach, which uses dynamic models of the robot and the environment to predict future states and optimize control actions accordingly koenemann2015whole . MPC controllers enable real-time adjustment of control parameters and generate robust and stable locomotion patterns.

Additionally, bio-inspired controllers draw inspiration from biological systems and principles to achieve humanoid locomotion. These controllers aim to replicate the neural and musculoskeletal structures found in humans, allowing for more natural and human-like movements colasanto2015bio .

These examples illustrate the diverse approaches and techniques employed in humanoid controllers to achieve stable and adaptive locomotion. By combining principles of dynamic stability, whole-body control, behaviour-based control, and incorporating machine learning, humanoid controllers continue to advance the capabilities of humanoid robots, enabling them to perform complex tasks, navigate challenging environments, and interact with the world in a more human-like manner.

1.6 Humanoid Locomotion in Slippery Environments

This section offers an in-depth analysis of existing studies concerning humanoid locomotion across unstructured environments, with an emphasis on navigating slippery terrains. The robotic capacity to traverse unregulated or unpredictable environments is vital for their practical utility in real-world contexts chen2020robust . The present review dissects the progress, challenges, and ongoing research covering multiple dimensions of humanoid movement in such conditions. It encompasses aspects like deciphering and manipulating ground reaction forces, harnessing learning techniques to interpret and extract meaningful patterns from such data, and the creation of advanced controllers to refine humanoid mobility.

It’s noteworthy that this PhD thesis includes dedicated literature reviews within each chapter, supplementing the comprehension of various factors contributing to the enhancement of humanoid locomotion. These reviews, together, furnish a thorough understanding of the discipline and its developments, offering useful perspectives for the research presented in this thesis.

Over the past decade, impressive improvement have been made in establishing resilient bipedal robot locomotion. This advancement has been steered by the synergistic blend of theoretical modeling and analysis via hybrid systems, application of sophisticated nonlinear control methodologies, and meticulous consideration towards mechanical design and hardware implementation ma2019dynamic .

Bipedal locomotion, as grounded in paradigms like ZMP vukobratovic2004zero and spring-loaded inverted pendulum Spring-loaded Inverted Pendulum (SLIP) models schwind1998spring , relies on a fundamental assumption of zero foot slippage. In this framework, the robot foot is regarded as an immovable pivot point, functioning effectively within the controlled environment of a laboratory with adequate floor friction. However, this approach falls short when faced with the complexities of natural outdoor terrains, which often present a medley of surfaces characterized by slipperiness, slight unevenness, or irregularities.

In the context of humanoid robots, the issue of foot slippage is typically treated either as an external disturbance to be preemptively addressed during gait planning kajita2004biped or as a dynamic challenge necessitating real-time detection and recovery through feedback control vazquez2013experimental .

Enhancing the locomotion of humanoid robots involves a continuous sequence of interactions between the robot’s feet and its environment. Within this context, the accurate detection of foot contacts assumes a pivotal role in several aspects of locomotion control koolen2016design ; herzog2016momentum ; neunert2018whole , including gait planning aceituno2018simultaneous ; winkler2018gait ; hereid2018dynamic , base state estimation bloesch2012state ; rotella2014state ; raghavan2018study ; piperakis2019outlier , and Center of Mass (CoM) estimation rotella2015humanoid ; piperakis2016nonlinear ; piperakis2018nonlinear . Consequently, to achieve truly agile and dexterous locomotion, it becomes imperative to achieve precise contact status estimation.

In the realm of contact detection, current methods can be broadly categorized into two primary groups:

• Direct Measurement Approaches

These approaches directly utilize measurements of ground reaction wrenches. In the literature, researchers have employed various algorithms for the precise analysis of GRFs. These include the utilization of F/T sensors fallon2014drift ; kuindersma2016optimization , contact/pressure sensors piperakis2018nonlinear ; bloesch2013state , IMUs sensors maravgakis2023probabilistic or a combination of F/T sensors with IMUs data lin2021legged .

• Kinematics and Dynamics-Based Approaches

These methods leverage principles of kinematics and dynamics to estimate the GRFs, which are then used to infer the contact status. In the literature, authors estimate the GRFs by capitalizing on the kinematics and dynamics of the robots, each employing different methods for estimation ortenzi2016kinematics ; hwangbo2016probabilistic ; neunert2017trajectory ; camurri2017probabilistic ; lin2021legged .

Various methodologies are employed to decipher GRFs and estimate the contact between the robot and the floor. These methods encompass a range of techniques, including Schmitt-Trigger fallon2014drift ; piperakis2018nonlinear ; kuindersma2015optimization , Kalman filtering bloesch2012state ; bloesch2013state , clustering using fuzzy c-means algorithms rotella2018unsupervised , probabilistic frameworks incorporating hidden Markov models hwangbo2016probabilistic , deep learning-based approaches lin2021legged ; piperakis2022robust , and unsupervised learning frameworks rotella2018unsupervised ; hoepflinger2013unsupervised .

In this study, we acknowledge the temporal evolution within the field, particularly the growing trend towards machine learning techniques. Subsequently, we harness the potential of neural networks for performing this classification, as detailed in the following sections of this thesis.

Comprehensive investigations into slip dynamics are relatively limited in the current research landscape. The majority of existing studies pertaining to low-friction foot contact primarily concentrate on preemptive slip avoidance ferreira2020humanlike ; khadiv2017optimal . Explicit examinations of slipping dynamics are predominantly oriented towards the domain of human locomotion mihalec2022balance ; trkov2019bipedal ; chen2015robotic , with a few noteworthy exceptions.

In mihalec2023balance , the authors presented a motion planning methodology that hinges on a two-mass Linear Inverted Pendulum (Two-Mass Linear Inverted Pendulum (TMLIP)) model, in conjunction with a frictional Whole-Body Operational Space (FWBOS) full-body recovery control strategy, visually depicted in Figure 1.6 and Figure 1.7. Specifically, Figure 1.6 illustrates the schematic representation of the TMLIP model and the utilized full-body model, while Figure 1.7 offers insight into the control diagram employed. This strategy aimed to calculate joint torques to counteract the effects of slip dynamics during locomotion. Notably, when the robot operated on surfaces with extremely low friction, it encountered challenges in maintaining gait balance. However, the authors asserted that, despite these challenges, their controller exhibited superior performance when compared to conventional controllers. Specifically, it outperformed them in terms of maintaining balance over time and achieving greater walking distances.

In radosavovic2023learning the authors devised a sim-to-real learning based method Figure 1.6: Left: schematic of the TMLIP model. Center: bipedal robotic walker. Right: schematic of a 5-link full body model mihalec2023balance .

for enabling real-world humanoid locomotion. The core component of their controller is a causal Transformer, which underwent training through autoregressive prediction of forthcoming actions based on a historical record of observations and actions, as visually illustrated in Figure 1.8. This approach underwent reinforcement learning training within a simulation environment and was subsequently deployed without any further adaptation ("zero-shot"). Their model underwent comprehensive evaluation within a high-fidelity simulation environment and showcased remarkable success in deployment across various types of floors. Notably, it faced challenges in the most demanding scenario, characterized by a surface with significantly lower friction, and did not excel in this particular case.

In lee2022adaptive the authors introduced strategies for stabilizing slippery motion within a Whole-Body Control (WBLC) framework, focusing on known and unknown slip scenarios. They used CoM re-planning based on parameter estimation and online weight adaptation to address unknown slip, improving stability. While effective in simulations, potential challenges include traction loss, parameter estimation inaccuracies, and instability, requiring further investigation. Real-world implementation remains untested, presenting additional hurdles like kinematic singularities and dynamics errors that need to be addressed in future research.

Figure 1.7: Control diagram for walking. Slip changes the results of TMLIP and FWBOS calculations mihalec2023balance .

For instance, in frizza2022study , an alternative perspective is offered, wherein the focus shifts towards modifying the robot’s feet rather than altering the robot’s gait or controller. They conducted experiments utilizing a compliant sole with variable stiffness (Figure 1.9), which resulted in notable improvements compared to the robot’s flat feet with fixed stiffness.

Despite the different approaches explored in this context, it is evident that slip dynamics remain a challenging problem with limited research efforts, stressing the need for further attention and exploration.

1.7 Brief Research Journey

My research journey began at the Institute of Electronics and Informatics Engineering of Aveiro (IEETA) and the Laboratory for Automation and Robotics (LAR) housed within the Mechanical Department of Aveiro University. This journey commenced with the creation of an innovative instrumented system, aptly named ITShoe, which was carefully developed for the precise measurement of GRFs experienced by a walking humanoid on a diverse array of surfaces. Notably, our focus lay on those surfaces characterized by low friction coefficients. We are sincerely grateful for the support provided from the Fundação Figure 1.8: Causal Transformer model trained by autoregressive prediction of future actions from the history of observations and actions. radosavovic2023learning .

para a Ciência e Tecnologia (FCT) through their funding of the project titled ’Automatic Adaptation of Humanoid Robot Gait to Different Floor-Robot Friction Coefficients,’ with the project reference number PTDC/EEI-AUT/5141/2014.

This inaugural phase of my research represented a holistic undertaking, encompassing the entire spectrum of activities involved in the development of the ITShoe, from conceptualization and design to material selection, construction, and the intricacies of setting up electronics. The overarching goal was to ensure the seamless and real-time measurement and transmission of force data. This pioneering phase culminated in the collection of valuable data using the robot+ITShoe architecture.

Subsequently, I seized the opportunity to further expand my horizons by joining the esteemed Institute Mihailo Pupin in Belgrade, Serbia. Here, I ventured into the field of

(a) Simplified model of the complaint foot.

(b) Deformable foot model used in dynamic simulations.

Figure 1.9: Variable stiffness feet for humanoid walking. The plantar arc is composed of ten blocks representing the ten phalanxes, connected between each other by rotational joints with viscoelastic elements. The plantar arch is connected with the frontal and backward arch through rotational passive joints frizza2022study .

machine learning, exploring a variety of tools and algorithms, with the objective of finding the most efficient and accurate approach to interpret the force data. My progress was guided and enhanced by the expertise of my dedicated supervisor at the institute. During my three-month stay at the Institute Mihailo Pupin, I actively participated in an ongoing research project focused on the adaptation of the NAO humanoid robot’s gait. Moreover, I had the opportunity to contribute to the scientific paper gui, which significantly enriched my research experience.

Upon returning to the LAR, the research journey continued to evolve as various computational intelligence techniques were explored, aiming to refine the data interpretation process. Progressive enhancements were introduced to both data gathering and analysis methodologies. Once the capability to classify different floors in real-time was achieved with the actual robot, a new chapter in the research journey was embarked upon.

I joined the Robotic Laboratory at Carlos III University of Madrid, and for four months, my focus shifted towards advancing the control mechanisms of humanoid robots. Here, collaboration with a different humanoid robot, TEO, provided valuable experience in both simulated and real-world environments.

Ultimately, my path led me back to Aveiro University, where the improved robot control system was integrated with the humanoid robot that had been a constant companion throughout the thesis. I owe a debt of gratitude to ISR for their support and provision of the robot for the duration of my research. By incorporating the controller into a simulated environment, we successfully unified all the work undertaken thus far. Moreover, we developed a controller that dynamically adapted its parameters based on the classification of the walking surface, thus enhancing the humanoid’s gait on slippery floors.

1.8 Thesis Organization

This thesis focuses on improving the adaptability of humanoid locomotion on slippery floors. The primary objective is to acquire real-time knowledge of these challenging surfaces in order to enable effective control of humanoid robots and ensure their successful navigation through such obstacles. This goal is achieved by addressing the initial hypotheses presented.

The thesis is structured into several chapters, including a general introduction (Chapter 1), one conference paper and three journal papers (Chapters 2-5), and a comprehensive discussion encompassing concluding remarks and future work (Chapter 6).

This thesis is structured into three interrelated parts that collectively tackle the challenge of enhancing humanoid locomotion adaptability on slippery floors:

• Part I (Chapter 2): This part is dedicated to the development of a wearable 3D force system known as Instrumented Shoes (ITshoes). The primary objective is to create a robust and versatile instrument that can accurately measure the ground reaction forces experienced by a walking humanoid robot. The ITshoes are designed to seamlessly integrate with various robot designs, although the manufactured units are specifically tailored for the NAO humanoid, while minimizing any disruption to the robot’s locomotion. By precisely capturing the ground reaction forces, this system lays the foundation for subsequent analysis and control strategies.

• Part II (Chapters 3-4): Building upon the data obtained from the ITshoes, this part focuses on the application of machine learning algorithms to address the challenge of classifying different types of slippery floors. The goal is to leverage the ground reaction force data collected by the ITshoes to train and deploy machine learning models capable of accurately identifying and categorizing various slippery surface conditions. This classification system provides valuable real-time information about the ground conditions, empowering the robot to make informed decisions and adapt its locomotion accordingly.

• Part III (Chapter 5): The emphasis of this part lies in enhancing the humanoid controller by leveraging the real-time knowledge of the floor conditions obtained from Part II. By incorporating the classified floor types into the control system, improvements can be made to optimize the robot’s locomotion and ensure its adherence to the specified task. The control system is designed to dynamically adjust the robot’s gait, balance, and motion planning based on the identified floor condition. These enhancements aim to maximize walking stability, efficiency, and adaptability, particularly when navigating slippery surfaces.

The chapters (Chapters 2-5) in this thesis encompass the research conducted during the PhD program and adhere to a coherent structure. Each chapter of this thesis corresponds to a scientific paper that has already been published, with the exception of chapter 5, which is currently under submission. As a result, every chapter includes an abstract, an introductory section that offers a comprehensive review of the state-of-the-art, details on the methodology employed (which may also be referred to as ’materials and methods’), presentation of the obtained results, discussion of the findings, and concluding remarks. Chapters in Parts I and II are supported by extensive experimental data collected from the physical system, providing a robust foundation for the research. In contrast, Part III, while featuring some physical developments, primarily relies on simulation due to hardware limitations in controlling the actual robot.

Part I

3D FORCE SYSTEM FOR GROUND REACTION FORCES ANALYSIS IN HUMANOIDS A Novel Wireless Instrumented System for Ground Reaction Forces Analysis in Humanoids

Published in Almeida L, Santos V, Silva F

IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC),

2018

DOI: 10.1109/ICARSC.2018.8374157

Chapter 2. A Novel Wireless Instrumented System for Ground Reaction Forces Analysis in Humanoids

2.1 Abstract

The measurement of ground reaction forces (GRFs) is crucial in the biomechanical analysis of gait and other motor activities. Current humanoid robots mimic some of the biological adaptation methods found in humans. Therefore, studying the GRFs in these robots allows not only to improve their overall performance, but also to extrapolate possible techniques for human walking rehabilitation. The balance of a humanoid robot may be compromised when it walks and, even more, when it walks across multiple grounds with different frictions. This paper presents a system to be seamlessly installed on a walking humanoid robot to measure normal and tangential ground reaction forces. The proposed solution is a cost-effective, lightweight and wirelessly instrumented shoe (ITshoe) for real-time measuring of the GRFs.

2.2 Introduction

Walking robots compete to mimic some of the biological adaptation methods found in humans. Studying these humanoid robots grants the possibility to deduce methods for human walking rehabilitation. During walking, the impulsive force between the stance foot and the ground may affect the stability and reliability of biped robots seriously. Bearing in mind the need to reach a properly and safely humanoid robot locomotion, with a precise and robust control, it is crucial to measure the contact force between its feet and the ground.

This work is part of a project entitled "Automatic adaptation of an humanoid robot gait to different floor-robot friction coefficients", that aims to achieve adaptive humanoid robot walking through several types of floors. Following this idea, the principal aim of this work is the development of a system capable of detecting and analysing the complete ground reaction forces (GRFs). The major requirements for such a system are small weight and size, self-contained and no impediment of functional mobility.

The current work presents a novel instrumented shoe (ITshoe) capable of measuring the three components of the GRFs. The solution detects and wirelessly transmits the GRFs in real-time, it is customizable for different robots, it allows more parameters than integrated solutions in robots, and it can be easily installed independently of the robot control. The ITshoe is designed to be used with the humanoid robot NAO and has two main parts: Outer Shoe (OSh) and Inner Shoe (ISh). The OSh is the instrumented part of the shoe, whereas the ISh is the link between the robot’s foot and the OSh.

The remainder of the paper is divided as follows. Section II presents a review of modern developments connected to this work, such as commonly used sensors and existing instrumented systems. The developed prototype is presented in Section III. The system’s architecture is outlined in Section IV. GRFs experiments and results are discussed in Section V. Finally, Section VI presents the conclusions and future challenges.

2.3 Related Work

Multiple systems have been developed for GRFs estimation, being commonly used in two major fields: robotics and biomechanics. Most of today’s humanoid robots are equipped with force sensing capabilities at the feet for stability purposes, based on commercial or custom-made solutions. Besides being expensive, commercial solutions adopted are tailored to specific needs in which the main concern is to provide an accurate estimate of the robot’s center-of-pressure (CoP). Kim et al. b1 developed an intelligent foot for a humanoid robot using two manufactured six-axis force/moment sensors (twelve sensors). The developed foot operates similarly to a human foot, but it turns out to be a little unpractical and an expensive solution, since it requires a challenging process to be used with existing humanoids. On the other hand, Şafak et al. b2 use four force sensitive resistors Force Sensitive Resistors (FSRs) to build a foot contact sensor for a biped robot. Although, no trials have been performed, over numerical simulations the authors achieved a viable method to monitor postural stability of biped robots.

Systems that allow quantitative analysis of human gait are normally classified into: ground mounted force platforms and wearable instrumented systems (e.g. shoes and insoles). Force plates have the disadvantage of limiting the number of steps that can be done and are expensive, hence, the increasing advantage of using portable instrumented systems. These systems are usually designed with strain gauges, piezoelectrics, force sensitive resistors (FSR) or 3- to 6-axis transducers. Different wearable systems have been designed and developed over the recent years in order to measure the GRFs, to estimate the coordinates of the CoP and to monitor gait events (see for example b3; b4; b5; b6). However, most of the solutions are designed to be used directly with humans, and few allow the measurement of the complete GRFs.

Zhang et al. b6 features a fully-portable instrumented insoles capable of measuring spatio-temporal gait parameters and CoP trajectories during walking and running assignments, using a multi-cell piezoresistive sensor. Crea et al. b7 presents a pressure-sensitive foot insole for real-time monitoring of plantar pressure distribution during walking. The system includes 64 pressure-sensitive elements and a electronic board to acquire and transmit the data through bluetooth with a sampling frequency of 100 Hz. The proposed solution is developed to do a gait analysis based only on the vertical ground reaction forces (vGRFs) and the coordinates of the CoP. The estimation of the vGRFs showed a high qualitative correlation despite the significant difference in terms of actual values measurements against a force-platform. Lincoln et al. b8 designed a low-cost silicone insole capable of measuring the complete three dimensional reaction force vector. The developed sensor features a error within 10% and appears to have a temperature dependency. Many solutions within this theme are developed, but none seems suited to be adapted and used with different humanoids.

2.4 ITshoe Prototype Design

Fig. 2.1 schematically illustrates the design of the ITshoe. It has a total weight of 210 g (approximately 4% of the humanoid robot NAO weight) and dimensions measuring 125 mm × 65 mm × 18 mm to accommodate all the necessary hardware. The shoe is divided into two parts: OSh (Fig. 2.1a) and ISh (Fig. 2.1b). The OSh is the instrumented part of the shoe and the ISh is used to change the robot foot geometry allowing a tight fit with the robot’s foot, thus ensuring a correct and complete force transfer between the foot and the OSh.

The building material used in the manufacture of the ITshoe is acrylic since it presents low density (1.18 g/cm^{3}) together with an acceptable modulus of elasticity (3.2 GPa). In addition, the acrylic allows the user to verify whether the hardware is well positioned after a specific experience, thanks to its transparency, and it is also an excellent electrical insulator.

(a) OSh.  (b) ISh.

Figure 2.1: ITshoe model. The position of the force sensor that measures the tangential and the normal forces is represents by the numbers 1 and 2, respectively (left).

2.4.1 ITshoe structure

The ITshoe presented in Fig. 2.2 is divided into three units: sensing, acquisition and streaming. The sensing unit is composed of piezo-resistive A301 flexiforce sensors (Fig. 2.2 block C, number 6): 4 with a standard force of 4.44N for the measurement of tangential forces (positioned at 45^{∘}) and 4 with a standard force of 111N to register the normal forces. The sensors have a reported typical linearity of ±3%, a repeatability of ±2.5% and a hysteresis of ±4.5% of the full scale output b9. When compared to similar low price sensors (e.g. interlink FSR), they have a shorter response time, a better linearity, and offer the possibility of registering larger forces b10 . Over the sensitive area of these sensors is added a semi-sphere to ease the force transfer.

Figure 2.2: ITshoes schematic structure. The green block (A) is the acquisition unit, the red block (B) is the streaming unit, and the blue block (C) is the sensing unit. The main elements of these units are subtitled with numbers: 1–Battery; 2–Step-up voltage regulator; 3–Micro-controller; 4–Bi-directional level converter; 5–WiFi module; 6–Force sensor.

The acquisition unit (Fig. 2.2 block A) is responsible for the data acquisition. This unit deals with the electrical conditioning (Fig. 2.2, number 3 and 4) and the power supply (Fig. 2.2 number 1 and 2). The voltage dividers assembled in the PCB allow the connection of the sensors to the micro-controller. The ADC has a 10-bit resolution and 625 kHz frequency to convert the force signals into digital information. To synchronize and adjust the operating voltage level of this unit, a logic level bi-directional converter is used. The entire system is powered by a small 750 mAh Li-Po battery through a voltage regulator.

The streaming unit receives the data across a serial communication with the microcontroller and forwards it to the server, through the ESP8266 module (Fig. 2.2, number 5). The PCB added to the ITshoe incorporates a voltage regulator LM2937 to match the ESP8266 required power 3.3 V.

2.4.2 Sensor calibration

To ensure more accurate readings, a calibration procedure upon the sensors is required. The calibration procedure starts by conditioning the sensors and applying 110% (or slightly more) of the maximum test load onto the sensor for approximately 3 seconds. After repeating this conditioning process, different weights are applied on the sensor and, using a multimeter, the resistance is recorded. The plot in Fig. 2.3 shows the calibration line obtained for one of the 4.44 N standard force sensors, where it is visible that there is a linear evolution between the force applied to the sensor and its conductance.

04.3 8.6 12.9 17.2 21.5 0123·10^{−3} Force [N] Conductance [S] Sensor Calibration \begin{matrix}y(x) \\ 5 · 10^{−4} x − 4 · 10^{−5} \end{matrix}

Figure 2.3: Force sensor calibration line.

Besides calibration, the sensors were also tested for their dynamic response. This concern occurs because the reaction forces that arise when a biped robot walks are variable, thus emerging the necessity to validate the dynamic response of the sensors. Hence, an experiment was conducted to analyze the sensors’ dynamic response by using a mechanical testing machine Shimadzu MMT-101N. With this machine, a 4 N force was applied to the sensor with 3 different velocities (50, 25 and 12.5 N/s).

Fig. 2.4 shows the sensor responses for the applied forces. For the lower velocity, the sensor maximum force detected exceeds 3% the applied force, while for the higher velocity the sensor is not able to reach the maximum applied force (5.5% error). Indeed, there is a slight delay or saturation of the dynamic response of the sensors for some velocities, but those deviations are not expected to compromise the suitability of the sensors to monitor robot locomotion. Actually, and as might have been expected due to the piezoelectric nature of the sensors, their response improves with higher velocities, thus seeming quite adequate to catch the finer, or higher frequency, transitory force reactions.

00.1 0.2 0.3 0.4 0.5 0.6 012345v = 50 v = 25 v = 12.5 Time[s] Force[N] Dynamic response Sensor A201 MMT-101N

Figure 2.4: Flexiforce sensor dynamic response.

2.5 System Architecture

The proposed system architecture (Fig. 2.5) aims to create a data provider–data processor system in which the data provider is a measurer module responsible for measuring data regarding all components of GRFs for each robot foot separately. This module is also responsible for relaying this information to the data processor. The data processor is accountable for storing, processing and presenting such data.

Figure 2.5: System basic architecture.

The data provider is assembled by two major modules: the control module and the capture module. The control module is responsible for all the initial settings required, from the time the data provider runs until it connects to the data processor. Firstly, starts by setting the WiFi module using AT commands. These commands allow the WiFi module to establish a connection with the data processor and further define the protocol as UDP. The capture module implements the capture process, being responsible for measuring data from all the sensors synchronously. The module operates based on a timer, ensuring periodic data acquisition for each sensor. The diagram in Fig. 2.6 illustrates the acquisition process.

The data processor is also formed by two major modules: the storage and the analysis Figure 2.6: Acquisition process. F = Force applied to the sensor; S = Sensor; Rv = Resistance variation; M = capture Module; T = Timer.

modules. On the one hand, the storage module receives, stores, and displays the gathered data for further analysis. Essentially, in ROS environment, two functions (nodes) are used to receive and print the raw data. The data processor is configured to work as an access point (AP) creating a wireless local area network (WLAN). It waits for the ITshoe to be connected and starts receiving UDP messages. Well-formed messages are processed and made available for display, and also stored in a comma separated format file for further analysis by external applications. The second function subscribes the received message and generates a graphic environment (Fig. 2.7) that facilitates the understanding of the data evolution, as well as the CoP behaviour.

On the other hand, the analysis module implements a data analysis interface in order to convert the raw data into forces and to extract relevant information. The proposed system is designed to measure and transmit raw data at a frequency of 100 Hz.

Figure 2.7: Graphical representation of the force values (example using the LITshoe). 1-ITshoes; 2- Humanoid robot NAO.

2.6 Experiments and Results

This section describes the experiments and data acquisition results. The raw data upload rate is evaluated, and the GRFs are detected during two humanoid robot situations: static position and locomotion for two different grounds.

2.6.1 Data upload rate percentage loss

The ITshoes were turned on and data was collected for one hour; table 2.1 shows the data upload percentage loss for that time interval. By observing the results, it can be concluded that the developed ITshoes have a very good upload rate since the data loss is negligible. The loss of data can be justified with the instability of the WiFi module.

Table 2.1: Data upload rate percentage loss for both ITshoes.

Bytes sent Bytes received Data loss [%]
Left ITshoe  360000  359894  0.03
Right ITshoe  360000  359783  0.06

2.6.2 Detection of the vGRFs for a static robot position

Fig. 2.8 illustrates the position of the humanoid robot where the vGRFs (sum of the values measured by the four sensors responsible to measure this component of the GRFs) are gathered. As can be seen in Fig. 2.9, despite the visible differences between the total forces measured by the two ITshoes, the sum 53.19(32) N ends up to be close to the robot weight: 5.4 kg, approx. 52.96 N.

Figure 2.8: Robot in a static position.

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 20 30 40 50 55 Time[s] Force[N] vGRFs (static robot position)

LITshoe: F n_{t}
RITshoe: F n_{t}
SITshoe: F n_{t}

Figure 2.9: vGRFs for a robot static position. Left instrumented shoe (LITshoe) normal force (red), right instrumented shoe (RITshoe) normal force (green), and total (SITshoe) normal force (blue).

2.6.3 Detection of the total GRFs during robot locomotion on two different floors

For this trial, the data is collected with the humanoid robot NAO walking on two different floors: laboratory floor and carpet. The total GRFs are divided into vertical ground reaction forces vertical Ground Reaction Forces (vGRFs), and horizontal ground reaction forces horizontal Ground Reaction Forces (hGRFs). The measured vGRFs are illustrated in Fig. 2.10. In order to have a better perception of the measured values, only three random steps are shown.

It is noticeable that the step force patterns from grounds A and B are distinct. On the other hand, for each ITshoe (left and right) and type of ground (A and B), the step patterns are visually quite similar among them. These observations are very interesting since they open the way for automatic detection of grounds based solely on the reaction

5678910 010 20 30 40 50 60 B A Time[s] Force[N] vGRFs (locomotion) Left: F n_{t} ; Right: F n_{t}  0 10 20 30 40 50 60 Force[N]

Figure 2.10: vGRFs for a walking humanoid robot on two different grounds. Excerpt of 3 steps for each ITshoe. A–laboratory ground; B–carpet.

force patterns. If precise absolute values for each foot are of concern, it may be required to have an additional calibration process by means of a ground truth source (e.g., a forcesensing platform).

The hGRFs can be represented in the sagittal and in the transverse axis as depicted in Fig. 2.11. Since the horizontal sensors are strategically positioned at 45^{∘}, it is quite simple to write the equations that represent the forces in these axis.

Figure 2.11: hGRFs axis detail. S1_45 to S4_45 represent the four horizontal force sensors.

The total horizontal force in the sagittal and transverse plane can be calculated as follows:

F h_{st} = \frac{\sqrt{2}}{2} [(S1 + S2) − (S3 + S4)]  (2.1)

F h_{st} = \frac{\sqrt{2}}{2} [(S2 + S4) − (S1 + S3)]  (2.2)

where F h_{st} is the total horizontal force in the sagittal plane, F h_{tt} is the total horizontal force in the transverse plane, and S1 to S4 are the sensors used to measure the tangential forces.

Fig. 2.12 and Fig. 2.13 illustrate the hGRFs. The data corresponding to the moment when the foot is in the air (swing foot) was removed from the plots.

Relatively to the hGRFs in the sagittal plane, a similarity between the data from the LITshoe and the RITshoe is visible for both grounds. This pattern may be useful to recognise different robot walking situations, since the robot balance will tightly affect

5678910 −3 −2 −1 01234B A Time[s] Force[N] hGRFs sagittal plane (locomotion) Left: F h_{st} ; Right: F h_{st}

-3 -2 -1 0 1 2 3 4 Force[N]

Figure 2.12: hGRFs in the sagittal plane for a walking humanoid robot on two different grounds. Excerpt of 3 steps for each ITshoe. A–laboratory ground; B–carpet.

5678910 −8 −6 −4 −2 0246B A Time[s] Force[N] hGRFs transverse plane (locomotion) Left: F h_{tt} ; Right: F h_{tt}

-8 -6 -4 -2 0 2 4 6 Force[N]

Figure 2.13: hGRFs in the transverse plane for a walking humanoid robot on two different grounds. Excerpt of 3 steps for each ITshoe. A–laboratory ground; B–carpet.

these forces. The hGRFs in the transverse axis, between the LITshoe and the RITshoe, seems to be opposite but similar in absolute value. This behaviour appears to be correct since the positive direction in this axis for the RITshoe is the negative direction for the LITshoe as stated in Fig. 2.11.

All these indicators suggest that automatic detection of different floors, among other studies in gait analysis and expectedly also ground friction coefficients, can be done by observing the ground vertical and horizontal reaction forces using these instrumented shoes. Automatic separation of steps from full sequence of data is fairly easy since the alternation of the stance and swing foot is detectable by the activity of normal forces.

Anyway, with combined or separate steps, the next challenge is to process data and conclude about the floor the robot is walking on. Several tools appear, and some have actually been preliminary tried, suited for that task, such as temporal and spectral analysis, or even machine learning techniques.

2.7 Conclusions and Future Work

This paper presents a system that is able to measure the GRFs, in real-time, of a walking humanoid robot. The ITshoe is a cost-effective, self-contained and low-power battery-powered device that does not affect the balance of the robot. The ITshoe design simplifies its use with a variety of humanoid robots, since it only requires a change in its coupling geometry, that can be extracted from the robot foot in question. Nevertheless, more research, refinements and extensions are desired to improve its functionality and accuracy.

Using computational techniques models, either in spectral analysis or machine learning, in the near future, it is expected to be possible to distinguish different floors by analysing the patterns, obtained by the force sensors, for different grounds. Additionally, it is also expected that this data can be used for several other studies within this theme, to improve the performance and balance of biped humanoid robots.

Part II

LEARNING-BASED ANALYSIS

TO CLASSIFY DIFFERENT SLIPPERY FLOORS Learning-based Analysis of a New Wearable 3D force System Data to Classify the Underlying Surface of a Walking Robot

Published in

Almeida L, Santos V, Ferreira J

International Journal of Humanoid Robotics, Vol. 17, No. 03, 2050011 (2020) DOI: https://doi.org/10.1142/S0219843620500115

Chapter 3. Learning-based Analysis of a New Wearable 3D force

System Data to Classify the Underlying Surface of a Walking

Robot

3.1 Abstract

Biped humanoid robots that operate in real world environments need to be able to physically recognize different floors to best adapt their gait. In this work we describe the preparation of a dataset of contact forces obtained with eight force tactile sensors for determining the underlying surface of a walking robot. The data is acquired for four floors with different coefficient of friction, and different robot gaits and speeds. To classify the different floors, the data is used as input for two common computational intelligence techniques Computational Intelligence Techniques (CITs): Artificial neural network Artificial Neural Network (ANN) and extreme learning machine Extreme Learning Machine (ELM). After optimizing the parameters for both CITs, a good mapping between inputs and targets is achieved with classification accuracies of about 99%.

3.2 Introduction

Most walking robots, operating in real world scenarios, do not have a control system with knowledge about the characteristics of the underlying surface. This information is of critical importance for fine-tune the gait parameters to the floor requirements, so that the humanoid robot can reach a properly and safely locomotion, with a precise and robust control.

To this aim, the state-of-the-art literature describes mostly the use of 3D sensors, inertial measurements units, force/torque or tactile sensors to gather data for further classification of the floors using, most commonly, mathematical and learning-based algorithms. Although the majority of the work on floor classification is presented for wheeled robots, mainly for outdoor environments, either through data obtained with 3D sensors (e.g. winkens2017hyperspectral ; elkabbany2010terrain ; laible2013terrain ; woods2013terrain ; manduchi2005obstacle ; abbas2013improvements ), inertial measurement units (e.g. lomio2019surface ; oliveira2017speed ; brooks2005vibration ; weiss2006vibration ; giguere2009clustering ; giguere2009surface ; bai2019three ; tick2012indoor ), or multiple sensors fusion (e.g. ojeda2006terrain ; brooks2012self ; weiss2008combination ; wietrzykowski2018terrain ), our interest in the literature is focused on legged robots, more precisely humanoids.

In this scope, examples of vision-based approaches can be found in filitchkin2012feature ; zenker2013visual ; zhu2019surf . These works use a bag-of-words (BoW) model and a speeded up robust features (SURF) to extract and treat the images features for further classification using support vector machine Support Vector Machine (SVM). Even though using vision has the advantage of collecting information from the floors before the locomotion action, the results can be unreliable. For example, two surfaces may look the same, but yet have a different friction property, which leads to the need of two different humanoid gaits. We believe that sensing through physical

Chapter 3. Learning-based Analysis of a New Wearable 3D force System Data to Classify the Underlying Surface of a Walking Robot

interaction is a unique and necessary approach to improve humanoid robots mobility skills.

In matsumura2014kind the authors used body-mounted inertial sensors in a small humanoid robot to identify ten different kind of common floor surfaces on which the robot is standing in home environments. To build the dataset, four full body motions are executed 60 times for each type of floor, and using a decision tree classifier, they achieved a precision of 85.7%. The highest misclassification is found for relatively hard floors, because these floors are more similar to each other from a point of view of the properties that affect the body motions, unlike what happens with the soft ones. Other work have shown the possibility of using this type of sensors, using data related to three outdoor surfaces.bermudez2012performance They implemented a SVM algorithm and used 25% of the data to test it, achieving a 93.8% overall accuracy.

In walas2016terrain they propose a classification of 5 terrain types, based on the force torque sensor readings installed on the humanoid robot WALK-MAN ankle. A total of 1250 steps of the humanoid robot over the different floors are extracted and two methods were used for data reduction: Fast Fourier Transform (FFT) and Discrete Wavelet Transform (DWT). The learning procedure is performed using SVM over a feature vector of 120 elements or 270 depending on the approach used to reduce the steps data size, FFT or DWT, respectively. The results look more promising for the DWT approach with a precision of about 95% vs. 91%.

In kertesz2016rigidity six indoor surface types were distinguished by fusing data from multiple built-in sensors of a Sony ERS-7, such as, force sensors, to collect the ground contact forces, accelerometer, infrared range and motor force sensors. The developed random forest model allowed the classification of the different floors with a cross-validation and accuracy of 96.2% and 94%, respectively, for one gait at a fixed speed. Similarly, by fusing different sensors, a six-legged robot is able to recognize 12 surfaces with a 95% precision.walas2015terrain We propose a novel method to classify four floors with different friction coefficient based only on the data obtained with eight force tactile sensors that are installed in an instrumented shoe that we developed on previous work, to be used with the humanoid robot NAO.almeida2018novel To interpret and recognize the data, two popular computational intelligent techniques (CITs) are explored: artificial neural network (ANN) and extreme learning machine (ELM). In order to obtain a more robust learning classifier the data is also collected for several gaits and speeds.

Past works, show the successful application of ANNs bai2019three ; ojeda2006terrain , and, to our knowledge, ELM is not yet explored for the humanoid floor classification problem.

The main contribution of this work is the validation that inexpensive tactile sensors together with common, widely used learning-based algorithms can be used to obtain reliable information which allows the correct classification of different floors, thus improving humanoid robots mobility skills.

Chapter 3. Learning-based Analysis of a New Wearable 3D force

System Data to Classify the Underlying Surface of a Walking

Robot

The reminder of the paper is divided as follows. Section II presents the materials and methods for the data collection and manipulation. The CITs implementation and experimental results are presented in Section III. Lastly, section IV presents the conclusions and future challenges.

3.3 Materials and methods

On previous research we developed an ITshoe (Fig. 3.1) to be seamlessly installed on a walking humanoid robot to measure real-time vertical and horizontal GRFs.almeida2018novel The ITshoe has two main parts, outer shoe and inner shoe, and it is designed to be used with the humanoid robot NAO. The inner shoe is used to link the robot’s foot and the outer shoe, whereas the outer shoe is the instrumented part of the shoe. The ITshoe is designed to measure and transmit raw data at a frequency of 100 Hz and it is composed by a sensing unit (eight A301 flexiforce sensors), acquisition unit (electrical conditioning and power supply), and a streaming unit (WiFi module). Fig. 3.2 illustrates an example of the measured GRFs (normalized) for two steps of a walking humanoid robot. The GRFs are divided into total normal force (vGRFs) and total horizontal force (hGRFs). The hGRFs can be represented in the sagittal and transverse plane as depicted in Fig. 3.1, and calculated as follows:

F h_{st} = \frac{\sqrt{2}}{2} · [(S1_{45} + S2_{45}) − (S3_{45} + S4_{45})],  (3.1)
F h_{tt} = \frac{\sqrt{2}}{2} · [(S2_{45} + S4_{45}) − (S1_{45} + S3_{45})],  (3.2)

where F h_{st} is the total horizontal force in the sagittal plane, F h_{tt} is the total horizontal force in the transverse plane and S1_{45} to S4_{45} are the sensors used to measure these tangential forces.

All datasets presented in this work are obtained with the ITshoe.

3.3.1 Floor classification

In this study, we attempt to identify which one of four different candidate floors the humanoid robot walked on, having as hypothesis that each ground type has a particular signature when identified by the force sensors that measure the contact forces between the biped feet and the ground. To further facilitate the data collection in the laboratory, we built three shoe soles (using the materials: teflon, aluminium and carpet) to be added to the ITshoe and used the ITshoe acrylic as the fourth material. Fig. 3.3 shows the shoe soles used in this work.

Their coefficient of friction Coefficient of Friction (CoF) is measured against a melamine sheet (base material) on which the robot walks with the distinct shoe soles. The friction coefficients of the testing materials are measured as follows: each shoe sole is positioned on a ramp of the melamine material; the ramp is tilted until the shoe sole starts to slide

Chapter 3. Learning-based Analysis of a New Wearable 3D force System Data to Classify the Underlying Surface of a Walking Robot

at a constant velocity; the tilting angle is measured; lastly, the CoF is calculated (equal to the tangent of the measured angle). Fig 3.4 illustrates the layout for the data acquisition and CoF measurement.

The chosen floors (shoe soles) have different friction coefficients, ranging from approximately 0.1 up to 0.5. We decided that the lower friction would be around 0.1 since the biped robot can not walk on more slippery floors, and above 0.5 we considered that the robot can perform well balanced locomotion actions. Regardless the friction range in study, in future work, we intend to extrapolate the results to a wider range. Table 3.1 presents the measured CoF for the different materials.

Table 3.1: Shoe soles coefficient of friction.

Material  Coefficient of Friction
Teflon-Melamine  0.11
Acrylic-Melamine  0.23
Aluminium-Melamine  0.35
Carpet-Melamine  0.52

3.3.2 Data manipulation

The output signals from each one of the available sensors are sampled and stored during real-time experiences for subsequent offline analysis. To get more representative and robust data we also decided to gather the data not only for the different floors, but also for different robot gait parameters, such as different step length (0.02, 0.04, 0.06 [m]) and step frequency (0.2, 0.4, 0.6). For these parameters we decided to use equally distanced values, being that for the higher limit of step length we chose the maximum step length (0.06 m) of this robot, and the maximum frequency (0.6) was chosen because larger values did not produce good enough locomotion actions, therefore making it impossible to collect data. To summarize, the robot steps force data is collected for all the 36 resulting combinations between the different floors and the variable gait parameters (4x3x3). Roughly 100 steps are registered for each combination.

Before classifying the floors with CITs, it is necessary to pre-process the raw data. The following procedure describes the developed algorithm methodology to extract the humanoid robot steps that are used as input for the different machine learning approaches. The algorithm takes as input a matrix filled with raw data of 8 flexiforce sensors, and outputs the robot steps separately and normalized in the range [-1,1], as follows:

(i) Use the calibration curves to convert each sensor (S(i)) raw data into forces, as described by the following equation:

F (i) = \frac{b(i)}{R(i) ( \frac{1024}{S(i)} ) − 1} × \frac{1}{m(i)} ,  (3.3)

Chapter 3. Learning-based Analysis of a New Wearable 3D force

System Data to Classify the Underlying Surface of a Walking

Robot

where R is the voltage divider resistor, m and b are the calibration curve slope and y-intercept respectively, and 1024 (2^{10}) refers to a 10-bit analog to digital converter (ADC).

(ii) Obtain the start and end of each step: F n(i) ≈ 0 (normal force = 0, robot foot is in the air).

(iii) Normalize each force F(i) to be inside the range [-1,1].

(iv) Interpolate each step using a linear interpolation to guarantee that all steps have the same size (this is a necessary stage since different gait parameters will generate steps with variable sizes).

(v) Define Input variables for the CITs:

(a) F n (Normal Force).

(b) F h_{st} (Tangential force in the sagittal plane).

(c) F h_{tt} (Tangential force in the transverse plane).

(d) F u_{st} (Equals to F h_{st}/F n).

(e) F u_{tt} (Equals to F h_{tt}/F n).

3.3.3 Data for CIT training, validation and testing

The input matrices have 100 lines, one for each point that represents a single step on a specific floor with certain gait parameters, and N columns, where N represents the number of steps collected for this study. We processed 80 steps for each combination (gait parameters+floor), thus obtaining an input matrix of 100x2880. Fig. 3.5 shows an example of the input data presented in the form of F h_{st} normalized between -1 and 1. The visible variation between steps of the same class is due to the diverse gait parameters used in the data collection, as expected, a different gait configuration will produce different contact forces.

Since we want to classify four different floors (classes), our target matrix has 4 lines by N number of steps, being that only one of the lines is filled with the value 1 and the others with the value 0 (e.g. [0,0,1,0]’, means that the target is the 3rd floor). The data is prepared and randomly divided into three subsets: the training set (50%), which is used for computing the gradient and updating the network weights and biases; the validation set (25%) to measure network generalization and to halt training when generalization stops improving; and the test set (25%) that is used to compare different ANNs as well as evaluate the ability of the network to correctly classify the floors. To make the classification process more accurate and less biased, the 36 data sets are randomly splitted into the three subsets, to ensure that each has 25% of the data represented by each of the classes. Table

Chapter 3. Learning-based Analysis of a New Wearable 3D force System Data to Classify the Underlying Surface of a Walking Robot

3.2 shows the final dimensions of the input and target matrices.

Table 3.2: Dimensions of the input and target matrices

Input  Target
Training  100x1440 4x1440
Validation 100x720  4x720
Testing  100x720  4x720

3.4 CIT experimental results

As described in chapter I two CITs (ANN and ELM), implemented in MATLABmatlab2018 are explored to classify the different floors.

3.4.1 ANN

The ANN studied consists of a generic feed-forward back-propagation neural network with three layers: input, sigmoid hidden and softmax output layer. An example of the studied ANN architecture can be seen in Fig. 3.6.

To tune and obtain the best ANN we developed many experiments, where we varied (1) the number of hidden neurons, (2) the back-propagation mathematical algorithms and (3) the way the inputs are presented to the network (as described in chapter III). Each defined input matrix is trained using 5 different backpropagation algorithms: Scaled Conjugate Gradient (scg), Conjugate Gradient with Powell (cgb), Resilient Back-propagation (rp), Variable Learning Rate Back-propagation (gdx) and Polak-Ribiére Conjugate Gradient (cgp). Additionally, each algorithm is trained 10 times for each number of hidden neurons ranging from 2 to 52. A total of 500 ANN are trained and tested for each training algorithm. The output of the ANN is an array with 4 numbers between zero and one, being that each number indicates the likelihood of the present floor being one of the four previously labelled and trained, where a value of 0 means not likely and 1 means likely.

After evaluating the performance results, with cross-entropy, for the various tested algorithms, the following Fig. 3.7 shows the overall performance of the ANNs for the diverse input variables, and only for the algorithm that best performed: gdx. It is visible in the chart, that the performance increases significantly as the number of hidden neurons increases up to approximately 20. From here, the increasing number of neurons does not affect the performance in more than 1%. In attempt to avoid overfitting and bearing in mind that higher number of neurons almost does not improves the performance, we decided that the optimal number of hidden neurons for this network is 20. Observing the chart, can also be seen that the defined inputs that outputs the best overall performance are the tangential forces in both sagittal and transverse planes (F h_{st} and F h_{tt}). On the other hand, although the normal forces are important to separate the robot steps, these

Chapter 3. Learning-based Analysis of a New Wearable 3D force

System Data to Classify the Underlying Surface of a Walking

Robot

are the ones that present the poorer results. Additionally, the inputs defined as F u_{st} and F u_{tt} present a lower performance when compared with the F h_{st} and F h_{tt}, possibly due to the negative effect of the normal forces.

The confusion matrix presented in Fig. 3.8 shows the testing accuracy for the best obtained ANN using as input the tangential forces in the transverse plane. The test confusion matrix show us that the network failed to correctly classify 24 out of 720 (3.3%) steps.

With knowledge of the best algorithm (gdx) and the best number of hidden neurons (20) we decided to train a new network using both tangential forces in order to improve the ANN accuracy. The new input matrix has 200 rows (100 for the F h_{st} and 100 for the F h_{tt}) by 2880 columns. The following confusion matrix presented in Fig. 3.9, shows that the testing accuracy increased 2.6% from 96.7% to 99.3%. It seems that the information from both tangential forces are relevant and necessary to best classify the different floors.

3.4.2 ELM

The ELM consists in a single hidden layer feed-forward network with a more efficient learning algorithm, which randomly chooses hidden nodes and analytically determines the output weights. One of the advantages of using ELM is its extremely fast learning speed, that most real-time applications require.

For the ELM analysis we decided to only use the tangential forces as input (200x2880) since it has been proven previously that these forces are the ones presenting the best results. The ELM was trained with a number of hidden neurons ranging from 100 to 600. In order to change the random variables value we decided to run 10 ELM for each number of neurons. Similarly to the previous approach the activation function used was the sigmoid function. Fig. 3.10 shows the training and testing accuracy for the different number of neurons.

Although both training and testing accuracy increases with the increasing number of hidden neurons, for a number of neurons bigger than 400, it can be seen that the testing accuracy stagnates around 97%. As result, we propose the optimized ELM with 400 hidden neurons. With this ELM architecture we obtained a testing accuracy of 98.5%. Fig. 3.11 shows the confusion matrix for the chosen ELM network. The test confusion matrix show us that the network failed to correctly classify 11 out of 720 (1.5%) steps.

3.4.3 ANN vs ELM: Classification time

For further application of the CITs in real-time it is important to compare both CITs not only by their final accuracy but also by their real required time for classification. As previously described the data is recorded at a rate of 100hz which means that if we need to take an action (e.g. adapt the humanoid robot gait based on the detected floor information so that the next robot step is optimized for the current floor), we have to do

Chapter 3. Learning-based Analysis of a New Wearable 3D force System Data to Classify the Underlying Surface of a Walking Robot

it in less than 10ms.

The previous results show that the ANN presents better classification accuracy when compared with the ELM, however the ANN takes about 145 times longer than the ELM to classify a floor. Using the software MATLABmatlab2018 , the ANN requires about 15ms to classify a single humanoid robot step. We expect that the application of this ANN using a computer programming language (e.g. C) will lead to a considerable reduction of the required time. In case of the necessary time still exceeds 10ms, the ELM appears as the best suitable solution for this problem.

3.4.4 Step percentage reduction

The previous approach allows only the correction of the robot gait, for the requirements of the different floors, after it performs a complete step. To ensure that the humanoid robot does not lose its stability before we adapt its gait, we also developed a study where we attempted to reduce the required time for the floor classification problem. For this purpose we evaluated the ability of the network to classify the different floors using only a percentage of the robot step data points. Multiple networks were fed with a decreasing percentage of data points that represent the robot step, and simulated using the optimal parameters described previously. The results are presented on Fig. 3.12.

Looking at the results, it can be seen that the network has a good accuracy with a loss of less than 2% even for steps represented only by 40% of the data points. For steps represented by 60% of the data points the network performance decreases by approximately 0.5%. It is seen that, without compromising much the performance we can use a percentage of the robot step to classify the floor and thus control the robot stability much earlier than what was allowed by the previous approach. Additionally, if the decrease in the network accuracy affects the results in real time, a new network can use the chosen percentage of data points to predict the next ones so that the accuracy increases again.

3.5 Conclusions

In this paper, we have addressed the problem of floor classification with a biped robot based only on the ITshoe force data and a learning-based analysis.

The 2880 labelled steps of a walking biped robot over four different floors are used to train and test two CITs. The comparison between the ANN and ELM performed in this study, suggests that the ANN is the best CIT to classify the different floors considering that the time required for classification is lower than 10ms. The ANN outperform the ELM with a final testing accuracy of 99.3% vs. 98.5%.

In the near future we expect that the application of this CITs in real-time, will not only help to classify the present floor but also, lead to a more efficient gait, since a controller can be developed to optimize the robot gait based on the classified floor.

Chapter 3. Learning-based Analysis of a New Wearable 3D force System Data to Classify the Underlying Surface of a Walking Robot

Figure 3.1: ITshoe schematic structure. The block (A) is the acquisition unit, the block (B) is the streaming unit, and the block (C) is the sensing unit. The main elements of these units are subtitled with numbers: 1- Battery; 2- Step-up voltage regulator; 3-Micro-controller; 4- Bi-directional level converter; 5- WiFi module; 6- Force sensor. On the right side it is visible the position of the eight force sensors and the reference axis used to decompose the tangential forces.almeida2018novel Figure 3.2: GRFs for a walking humanoid robot. Example of two measured steps represented as normal and tangential forces.

Figure 3.3: Shoe soles materials. A- teflon; B- acrylic; C- aluminium; D- carpet; Emelamine sheet.

Figure 3.4: Layout for the data acquisition and CoF measurement. On the left side is the melamine sheet on which "n" robot steps are measured and the humanoid robot with the ITshoes, and on the right side is presented the melamine ramp used to measure the CoF of the materials above identified.

Figure 3.5: Example of the input dataset F h_{st} normalized. The chart shows 720 processed steps for each floor.

Figure 3.6: Neural Network architecture.

Figure 3.7: ANN overall performance for the diverse input variables.

Figure 3.8: Best ANN testing confusion matrix using the tangential forces in the transverse plane as input for the network.

Figure 3.9: Best ANN testing confusion matrix using both tangential forces as input for the network.

Figure 3.10: ELM training and testing classification performance.

Figure 3.11: ELM testing confusion matrix using both tangential forces as input for the network.

Figure 3.12: ANN testing performance for different percentage of step data points.

Real-time LSTM-RNN Classification of Floors with Different Friction Coefficients for a Walking Humanoid Robot Wearing a 3D Force System

Published in

Almeida L, Santos V, Ferreira J

IEEE Sensors Journal( Volume: 21, Issue: 24, 15 December 2021) DOI: 10.1109/JSEN.2021.3124854

Chapter 4. Real-time LSTM-RNN Classification of Floors with Different Friction Coefficients for a Walking Humanoid Robot

Wearing a 3D Force System

4.1 Abstract

In the study of biped humanoid robots it is crucial to achieve high precision and robustness in locomotion. Humanoid robots that operate in real world environments need to be able to physically recognize different grounds to best adapt their gait without losing their dynamic stability. This work proposes a technique to classify in real time the type of floor from a set of possibilities learnt off-line. Hence, the paper describes the collection and preparation of a dataset of contact forces, obtained with a wearable instrumented system, mixed with the information of the robot internal inertial sensor to classify the type of underlying surface of a walking humanoid robot. For this classification, the data are acquired for four different slippery floors at a rate of 100 Hz and it is used as input for a long short-term memory Long Short-Term Memory (LSTM) recurrent neural network (RNN). After testing different learning models architectures and tuning the models parameters, a good mapping between inputs and targets is achieved with a test classification accuracy greater than 92 %. A real time experiment is presented to demonstrate the suitability of the proposed approach for the multi-classification problem addressed.

4.2 Introduction

The study of humanoids increasingly contributes to several scientific areas of which it is possible to extrapolate improvements to our lives, such as in walking rehabilitation, dangerous works, or elderly assistance. To achieve a robot capable of servicing and assisting people, first it has to be able to perform fundamental locomotion tasks such as balancing and walking pratt2010toward . Despite the progress and efforts made in the past years, humanoid locomotion is still a challenging problem without definitive solution. Developing a good system that allows a biped robot to walk on unknown and diversified floors, e.g. slippery floors, requires the system to be intelligent and autonomous to adapt in real time so that the robot can successfully overcome the barriers found during locomotion tasks siciliano2016springer . In most of the bipedal locomotion approaches, hard contacts with the ground are assumed, although, in real life scenarios this is normally not accurate. Despite the several developments over the years, there are no explicit implementations which deal with the changing floor properties hopkins2015design . This oversight may lead to disastrous consequences, e.g. the biped robot falls while walking on a non-modeled ground, most of the times preventing the robot to continue its locomotion tasks.

Having an intelligent algorithm that allows the robot to identify the terrain with good accuracy, using force sensors installed on its feet or assembled under it, will give the possibility to eliminate most of the falls while walking on different surfaces walas2016terrain ; multi_axis_review . The increasing progresses made in the areas of artificial intelligence and machine learning, lead to a significant impact in this field. With the rise of techniques such as neural networks, recurrent neural networks Recurrent

Chapter 4. Real-time LSTM-RNN Classification of Floors with Different Friction Coefficients for a Walking Humanoid Robot Wearing a 3D Force System

Neural Networks (RNNs), deep learning and reinforcement learning, humanoids can now perform tasks that previously seemed far-fetched.

In this work, the capabilities of a Long Short-term Memory Recurrent Neural Network (LSTM-RNN) (first presented in schmidhuber1997long ) were analysed to classify the underlying surface of a walking robot. To feed the network, a wearable instrumented system assembled to a robot foot was used to measure the ground reaction forces (GRFs), and the internal robot inertial sensors were used to measure the body accelerations and inclinations on different slippery floors. The choice of this type of network was mainly due to its feedback loop which serves as a kind of memory. This means that the past inputs leave a footprint on the model that is expected to be an asset, e.g., when the humanoid robot moves from one floor to another. These networks are capable of recognizing temporal encrypted patterns from dynamic data, which is what happens between the interaction of the humanoid’s foot and the different floors; indeed, humanoid walking can be considered a time-dependent task. Additionally, our approach does not require a time window for offline processing, hence, the classification can be done online at every new robot step. The main contribution of our work is the classification of slippery floors using a novel instrumented system that can be adapted to different humanoid robots. This classification will provide knowledge of the characteristics of different floors on which the robot walks to the humanoid controller, thus allowing it to adapt the robot gait according to the different slippery floors requirements.

Several published works in the field of humanoid robots show the applicability and capabilities of the LSTM-RNN Yu20201335 ; Ko20201200 ; Lobos-Tsunekawa20183247 ; Kerzel2019 ; Bhattacharjee2018 ; chalvatzaki2019lstm ; bi_lstm . For example, in how2014multiple a LSTM-RNN model was used to classify six different robot behaviours based on ten robot joint time sequences. In zhao2018collaborative it was used to generate a robotic motion from the observations of the human movements to achieve fast and responsive human robot collaborative tasks, avoiding the trouble of solving an inverse kinematics or motion planning problem. In yu2020walking kinect sensors were used to obtain walking information of the human body under different slopes, and the data collected are used to feed a LSTM neural network to learn the degrees of freedom of multiple lower limb joints, classify and recognize the different slopes and to compensate the robot’s ankle joints based on the slope inclination. The results show a robot NAO able to walk on different slopes. In another example li2018mobile , the authors used a LSTM network to classify motor fault in mobile robots achieving an accuracy of 87 %. In li_sequential_2021 the authors used the robot’s foot soles pressure sensors and inertial measurement unit (IMU) to feed a LSTM network. They used the sensors to calculate two-directions center of pressure (CoP) and the IMU to obtain two-direction acceleration. The authors report that the robot walking process based on the LSTM output is better than when using a fixed gait. Another example using pressure sensors data and a LSTM network is addressed in zhang_fingervision_2018 . The authors used these sensors to collect

Chapter 4. Real-time LSTM-RNN Classification of Floors with Different Friction Coefficients for a Walking Humanoid Robot

Wearing a 3D Force System

data while interacting with different daily objects. The developed LSTM presented a 97.62% accuracy on the test dataset, being that the authors expect that this good slippery classification can improve the robot grasping skills, leading to a better contact and smoother interaction/manipulation.

Different LSTM architectures and their combination with other networks have also been increasingly studied and applied in the humanoid field. In 9292620 the authors used a bidirectional LSTM-based network which makes use of historical measurements of system states to predict humanoid fall probability in real-time. In farazi2017online a deep LSTM network was trained with simulated data and fine-tuned on a set of real data to track and identify Robots with identical appearance. Since the analysed data are transmitted via Wi-Fi, some delay or even data loss are expected, hence the importance of using a LSTM. Similarly, in zhen_walking_2019 a deep network LSTM was also used, but here the authors fed the network with acceleration signals and used it to detect gait-phases of a walking human, presenting a F-score higher than 92%. In li_cnn_2019 a LSTM network was combined with a convolutional neural network (CNN) to improve the human-robot interaction. The CNN was used to extract visual features and the LSTM network was used to find the relationship between these features and six basic emotions. The humanoid is able to adapt its response based on the human emotion classified with the LSTM model.

Our approach is comparable to the floor classification problem addressed by wang2020situ since a LSTM network is explored to solve the problem, but the approaches differ because wang2020situ uses the humanoid foot sinking state and the force reads from a load cell embedded in the robot’s ankle to classify deformable terrains, whereas we combine data from the robot inertial sensor and eight force sensors to classify different slippery floors. They achieved 95 % accuracy on average during experiments.

The remainder of this work is divided as follows. Section II presents the materials and methods for the data collection and manipulation. The LSTM network implementation, tuning and online experimental results are presented in Section III. Lastly, Section IV presents the conclusions and future challenges.

4.3 Materials and methods

In previous research activities almeida2018novel , an instrumented system was developed to be seamlessly assembled on the walking humanoid robot NAO to measure real-time vertical and horizontal ground reaction forces (GRFs). The GRFs are divided into total normal force (vGRFs) and total horizontal force (hGRFs). The developed system is a cost-effective, lightweight and wirelessly instrumented shoe (ITshoe). The ITshoe used on this work is presented in Fig. 4.1.

The hGRFs can be represented in the sagittal and transverse plane, as depicted in Fig.

Chapter 4. Real-time LSTM-RNN Classification of Floors with Different Friction Coefficients for a Walking Humanoid Robot Wearing a 3D Force System

Figure 4.1: ITshoe schematic structure. The red block (A) is the acquisition unit, the green block (B) is the streaming unit, and the blue block (C) is the sensing unit. At the top, the image shows the position of the eight force sensors and the reference axis used to decompose the tangential forces.

4.1, and calculated as follows:

F h_{st} = \frac{\sqrt{2}}{2} · [(S1_{45} + S2_{45}) − (S3_{45} + S4_{45})]  (4.1)
F h_{tt} = \frac{\sqrt{2}}{2} · [(S2_{45} + S4_{45}) − (S1_{45} + S3_{45})],  (4.2)

where F h_{st} is the total horizontal force in the sagittal plane, F h_{tt} is the total horizontal force in the transverse plane and S1_{45} to S4_{45} are the four Flexiforce sensors, positioned at 45^{∘}, used to measure the tangential forces.

Chapter 4. Real-time LSTM-RNN Classification of Floors with Different Friction Coefficients for a Walking Humanoid Robot

Wearing a 3D Force System

The ITshoes are divided into three units (see Fig. 4.1):

Sensing unit - Composed of eight piezo-resistive A301 flexiforce sensors;

Acquisition unit - Deals with the electrical conditioning and power supply;

Streaming unit - Receives the data across a serial communication with the microcontroller and forwards it to the server, through the ESP8266 Wi-Fi module.

The robot NAO also communicates with the server through Wi-Fi and its main software is used to make it walk. The high-level functions allow to define the walking distance and gait configuration parameters, such as, step length, height, frequency and torso translation along the X and Y axis. The server runs the open-source system ROS that deals with all the data flow and communications with the ITshoes and the Robot (for further details, see almeida2018novel ). The dataset used for this work were recorded using the ITshoes sensors and the humanoid robot NAO internal sensors at a frequency of 100 Hz.

4.3.1 Floor multi-classification problem

This work extends the study presented in almeida2020learning to classify different slippery floors on which the humanoid robot NAO has to walk while wearing the ITshoes. Our hypothesis is that each floor has a particular characteristic when identified by the ITshoes force sensors together with the robot NAO inertial unit, which is positioned in the center of the body of the humanoid NAO, namely data from its 3-axis gyroscope, acceleration and body inclination. Fig. 4.2 shows the layout used to collect data for the addressed multi-classification problem.

Figure 4.2: Layout for data collection with four different floors.

The system NAO+ITshoes is used to collect data while the humanoid NAO walks on four different slippery floors: Polytetrafluoroethylene (PTFE); Aluminium; Polyethylene high-density (PE-HD); and Melamine floor. We only focused on the slippery characteristic of these floors since the forces that the humanoid NAO exert on these rigid and flat floors while walking are not significant and do not produce meaningful elastic or permanent deformation. However, our classification can be associated with different material properties, and can also be used to classify different floors such as deformable floors. Table 4.1 presents the coefficient of friction (CoF) for the different materials used as the walking floors. Acrylic is the base material of the ITshoes.

4.3.2 Data manipulation

Before classifying the floors using the LSTM-RNN, it is necessary to pre-process the raw data. The following procedure describes the developed algorithmic methodology to extract the data that corresponds to the humanoid robot steps, and to format its representation

Chapter 4. Real-time LSTM-RNN Classification of Floors with Different Friction Coefficients for a Walking Humanoid Robot Wearing a 3D Force System

Table 4.1: Floor-Shoe coefficient of friction

Material  Coefficient of Friction
Acrylic-Polytetrafluoroethylene (PTFE)  0.11
Acrylic-Aluminium  0.20
Acrylic-Polyethylene high-density (PE-HD)  0.26
Acrylic-Melamine  0.33

to be used as inputs for the learning approach. The algorithm reads the recorded raw data from the database and outputs only the data that corresponds to moments when the robot’s foot is in contact with the floor. The main steps of the algorithm are as follows:

(a) Use the calibration curves to convert each sensor (S(i)) raw data into forces, as given

by:

F (i) = −\frac{b(i)}{R(i) ( \frac{1023}{S(i)} − 1)} \frac{1}{m(i)} , i ∈ \{1, ..., 8\},  (4.3)

where R(i) is the voltage divider resistor, m(i) and b(i) are the calibration curve slope and y-interception respectively, and 1024 (2^{10}) refers to a 10-bit analog to digital converter (ADC);

(b) Obtain the start and end of each step: the points where F n(i) ≈ 0 (normal force ≈ 0, robot foot is in the air);

(c) Use the indices i for which F n(i) ≠ 0 to filter the data points for all the recorded variables;

(d) Normalize each data point to be in the range [-1,1] ;

(e) Reshape the data according to the LSTM model needs.

4.3.3 Data for LSTM training, validation and testing

The resulting dataset for this work includes 27800 labeled sequences. Each input sequence consists of 11 features, each composed of 50 samples. Despite the fact that the measured vertical forces are crucial for the data collection and manipulation, these were not used as features to train the network due to the dynamics involved in the robot step being so small that the variability of the normal force sensors are of limited use almeida2020learning . The 11 selected features to represent the robot’s behaviour on the different studied floors are as follows:

F h_{st} Horizontal force in the sagittal plane (N);

F h_{tt} Horizontal force in the transverse plane (N);

Chapter 4. Real-time LSTM-RNN Classification of Floors with Different Friction Coefficients for a Walking Humanoid Robot

Wearing a 3D Force System

G_{x} Gyroscope X axis (rad/s);

G_{y} Gyroscope Y axis (rad/s);

G_{z} Gyroscope Z axis (rad/s);

Acc_{x} Accelerometer X axis (m/s^{2});

Acc_{y} Accelerometer Y axis (m/s^{2});

Acc_{z} Accelerometer Z axis (m/s^{2});

BI_{x} Body inclination X axis (rad);

BI_{y} Body inclination Y axis (rad);

BI_{z} Body inclination Z axis (rad).

The data are prepared and randomly divided into three subsets: the training set (60 %), which is used for computing the gradient and updating the network weights and biases; the validation set (20 %) to measure network generalization and to halt training when generalization stops improving, and the test set (20 %) that is used to compare the different LSTM networks, as well as evaluate the ability of the network to correctly classify the floors. To make the classification process more accurate and less biased, a 10-fold cross validation is applied to the data so that the LSTM model is trained 10 times, each time with a different train/test split. Table 4.2 shows the final dimensions of the input and target matrices.

Table 4.2: Dimensions of the input and target matrices

Input  Target
Training  (16680, 50, 11) (16680, 1, 4)
Validation (5560, 50, 11)  (5560, 1, 4)
Testing  (5560, 50, 11)  (5560, 1, 4)

Fig. 4.3 illustrates the 50 chosen samples of the normal force component from a full humanoid robot step. All the selected features for this classification problem use also 50 samples corresponding to the same time steps.

4.4 LSTM-RNN experimental results

Recurrent Neural networks (RNNs) are the feed-backward version of the conventional feed-forward neural networks. They have a cyclic connection architecture that allows them to update their current state based on past states and current input data. The standard RNN topology suffers from the vanishing gradient graves_supervised_2012 . To over-

0 20 40

50 62 50 62 50 62

Normal Force (N)

Figure 4.3: Illustration of the 50 chosen samples of the normal force of a robot step to be used on the classification of the robot walking floor.

come this problem, Schmidhuber and Hochreiter schmidhuber1997long developed the Long Short Term Memory (LSTM) unit presented in Fig. 4.4.

\begin{aligned}× & & + \\ & & × & & × \\ σ & σ & tanh & σ \end{aligned} tanh C_{t−1} h_{t−1} C_{t} h_{t} x_{t} h_{t} f_{t} i_{t} C̃_{t} O_{t}

Figure 4.4: LSTM cell structurevarsamopoulos2018designing .

The LSTM unit, also known as "memory block" enables the network with the capacity to store and access information over long periods of time. LSTMs achieve this through their 3-gate architecture, which consists of an input, forget and output gates. The input gate decides which input information will be used to update the memory state. The forget gate decides which information to keep or erase from the memory block. Finally, the output gate decides based on the current input and the stored memory which information to output. The LSTM models developed in this work were built using the open source neural network library Keras, running on top of the machine learning platform TensorFlow. The API Keras is written in Python. Several LSTM networks configurations were implemented to classify the different floors from the input layer of 11 features and 50 time steps. The base model used to start training was a LSTM layer with a sigmoid activation function and 50 hidden neurons, a fully-connected output layer with activation function "softmax", and 4 hidden neurons corresponding to the four floors to be classified. The optimizer used in all the training trials was the Adam optimizer kingma2014adam because of its ability to converge quickly while traditionally performing better than most other optimizers chang2018effective ; wang2020power . Additionally, an early stop strategy is applied

Chapter 4. Real-time LSTM-RNN Classification of Floors with Different Friction Coefficients for a Walking Humanoid Robot

Wearing a 3D Force System

to the training process to halt it when the loss of the model stops improving. The loss of the model was evaluated using a categorical cross-entropy function, calculated according to (4.4)

L = − ∑_{c=1}^{M} y_{o,c} log(p_{o,c})  (4.4)

where M is the number of classes (different floors), y_{o,c} indicates if the class label c is the correct classification for the observation o, and p_{o,c} is the predicted probability of the observation o is of class c. Before tuning the LSTM model hyper parameters, a dropout layer was added to the model since it is commonly used to fight over-fitting and to improve the model performance. This layer is used as a regularization method were input and recurrent connections to LSTM units are probabilistically excluded from activation and weights updates while training a network. It randomly sets input units to 0 with a chosen frequency f_{d} (fraction of the input units to drop) at each step during training time. Inputs not set to zero are scaled up by 1∇ ·(1 − f_{d}) such that the sum over all inputs is unchanged. Fig. 4.5 exemplifies the dropout behaviour for a standard neural network with two hidden layers.

Figure 4.5: Dropout behaviour. On the left it is visible a standard neural network and on the right an example of the same network after applying dropout. The units presented with a cross have been dropped srivastava2014dropout .

Multiple LSTM models architectures were implemented and trained with a varying number of hidden neurons, different activation functions, batch sizes, learning rates and dropout probabilities. Table 4.3 shows the evaluated range for some of these parameters as well as the chosen value that produced the overall best results.

Table 4.3: Range of network parameters evaluated.

Range  Best
Hidden neurons  50 - 500  300
Learning rate  0.00001 - 0.1 0.0001
Batch size  10 - 500  256
Dropout probability  0.1 - 0.5  0.2

Fig. 4.6 shows the optimized model obtained for this multi-classification problem using the chosen best parameters. This model has three layers: a LSTM layer with 300 hidden neurons and "tanh" activation functions, a dropout 0.2 layer and a fully-connected layer with four hidden neurons and "softmax"activation functions.

Figure 4.7 presents the optimized model accuracy/epochs. This model presents a 96.31 % and 89.63 % training and validation accuracy, respectively.

Table 4.4 presents the classification report of this LSTM model where the metrics

Chapter 4. Real-time LSTM-RNN Classification of Floors with Different Friction Coefficients for a Walking Humanoid Robot Wearing a 3D Force System

LSTM 300 (tanh)

Dropout 0.2 Dense 4 (softmax)

Input Layer

Ouput Layer

Figure 4.6: Optimized LSTM model.

0 2 4 6 8 10 12 14 16 18 20 22 40 60 80 100 Epochs Accuracy (%) Training Validation

Figure 4.7: LSTM model, Accuracy vs Epochs.

Table 4.4: LSTM model Classification report.

Precision Recall F1-score Samples
Acrylic-PTFE  0.98  1.00  0.99  1620
Acrylic-Aluminium  0.88  0.92  0.90  1510
Acrylic-PE-HD  0.93  0.84  0.89  1290
Acrylic-Melamine  0.93  0.89  0.91  1140

Precision, Recall and F1-score are used to evaluate the model’s capability of correctly classifying the different floors.

Figure 4.8 presents the confusion matrix obtained the applying the network to the testing set. From there it can be seen that the model was able to correctly classify 92.09 % of the test set data (confusion matrix true positives plus true negatives divided by total of samples), which are data never presented to the network before. Overall, the differences between the high scores achieved for the first class compared to the other classes can be justified not only because of the uneven distribution of the four classes, but also because the floor properties, more specifically the CoF of the first class differs more from the remainder. The gap between the CoF for the remainder classes is similar and, as it can be seen, the results for these classes are very balanced.

4.5 Online experiment

After training and improving the LSTM network, a real-time experiment of the NAO robot walking on different floors, as illustrated in Fig. 4.9, was carried out to validate the model. The gait parameters for the walk are as follows: 0.02 m step length; 0.02 m step height; 50% of robot’s maximum step frequency; and 0.1 rad torso rotation around Y.

Figure 4.10 plots the classification results of the LSTM network, and at the top, some

Chapter 4. Real-time LSTM-RNN Classification of Floors with Different Friction Coefficients for a Walking Humanoid Robot

Wearing a 3D Force System

PTFE {100.0% \atop 1620} {2.0% \atop 30} {4.7% \atop 60} {0.0% \atop 0} Alum. {0.0% \atop 0} {92.1% \atop 1390} {8.5% \atop 110} {7.0% \atop 80} PE-HD {0.0% \atop 0} {2.6% \atop 40} {84.5% \atop 1090} {3.5% \atop 40} Melam. {0.0% \atop 0} PTFE {3.3% \atop 50} Alum. {2.3% \atop 30} PE-HD {89.5% \atop 1020} Melam. Output Class Target Class

Figure 4.8: Confusion matrix of the testing dataset.

NAO PTFE Aluminium PE-HD Melamine

Figure 4.9: Layout for the real-time on-line experiment.

snapshots of the experiment

are shown. In this experiment, the robot walked 1.80 meters on the four floors starting on the PTFE floor and ending on the melamine surface. It is important to mention that we considered as being the correct floor on which the robot walks when the front part of the robot foot is at least 0.04 m over that floor. The detailed results show that the network, although not always with 100 % certainty, classified correctly (with a confidence always larger than 50 %) the different floors for the vast majority of the robot’s steps. Fig. 4.11 presents the confusion matrix for this experiment, where the LSTM model classified correctly 87 of the 90 (≈97 %) robot steps. Indeed, the network correctly classified all the steps on the PTFE and PE-HD floors, and misclassified only two steps on the aluminium floor and one step on the melamine floor. Overall, it can be seen that the LSTM model is suitable for this multi-classification problem. The model presented requires approximately 541 milliseconds to classify one input in real time (from data collection to obtaining a network output). Since the data are recorded at 100 Hz and we are using the first 50 data points (500 milliseconds) to feed the model, it means that the model classification time (41 milliseconds) represents only 7.5 % of the total time required for this multi-classification problem. Basically, after one robot step we need to wait 41 milliseconds before taking the procedures to optimize the next step. In the future, if we need to decrease the overall required time, we can focus on strategies to reduce the data points used to feed the model. We tested the LSTM network’s ability to classify correctly the different slippery floors using less timesteps and the results are shown in Fig. 4.12. We observed for example that the network test accuracy exhibits a loss of less than 4 % and 9 % when using 40

0 25 50 75 100 a b c d PTFE(%)

0 1

0 25 50 75 100

e f g h

Aluminium(%)

0 1

0 25 50 75 100 i j k l PE-HD(%)

0 1

0 5 10 15 20 25 30 35 40 45 50 0 25 50 75 100 m n o Time(s) Melamine(%)

0 1

Figure 4.10: Individual results on floor classification of NAO walking experiment in realtime using 50, 40 and 30 timesteps. At the top, some snapshots of NAO walking on the different surfaces are presented, and their corresponding output is presented, at the bottom, on the charts for each foot/shoe. ∘ Left ITshoe(50); • Right ITshoe(50); □ Left ITshoe(40); ■ Right ITshoe(40); △ Left ITshoe(30); ▲ Right ITshoe(30); Real floor on which the robot walks (it is 1 when walking on the current floor).

and 30 timesteps, respectively. In Fig. 4.10 it is visible that the network was still able to correctly classify most of the steps during the online experiment, using the 40 and 30 timesteps model, although with less confidence when compared with the 50 timesteps model. Fig. 4.13 and 4.14 show the confusion matrices for these two networks with less timesteps. The LSTM model with 40 timesteps classified correctly 86 of 90 robot steps, that is, one more misclassified step when compared to the 50 timesteps model, and the LSTM model using 30 timesteps only classified correctly 80 of 90, seven less than the original 50 timesteps model. In conclusion, it is possible to reduce the timesteps used to classify the different floors and still have a model able to classify correctly most of the robot’s steps.

In order to assess the relative importance of the variables involved in this process of floor identification, different training strategies were done using different sets of input variables, namely with horizontal forces only, robot own IMU only, and the combination of both. Fig. 4.15 shows the average test accuracy of the LSTM model when using only the horizontal forces (HFs), only the IMU data and lastly, both together (HFs+IMU) as

Chapter 4. Real-time LSTM-RNN Classification of Floors with Different Friction Coefficients for a Walking Humanoid Robot

Wearing a 3D Force System

PTFE {100.0% \atop 19} {0.0% \atop 0} {0.0% \atop 0} {0.0% \atop 0} Alum. {0.0% \atop 0} {92.0% \atop 23} {0.0% \atop 0} {4.8% \atop 1} PE-HD {0.0% \atop 0} {8.0% \atop 2} {100.0% \atop 25} {0.0% \atop 0} Melam. {0.0% \atop 0} PTFE {0.0% \atop 0} Alum. {0.0% \atop 0} PE-HD {95.2% \atop 20} Melam. Output Class Target Class

Figure 4.11: Confusion matrix of the real-time experiment.

1 5 10 15 20 25 30 35 40 45 50 20 40 60 80 100 Timesteps

Test Accuracy (%)

Figure 4.12: LSTM network testing accuracy using different sizes of timesteps.

was used throughout this work. We can already observe that the data coming from the ITShoes has a much larger performance in the classification than the IMU data alone, but we also observed that when we put together the data coming from the ITshoes and the IMU, the classification results further improve about 8 %.

4.6 Conclusions

This work addresses a multi-classification problem using a LSTM recurrent neural network model based on the ITshoes force data together with the robot internal inertial sensor data. A dataset of 27800 labelled steps of a walking biped robot over four different floors were collected and used to train, validate and test the multiple learning models. After several attempts to optimize the LSTM base model, and after tuning its parameters, we achieved a network capable of classifying the different floors with an accuracy of approx-

Chapter 4. Real-time LSTM-RNN Classification of Floors with Different Friction Coefficients for a Walking Humanoid Robot Wearing a 3D Force System

PTFE {100.0% \atop 19} {4.0% \atop 1} {0.0% \atop 0} {0.0% \atop 0} Alum. {0.0% \atop 0} {88.0% \atop 22} {0.0% \atop 0} {4.8% \atop 1} PE-HD {0.0% \atop 0} {8.0% \atop 2} {100.0% \atop 25} {0.0% \atop 0} Melam. {0.0% \atop 0} PTFE {0.0% \atop 0} Alum. {0.0% \atop 0} PE-HD {95.2% \atop 20} Melam. Output Class Target Class

Figure 4.13: Confusion matrix of the testing dataset using 40 timesteps.

PTFE {89.5% \atop 17} {8.0% \atop 2} {0.0% \atop 0} {0.0% \atop 0} Alum. {10.5% \atop 2} {84.0% \atop 21} {4.0% \atop 1} {4.8% \atop 1} PE-HD {0.0% \atop 0} {8.0% \atop 2} {92.0% \atop 23} {4.8% \atop 1} Melam. {0.0% \atop 0} PTFE {0.0% \atop 0} Alum. {4.0% \atop 1} PE-HD {90.5% \atop 19} Melam. Output Class Target Class

Figure 4.14: Confusion matrix of the testing dataset using 30 timesteps.

imately 92 %. The developed online experiment also validated the LSTM-RNN approach to this classification problem, as it correctly classified 87 out of 90 robot’s steps. For future work we will develop a thorough study of the impact of the data coming from the ITShoes and the IMU to validate the importance of the wearable instrumented system. Additionally, in the future it is expected to use this classification to optimize the humanoid robot controller, since it will be easier to achieve an efficient and stable humanoid gait if its controller has information about the floor where it walks on. Following this idea, we also expect that this network can be adapted to be used with different classes by analysing how close a new input will be from one of the studied classes. In fact, we observed that even in cases where classification shows smaller confidence in one material, the next best HFs IMU HFs+IMU 0 25 50 75 100 92% 84% 58%

Test Accuracy (%)

Figure 4.15: LSTM average test accuracy when using only the Horizontal Forces (HFs), only the data coming from the robot’s IMU and all the features (HFs+IMU) as described in sub-section 4.3.3.

option is, most often, a material with a neighbour friction coefficient. This shows that this solution is able to detect properly the closest friction coefficient from the set of possibilities trained. So, if enough materials are used to represent the required resolution of friction coefficients, the robot will be able to assess with adequate accuracy the interval of values for the friction coefficient where it is walking on, thus allowing the decision of measures to take to improve walking control. Although this classification is focused on the friction property of the materials, we will also consider to study other characteristics such as damping and stiffness not only to have a richer set of parameters to adapt the robot controller but also to broaden the possibility to detect different floors, i.e deformable floors.

When we classify the different but known floors we know in advance the floor properties/characteristics and the main challenge where more characteristics of the floors can be handy will be in the future when we try to interpolate this known properties with the classification of unknown floors.

Part III

Adaptive humanoid controller to Improve Humanoid Locomotion

on Slippery floors Enhancement of Humanoid Robot Locomotion on Slippery Floors using an Adaptive Controller

Submitted

Chapter 5. Enhancement of Humanoid Robot Locomotion on Slippery Floors using an Adaptive Controller

5.1 Abstract

This paper presents a comprehensive strategy to improve the locomotion performance of humanoid robots on various slippery floors. The strategy involves the implementation and adaptation of a Divergent-Component-of-Motion (DCM) based control architecture for the humanoid NAO, and the introduction of an Embedded Yaw Controller Embedded Yaw Controller (EYC), which is based on a Proportional-Integral-Derivative Proportional-Integral-Derivative (PID) control algorithm. The EYC is designed not only to address the slip behavior of the robot on low friction floors, but also to tackle the issue of nonstraight walking patterns that we observed in this humanoid, even on non-slippery floors. A series of locomotion experiments are conducted in a simulated environment, where the humanoid step frequency and PID gains are varied for each type of floor. The effectiveness of the strategy is evaluated using metrics such as robot stability, energy consumption, and task duration. The results of the study demonstrate that the proposed approach significantly improves humanoid locomotion on different slippery floors, by enhancing stability and reducing energy consumption. The study has practical implications for designing more versatile and effective solutions for humanoid locomotion on challenging surfaces, and highlights the adaptability of the existing controller for different humanoid robots.

5.2 Introduction

Humanoid robots are anthropomorphic machines that aim to emulate the locomotion and behaviour of humans. Although significant advancements have been made in robot’s locomotion, the challenge of navigating on slippery floors remains a difficult and unresolved issue, with significant potential for further improvements. Just as humans use their senses to adapt to changing environments, we believe that humanoid robots should rely on sensor feedback to achieve the same. By utilizing sensor data, robots can better adjust to varying surfaces and conditions, improving their ability to navigate on slippery floors and other challenging terrains ding_walking_2018 , as well as perform seemingly simple tasks like setting their home position kim_adjustment_2008 .

On past studies doi:10.1142/S0219843620500115 ; 9598809 , it was hypothesized that adapting the controller of the robot based on the known friction coefficient of various floors could lead to efficient and effective locomotion on such surfaces. Building upon the development of instrumented shoes (ITShoes) and machine learning algorithms that enable real-time detection of the coefficient of friction for various floor types, this work proposes a comprehensive strategy to enhance humanoid locomotion on slippery surfaces through adaptive control. In addition to addressing the challenges posed by varying slippery floor conditions, a peculiar behavior was also observed in the humanoid when instructed to walk on a straight path. This behavior is attributed to its unconventional building, which will be explain in the next section. As such, the proposed controller aims not only to address slip behavior on slippery floors, but also to mitigate the issue of non-straight walking patterns resulting from the humanoid’s particular mechanical construction.

Several techniques have been proposed in the literature for generating Center of Mass (CoM) trajectories based on the concepts of Zero-Moment Point (ZMP) VUKOBRATOVIC19721 and Linear Inverted Pendulum (LIP) Kajita2001The3L . In recent years, there has been significant research on bipedal locomotion, which has resulted in the introduction of new concepts such as the Divergent Component of Motion Divergent Component of Motion (DCM) and Virtual Repellent Point (VRP) Englsberger2013ThreedimensionalBW ; 3960bde270c5435b86c992f857e45dd9 . These concepts allow for the decomposition of the second-order dynamics of the CoM into two first-order linear dynamics, where the CoM converges to the DCM (stable dynamics) and the DCM diverges away from the VRP (unstable dynamics) Englsberger2013ThreedimensionalBW . These concepts have been utilized in various methods of generating CoM trajectories and stabilizing humanoid robots during locomotion.

The strategy described in this paper is founded on an implementation and adaptation of a DCM based control architecture. The DCM concept has been applied to various types of robots, including bipeds 9424385 ; 8404119 ; 9034996 ; doi:10.1142/S0219843619500348 , quadrupeds griffin2022quadrupedal ; wagner2021applying , and exoskeletons mehr2021intelligent ; xu2022dmp , and has been shown to be effective in achieving stable and dynamic locomotion. It has also been used in applications such as prosthetics and rehabilitation robotics, where maintaining balance and stability are critical for safe and effective movement. We follow the DCM-based control architecture presented in 1809.02167 , where the authors present and compare several DCM-based implementations of a layered control architecture. A unicycle-based planner is used to generate desired DCM and foot trajectories for the fixed trajectory optimization layer. The simplified model control layer comprises two main controllers for DCM tracking, along with an additional inner controller that utilizes 6-axes Force Torque sensors (F/T) to ensure precise tracking of CoM and ZMP 4359259 . The two main controllers are: an instantaneous feedback controller and a Model Predictive Control (MPC) approach. One common approach to generate feasible CoM trajectories is to combine the Linear Inverted Pendulum Model (LIPM) with MPC techniques, which are also referred to as Receding Horizon Control (RHC) Herdt2010OnlineWM . The MPC is a control technique that uses a mathematical model of the system to predict its future behaviour, and then optimizes the control inputs over a finite time horizon to minimize a cost function while satisfying constraints. The Whole-Body Quadratic Programming (WBQP) control layer ensures the tracking of the desired CoM and feet trajectories by presenting velocity and inverse kinematics.

In the current work, we propose an Embedded Yaw Controller (EYC) as an extension to the existing control strategy based on the MPC controller from 1809.02167 . The EYC is specifically designed to address slip behavior on slippery floors and to rectify the nonlinear paths observed due to the robot’s suspicious building technique. It functions as a Proportional-Integral-Derivative (PID) controller, which allows us to dynamically control and adapt the humanoid gait in real-time. This approach draws upon the well-established use of PID controllers in optimizing various aspects of humanoid locomotion, such as stability, efficiency, and robustness of walking, as demonstrated in prior works such as nguyen2020balancing ; bestmann2022bipedal ; xu2022design . Thus, in this work, the digital PID control scheme is leveraged to implement the EYC for precise control of the humanoid gait, by continuously monitoring its torso orientation in response to slip behavior and non-linear path observations.

The remainder of the paper is structured into the following sections. Section 5.3 provides an overview of the DCM-based control architecture that has been implemented, covering key concepts, components, and its integration into the NAO robot. Furthermore, the integration of an EYC within the architecture is detailed. Section 5.4 describes the preliminary locomotion experiments setup and the experiments conducted with varying step frequencies and PID gains. The metrics used to evaluate the performance of the system are also discussed, and the results obtained are presented. Finally, in Section 5.5, conclusions are provided, highlighting the achieved improvements, and recommendations for future work are given.

5.3 Humanoid Whole-body Controller

The whole-body controller used in this work is based on the three-layer control architecture developed and implemented on the iCub robot 1809.02167 . The three layer architecture is presented in Figure 5.1.

Control Unit Trajectory Opti- mization Simplified Model Whole Body Control Humanoid Robot Desired DCM Desired CoM Velocity Feet Pose Joint Velocity Joints & CoM Velocity Contact Wrenches EYC Yaw Angle Desired Yaw iCub NAO+ITShoes

Figure 5.1: Modified humanoid 3-layer control architecture 1809.02167 with an additional EYC block. The EYC is a controller used to adapt the robot’s motion to low-friction environments and to correct the unexpected curved path of the Humanoid NAO by adjusting its behavior based on feedback from an IMU (Inertial Measurement Unit) sensor

This work demonstrates the potential of the Embedded Yaw Controller (EYC) by successfully integrating it into the three-layer control architecture, where it effectively addresses slip behavior and rectifies non-linear paths resulting from the robot’s particular building technique. Furthermore, the versatility of the three-layer controller is demonstrated by successfully implementing it on a different humanoid robot, highlighting its potential for application in a wide range of robotic systems.

5.3.1 Trajectory optimization

The purpose of this layer is to plan and evaluate the desired footstep positions and the desired feet and DCM trajectories for a humanoid robot. The robot is approximated as an unicycle, with the feet represented by the unicycle wheels. The footsteps are planned by sampling the unicycle trajectories, with each position associated with a time instant. The impact time is considered as a decision variable, which allows for the selection of feet positions, duration, and step length. Once the footsteps are planned, the desired feet trajectory is obtained by cubic spline interpolation. The DCM trajectory is chosen so as to satisfy a specific time evolution. The DCM trajectory along the walking pattern can be computed recursively, but the presented planning method only takes into account single support phases. To ensure that the ZMP trajectory remains smooth and continuous, it is important to have a reference trajectory for the DCM with smoothly varying derivatives. This means that the rate of change of the DCM trajectory should be continuous, allowing for seamless transitions between different segments of the trajectory. To achieve this, a third-order polynomial is used to smoothly connect the edges of the DCM reference trajectory. The parameters of the polynomial are carefully selected to ensure that both the velocity and position of the trajectory satisfy certain boundary conditions, ensuring a continuous and smooth motion 1809.02167 .

5.3.2 Simplified Model

The middle layer of the robot’s control system utilizes a simplified model control layer that employs a MPC DCM controller. The purpose of this controller is to ensure that the robot’s DCM accurately tracks its intended trajectory. The DCM is a specific point in space that represents the robot’s movement and serves as the point at which the robot would fall if it stopped moving.

To achieve this level of precision, the humanoid robot’s movement is approximated using the LIPM, a widely recognized model for walking on flat surfaces kajita20013d . The LIPM guarantees that the CoM of the robot remains at a constant height on a horizontal plane while walking. The MPC algorithm utilizes this model to predict the robot’s future movements and generates a control signal that maintains its stability and trajectory.

This algorithm takes into account the desired trajectory and the robot’s motion constraints to generate a smooth trajectory for the ZMP, which is essential to the robot’s stability. The ZMP must remain within the robot’s support polygon to avoid falling.

In this middle layer, an independent CoM-ZMP controller ensures that the robot’s CoM and ZMP are properly tracked using a force torque sensors (F/T). This inner CoM-ZMP controller is presented in 4359259 and is based on the Kinematic Resolution of CoM Jacobian (KRCJ) method with embedded motion. KRCJ is a mathematical method for computing the robot’s center of mass and its Jacobian matrix, which describes the relationship between the robot’s joint angles and its CoM.

This approach aims to enhance the robot’s stability and balance during walking by adjusting its joint angles in real-time using the KRCJ method. The embedded motion technique allows the robot to adapt its movements to changes in the environment, such as uneven terrain or unexpected obstacles 1809.02167 .

5.3.3 Whole Body QP Control

The third layer of the control architecture implemented in this work is the WBQP control layer. WBQP is a common approach used in robotics for generating motion plans or control commands that optimize the performance of the robot’s whole body motion while accounting for different constraints. Its main responsibility here is to track the desired CoM and foot trajectories, employing velocity and inverse kinematic controllers to ensure accurate and smooth movement control. The achievement of this control objective involves defining a cost function, which is minimized to obtain the desired robot velocity 1809.02167 .

5.3.4 Implementation in the NAO robot

The entire architecture was adapted to be used in the humanoid NAO. The NAO robot displays certain peculiarities observed in both simulations and the real system. The most significant issue is that the robot does not maintain a straight path when commanded to do so. This behaviour is illustrated in Figure 5.2, where it can be seen that the humanoid changes direction despite being commanded to walk a straight line.

Figure 5.2: Illustration of the NAO robot’s inability to maintain a straight path. The robot was commanded to walk in a straight line, but deviated from its intended path

The issue of the robot deviating from a straight path is more prominent on slippery floors, but it also occurs on floors with a high friction coefficient. We suspect that the problem is related to the physical configuration of the pelvis joint LHipYawPitch and RHipYawPitch, which are driven by only one motor and, thus, cannot be controlled independently, with priority given to LHipYawPitch. It is possible that the controller has a glitch when sending the desired position for both joints, leading to the observed behaviour.

Illustrative trajectories are depicted in Figure 5.3, showcasing the desired straight trajectory and the actual trajectories of the humanoid.

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 −0.4 −0.2 0 Forward distance (m)

Lateral deviation (m) trajectory 1 trajectory 2 trajectory 3 desired trajectory

Figure 5.3: Visualizing humanoid locomotion behavior: trajectories illustrating straight walking command execution

To enforce the locomotion along a straight path and correct the lateral deviations of the robot, the EYC control system was developed based on the measurement of robot’s heading using the onboard Inertial Measurement Unit (IMU). This controller dynamically adjusts the target set-point to continuously correct the robot’s direction of motion by monitoring the humanoid’s torso orientation and compensates for changes in direction that can occur when the robot slips while walking on very slippery floors.

The effectiveness of this modification was also evaluated through experiments on the humanoid robot NAO, and the results showed that the EYC was able to significantly improve the robot’s stability and reduce the impact of slips on its movements. A joint velocity limit was also added to the controller as a safety measure to ensure that the robot’s joint velocity limits were not exceeded during movement. This modification was necessary to ensure the safety and stability of the robot’s movements, especially during dynamic motions that require quick changes in joint velocities.

To implement this architecture in the NAO robot, a detailed step-by-step process was followed. First, the necessary software dependencies for the walking module were installed, along with YARP and any required environmental variables. Next, the robot was modeled using both the Unified Robot Description Format (URDF) and the Simulation Description Format (SDF) files. This involved defining joint and link structures in the URDF, as well as specifying physical and environmental properties in the SDF, such as friction coefficients. With the robot model in place, the focus shifted to adapting the trajectory generation parameters to suit the specific requirements of the NAO. This involved setting limits for various gait parameters, such as step length, frequency, and height, among others, to ensure that the robot could walk smoothly and efficiently. Additionally, the trajectory generation parameters were fine-tuned to ensure that the robot’s motion was stable and well-coordinated. Throughout the implementation process, all necessary adaptations were made to ensure that the model was comprehensive and incorporated all the joints and sensors needed to achieve optimal performance. To achieve a final implementation that is both robust and reliable, it was necessary to adapt numerous parameters and conduct extensive testing. Some of these adaptations are presented in Table 5.1:

Item  Values
URDF  Joints, links, and robot description.

SDF Model, physical properties, environmental and material properties (i.e. friction coefficients).

Whole-body configs

Joints, frames, sensors, devices and others

Trajectory generation Gait parameters, such as minimum and maximum step length, height, width, duration, landing velocity and others.

Logger Joints position, DCM, CoM, ZMP, sensors, EYC inputs/outputs and others.

Walking controller configs Inner ZMP parameters, forward and inverse kinematics, joint’s PID and others

Gazebo Physical environmental, such as solver and constraints parameters

Table 5.1: Points adapted in the original walking controller to fit the NAO requirements.

To measure the forces and torques that occur on the ankle of the NAO humanoid robot during the simulated experiments, a Force/Torque (F/T) sensor was added to the robot’s ankle model. While this provided accurate measurements in the simulated environment, the real robot does not have this sensor. To overcome this limitation, we aim to utilize the instrumented shoe (ITshoe), as shown in Figure 5.4, which was previously developed in our laboratory 8374157 .

The ITshoe is equipped with sensors that can measure the total ground reaction forces during walking. These measurements are essential to enhance the robot’s interaction with the ground leewholebody2021 , and they can be utilized to calculate the forces and torques acting on the ankle. To simulate the output of a F/T sensor in future real experiments, the measured ground reaction forces have to be translated to the ankle location using the Equation 5.1 and Equation 5.2.

F⃗_{AZ} = F⃗_{SZ} + (m_{is} + m_{rf} + m_{aj}) · g⃗,  (5.1)

where F⃗_{AZ} represents the total ankle force in the z direction, F⃗_{SZ} the total force in the z direction measured by the ITShoe and m_{is}, m_{rf} and m_{aj} denote the masses of the components between the sensors and the robot’s ankle:

[ τ⃗_{x} ; τ⃗_{y} ; τ⃗_{z}] = ∑_{i=1}^{n} [ \begin{matrix}r_{iy} · F⃗_{iz} + r_{iz} · F⃗_{iy} \\ r_{iz} · F⃗_{ix} + r_{ix} · F⃗_{iz} \\ r_{ix} · F⃗_{iy} + r_{iy} · F⃗_{ix} \end{matrix} ] ,  (5.2)

Figure 5.4: ITshoe schematic structure. The red block (A) is the acquisition unit, the green block (B) is the streaming unit, and the blue block (C) is the sensing unit. At the top, the image shows the position of the eight force sensors and the reference axis used to decompose the tangential forces 9598809 .

where τ⃗_{x}, τ⃗_{y} and τ⃗_{z} represent the total ankle torque in the corresponding x, y and z directions, r_{ij} , for j ∈ x, y, z, are the distances between the ITShoe sensors and the robot’s ankle, and F⃗_{ij} , for j ∈ x, y, z, are the forces measured on the corresponding sensors in the ITshoe.

Then, in the Gazebo simulator, the physical parameters, such as mass, friction coefficients, inertias, and joint damping, were fine-tuned and optimized to ensure that the simulated robot’s behavior closely emulated the real-world physics and dynamics of the actual robot.

To capture and analyse the relevant data for each experiment, all the DCM based controller variables, along with the robot’s joint, ZMP, CoM, F/T and IMU sensor data, and other essential variables were recorded within a 10 ms controller cycle. By gathering this information, we were able to gain a comprehensive understanding of the robot’s behaviour and performance in response to different slippery floors. In the following sections, it is demonstrated how effectively this data was utilized and manipulated to draw meaningful conclusions about the effectiveness of the methods used.

The control architecture was implemented using YARP, an open-source robotics middle-ware, and programmed in C++ language. More details can be found at the walking controllers repository^{1}.

The layout to simulate the humanoid robot navigating on different types of floors was implemented using the Gazebo simulator, as illustrated in Figure 5.5.

1 https://github.com/robotology/walking-controllers

Robot NAO ITshoes Slippery floors X Y Z

Figure 5.5: Layout of the simulated environment featuring the NAO robot and various types of slippery floors in the Gazebo simulator.

EYC implementation

The role of the Embedded Yaw Controller (EYC) is to continuously monitor the robot’s orientation, specifically the YAW angle, during locomotion. When the EYC detects that the robot is deviating from the commanded trajectory, it calculates a new YAW angle using a digital PID formulation ASTROM20011163 , as shown in Equation 5.3, in order to adapt the robot’s trajectory accordingly.

u_{0} = −K_{u1}u_{1} − K_{u2}u_{2} + K_{e0}e_{0} + K_{e1}e_{1} + K_{e2}e_{2}.  (5.3)

This controller relies on a set of carefully selected and tuned coefficients, which are defined in Equation 5.4 to Equation 5.14. These coefficients are used in the computation of the control input, u_{0}, which depends on the previous control inputs u_{1} and u_{2}, as well as the error terms e_{0}, e_{1}, and e_{2}. The equation includes the derivative terms K_{u1} and K_{u2}, and the proportional, integral, and derivative gains K_{e0}, K_{e1}, and K_{e2}, given by:

K_{u1} = \frac{a_{0}}{a_{1} − a_{2}} ,  (5.4)
K_{u2} = \frac{a_{0}}{a_{1} + a_{2}} ,  (5.5)
K_{e0} = a_{0}b_{0},  (5.6)
K_{e1} = a_{0}b_{1},  (5.7)
K_{e2} = a_{0}b_{2},  (5.8)

where

a_{0} = 1 + NT_{s},  (5.9)
a_{1} = −(2 + NT_{s}),  (5.10)
a_{2} = 1,  (5.11)
b_{0} = K_{p}(1 + NT_{s}) + K_{i}T_{s}(1 + NT_{s}) + K_{d}N,  (5.12)
b_{1} = −(K_{p}(2 + NT_{s}) + K_{i}T_{s} + 2K_{d}N),  (5.13)
b_{2} = K_{p} + K_{d}N.  (5.14)

To trigger the controller, a timer interrupt service routine is used, and a set of global variables is defined, including the command input r, plant output y, error terms e_{0}, e_{1}, and e_{2}, control inputs u_{0}, u_{1}, and u_{2}, and the coefficients a_{0}, a_{1}, and a_{2}. The error term is computed by taking the difference between the desired output and the current plant output. The proportional, integral, and derivative gains are initially set to K_{p} = 1.00, K_{i} = 0.02, and K_{d} = 0.01, respectively, while the filter coefficients are set to N = 20, and the sampling time is set to T_{s} = 0.01 s.

These variables and parameters are critical to the operation of the PID controller, and have been empirically selected and tuned to ensure an initial effective stabilization of the robot’s orientation on slippery floors. Further explanation will be provided in section 5.4 to elaborate on the specific process of how these variables and parameters were determined based on practical experimentation and observation.

To estimate the robot’s orientation, data obtained from the NAO robot IMU sensor was utilized, which provided measurements of the robot’s angular velocity and linear acceleration. To estimate the yaw angle, we applied an Extended Kalman Filter (EKF), which is a recursive algorithm that estimates the state of a system that is subject to uncertain and noisy measurements Julier1997NewEO .

The EKF uses a quaternion to represent the orientation of the system, where the scalar component represents the orientation of the system, and the three vector components represent the axis of rotation. The prediction stage of the filter propagates the quaternion forward in time using the system dynamics, while the update stage fuses the predicted state with new measurements. In the case of an IMU, new measurements include measurements of angular velocity. To update the state estimate with the new measurements, the filter models the measurement process as a linear function of the state, corrupted by additive Gaussian noise. Finally, the EKF computes the optimal estimate of the state by minimizing the mean squared error between the predicted state and the measurements. By applying this algorithm, we were able to obtain an optimal estimate of the robot’s torso YAW angle over time. This estimate was then used as input to the EYC controller to stabilize the robot’s trajectory, taking into consideration not only the slippery condition of the floors, but also the peculiar issue related to the humanoid’s non-straight walking patterns mentioned earlier. The EYC controller, utilizing the PID formulation, adjusts the robot’s trajectory to ensure more stable and accurate locomotion, mitigating the effects of both slippery surfaces and the humanoid’s unique build.

5.4 Locomotion experiments

In this section it is presented the experiments carried out to collect locomotion data in order to analyse it and improve the humanoid controller based on the slippery floor it is faced with. Firstly, we performed an experiment where we attempted to study the impact of the robot’s step frequency on the different floors by measuring the amount of slippage occurred during these experiments. According to the information obtained from these experiments, it was found that the slowest step frequency was the most preferred option across all floors, rather than a specific frequency for each individual floor. While this insight provides valuable information for gait adaptation on slippery floors, it is inconclusive towards our goal of adapting the controller to the specific floor the robot is traversing. As a result, a second experiment was conducted where the optimized parameters for the PID controller specific to each floor were obtained.

Secondly, in order to evaluate the following experiment, performance metrics are defined, specifically, the time and energy spent on the task, and the stability of the robot during these tasks. Lastly, the experiments to obtain the optimized PID parameters that allow the robot to achieve the best performance on each floor individually to improve the overall experience that is when the robot walks on the four different floors are presented. Four different slippery floors that were previously studied were used to test the stability and slip recovery ability of the robot 8374157 . The floors used in this study were characterized by their coefficient of friction when in contact with the humanoid robot’s instrumented shoe (ITshoe). Specifically, the floors exhibited coefficients of friction of 0.33, 0.26, 0.20, and 0.11, respectively, when in contact with the ITshoe.

5.4.1 Step frequency experiments

In this study, a series of experiments were conducted in a simulated environment to investigate the impact of humanoid locomotion on different slippery floors. The aim was to gain a deeper understanding of the relationship between the robot’s step frequency, which refers to the time it takes for the robot to perform one step, and the friction coefficients of four different floors to identify the best humanoid gait for real-world settings. To achieve this, a controlled experiment was designed and implemented, in which the humanoid step frequency was varied and the humanoid CoM was measured under different types of slippery floors.

The experiment involved the humanoid robot walking five times for 60 seconds on each of the four different floors, using five different frequencies. The chosen frequencies were the maximum frequency of the default humanoid NAO, the minimum frequency, and three equally separated frequencies between the maximum and minimum frequencies: 0.43, 0.47, 0.51, 0.55, 0.59 s/step. After collecting data from 100 experiments (i.e., 4 floors × 5 frequencies × 5 repetitions), data manipulation was performed to quantify the extent of slippage of the robot on each floor. To achieve this objective, the average absolute Y-position of the CoM for each experiment was calculated.

In the next step, the average change in the direction of the robot’s feet at each step was computed. This was done by taking the derivative of the average CoM Y-position data points. Then, a second derivative was applied to obtain the values that represent how much the robot slipped on each step. To exclude small slip actions that may occur due to possible errors associated with the robot’s joints, controller, and/or the simulator environment, a slip threshold was added to consider only significant slip actions. The optimal threshold was determined empirically by observing the various outputs of the second-order derivative and selecting the value that best matched the noticeable slip observed in the different experiences. Specifically, a threshold of 1 × 10^{−6} m s^{−2} was chosen. To normalize the results, the accumulated slip was divided by the number of steps in each experience since different step frequencies represent different amounts of steps. Equation 5.15 describes the steps used to quantify the robot’s slip in each experience.

S(i) = \frac{d^{2}A(i)}{dt^{2}},  |S(i)| > θ ⇒ T = \frac{∑ |S(i)|}{n},  (5.15)

where S represents the second-order derivative of the discrete sequence A(i), which is calculated as the difference between A(i) and A(i − 1), and A(i + 1) and A(i), i.e., S(i) = (A(i) − A(i − 1)) −(A(i + 1) − A(i)) = 2A(i)−A(i−1)−A(i+1). This equation is used to estimate the rate of change of the CoM Y-position at each step, which is a key parameter for detecting slips. θ represents the slip threshold, n represents the number of steps and T represents the total slippage that occurred in a complete experience. It is important to note that the absolute value of each detected slip was summed since we did not distinguish the direction of the slip.

As an example of the procedure described, Figure 5.6 and Figure 5.7 depict the CoM Y curves obtained for three examples of the resulting lowest slip sum and three examples of the resulting highest slip sum, respectively.

0510 15 20 25 30 35 40 45 50 −0.1 00.1 0.2 steps CoM Y (m) example 1 example 2 example 3

Figure 5.6: Absolute centre of mass position in the Y-axis for three exemplary walking trials on a flat surface resulting in the least amount of slip

010 20 30 40 50 60 70 00.2 0.4 steps CoM Y (m) example 1 example 2 example 3

Figure 5.7: Absolute centre of mass position in the Y-axis for three exemplary walking trials on a flat surface resulting in the highest amount of slip

The slip corresponding to the previous examples is illustrated in Figure 5.8 and Figure 5.9, respectively. Among these examples, we selected floor 3 with the lowest frequency as the best representative and floor 1 with the highest frequency as the worst representative.

The overall results of the experiment are presented in Figure 5.10.

The study indicates that a higher nominal step frequency, which corresponds to the robot taking longer time durations per step, results in more favorable outcomes in terms of reduced slipping on all types of floors. Moreover, we observed a proportional relationship between step frequency and outcomes, with lower frequencies resulting in higher slip sums and vice versa. These findings demonstrate that, similar to humans, robots must adjust their gait to maintain stability on slippery surfaces. In our study, this involved reducing the step frequency to minimize the risk of slipping or falling.

Given that a specific step frequency for each floor was not identified, this study fell short of our goal of real-time adaptation of the controller to optimize walking on diverse surfaces. To address this limitation, we conducted a follow-up experiment focused on the EYC controller. The results of this study provided valuable insights into optimizing the

0510 15 20 25 30 35 40 45 50 00.5 11.5 ·10^{−6} steps Slip (m s^{−2} ) |e1| |e2| |e3|  Threshold

Figure 5.8: Slip and threshold visualization for the top three experiences with the least amount of slip (e1, e2, e3 represent examples 1, 2, 3)

0510 15 20 25 30 35 40 45 50 55 60 65 0123·10^{−6} steps Slip (m s^{−2} ) |e1| |e2| |e3|  Threshold

Figure 5.9: Slip and threshold visualization for the top three experiences with the highest amount of slip (e1, e2, e3 represent examples 1, 2, 3)

controller’s parameters to improve the robot’s stability and mobility on various floors.

5.4.2 EYC adaptive controller experiments

For the EYC controller to adapt the humanoid’s locomotion based on its orientation, the PID parameters that would yield the best results were investigated. To determine the suitable PID parameters for stabilizing the humanoid robot orientation on slippery floors, a series of experiments on gazebo simulator was performed. Before describing the experiment in detail, it is essential to define the metrics used to evaluate the robot’s performance on different surfaces.

Floor 1 μ = 0.33 Floor 2 μ = 0.26 Floor 3 μ = 0.20 Floor 4 μ = 0.11 0123452.8 2.7 2.5 2.8 3.2 3.3 2.9 2.8 3.4 3.4 3.4 3.3 3.6 3.6 3.4 3.5 4.2 44.2 4Slip Sum (10^{−6} ) (m s^{−2} ) nsd 0.58 nsd 0.54 nsd 0.51 nsd 0.47 nsd 0.43

Figure 5.10: Comparison of slip sum for different step frequencies and slippery floors, where ’st’ denotes the step time in seconds for a robot’s stride.

Performance metrics

In order to assess and compare the performance of the controller and define the best parameters, some metrics are proposed: task time duration (T_{D}), stability (S_{idx}), and energy consumption (E_{C}). These metrics have been established and utilized in previous literature, as evidenced by aller2019state .

The task time duration refers to the time necessary for the humanoid robot to complete a given task, such as walking a certain distance on a slippery surface. This metric allows us to compare the efficiency of different control strategies in terms of completing tasks quickly and accurately.

The stability of the robot was measured using the root mean square error (RMSE) of the Center of Mass (CoM) and Zero Moment Point (ZMP) trajectories in relation to ideal straight line trajectories, where, for a generic quantity A:

RMSE(A) = \frac{1}{N} ∑_{i} \sqrt{(A_{i}^{real} − A_{i}^{plan}) ^{2}}.  (5.16)

Each component of RMSE, including CoM in x and y directions and ZMP in x and y directions, was given equal weight, accounting for \frac{1}{4} of the total weight in the calculation of RMSE. A lower RMSE indicates better stability, as the robot is better able to maintain its balance and avoid falling on the slippery surface. The stability index is defined as:

S_{idx} = \frac{1}{4} (RMSE_{CoM_{x}} + RMSE_{CoM_{y}} + RMSE_{ZMP_{x}} + RMSE_{ZMP_{y}}) .  (5.17)

Finally, to evaluate the energy balance of the control strategies, the energy consumption of the different experiments was compared using a simplified metric. Given the relatively small and lightweight nature of the NAO humanoid robot, the contribution of friction and other energy losses was negligible in comparison to the energy consumed by the robot’s actuators. Therefore, for simplicity and practicality, the energy consumption was approximated by integrating, for all time steps (k = 0, ..., N_{T}), the product of the joint torque (τ_{i}) with the angular velocity (ω_{i}) for all the robot joints (i = 1, ..., N_{J}), during each Δt. This allowed us to focus on the energy consumed by the joints as the main sources of energy consumption for the robot. The energy consumption is defined as:

E_{C} = ∑_{k=0}^{N_{T}} ∑_{i=1}^{N_{J}} τ_{i}(k)ω_{i}(k)Δt.  (5.18)

This simplified metric enabled us to compare the performance of the different control strategies and identify the most energy-efficient approach for the tasks at hand, while disregarding other factors, such as robot size, design, environment, and power source that would have contributed only marginally to the overall energy budget.

The performance of the implemented digital PID controller on four different slippery floors was evaluated using a combination of the three metrics, as shown in Equation 5.19.The system’s performance decreases as the value of M increases.

M = \frac{1}{3} ( \frac{T_{D}}{T_{D_{max}}} + \frac{E_{C}}{E_{C_{max}}} + \frac{S_{idx}}{S_{idx_{max}}} ) ,  (5.19)

where T_{D_{max}} , E_{C_{max}} , and S_{idx_{max}} correspond to the maximum values obtained across all the experiments.

The equation calculates the overall performance metric M, which is a weighted sum of normalized duration, energy, and the average RMSE of the CoM and ZMP positions over both axes. The normalization was performed using the maximum value of each metric to respectively scale them to a common range.

Digital PID controller optimization

To adjust the PID parameters, the initial setup was manually tuned to establish a baseline for comparison. Data was then collected on the robot’s performance on four different slippery surfaces, systematically varying the P, I, and D gains. This allowed for the evaluation of the impact of each parameter on the robot’s stability and mobility and identification of the optimal combination of parameters for each surface. By using this approach, the PID controller’s parameters were fine-tuned to enhance the robot’s performance on a variety of surfaces, ensuring optimal stability and mobility.

After each experiment, the results were evaluated using the defined metrics. Due to the complexity of the model, an analytical approach was not feasible, and an empirical approach was employed to identify the set of PID parameters that resulted in the best performance according to the metrics. Finally, the parameters that yielded the best performance across all experiments were selected for further analysis and comparison.

As will be demonstrated in this section, the optimized value of the integral parameter will be set to zero, effectively reducing the PID controller to a PD controller.

Optimized PID gains To illustrate the results of our experiments, we present three graphics showcasing the performance of the controller on the first floor, which has a coefficient of friction of 0.33. Chart in Figure 5.11 displays the performance task results for five repetitions for each variation in the proportional gain while fixing the integral and derivative parameters. During the experiment, the controller was used while the robot walked 2 meters on the floor. Chart in Figure 5.12 presents the performance results using the optimized proportional gain and the fixed derivative gain while varying the integral gain. Finally, Chart in Figure 5.13 shows the results obtained with the optimized proportional and integral gains while varying the derivative gain.

0.50 0.90 0.95 0.96 0.97 0.98 0.99 1.00 1.01 1.02 1.10 0.6 0.8 1 0.9 0.93 0.87 0.89 0.87 0.85 0.9 0.87 0.91 0.89 0.93 Proportional gain

Task result

Figure 5.11: Experiment proportional gain impact on the task performance.

0.00 0.01 0.02 0.03 0.05 0.10 0.20 0.6 0.8 1 0.85 0.91 0.86 0.88 0.89 0.91 0.92 Integral gain

Task result

Figure 5.12: Impact of integral gain on task performance.

The optimized digital PID gains were determined as P=0.98, I=0.00, and D=0.02 for the first floor, using it as an example for our task. The same procedure was repeated for each floor, resulting in the corresponding optimized PID gains, as shown in Table 5.2.

0.00 0.01 0.02 0.03 0.05 0.10 0.6 0.8 1 0.96 0.85 0.94 0.89 0.91 0.93 Derivative gain

Task result

Figure 5.13: Experiment derivative gain impact on the task performance.

CoF

0.33  0.26  0.20  0.11
P  0.98  0.99  1.01  1.04
I  0.00  0.00  0.00  0.00
D  0.01  0.01  0.01  0.01

Table 5.2: Best PID gains for each floor

As the floor becomes increasingly slippery, we observe that the proportional gain (P) of the controller increases to enhance its responsiveness. This is because a more slippery surface tends to induce slower responses from the system to changes in orientation, necessitating a higher P value to achieve the desired level of control. Our empirical analysis revealed that the optimal integral term for this controller is I=0, indicating that the system is able to compensate for any steady-state error without requiring the integral term. Furthermore, the derivative gain obtained was D=0.01 for all experiments, suggesting that the current level of derivative gain is appropriate and does not require adjustments in response to changes in floor slipperiness.

Performance evaluation: variable vs. fixed PD gains To demonstrate the effectiveness of the approach, a series of experiments were conducted in which the humanoid robot walked 2 meters on four different slippery floors, each 50 cm long, and repeated the process five times. The performance of using a fixed-gain PD controller with the best gains for the first floor was compared against a variable-gain PD controller with optimized gains for each floor that the robot walked on. The results presented in Table 5.3 show that implementing the best PD gains for each floor led to a significant improvement in the robot’s overall performance. On average, the robot achieved an approximately 10 % increase in performance, with the most noticeable improvements observed on the last two floors, which had lower friction. The primary factor contributing to the improvement observed with the variable PD control method was the robot’s resulting enhanced stability performance. Moreover, using the variable PD also resulted in lower energy consumption during task execution. However, the difference in task completion time between the two PD control methods was insignificant, with a slightly better performance observed for the fixed PD. Overall, when considering the three metrics with equal weight, the impact of the variable PD control method is evident, particularly on floors with reduced friction.

Variable PD per floor  Fixed PD on all floors
M floor 1  0.9997  0.9999
M floor 2  0.9773  0.9996
M floor 3  0.7568  0.9998
M floor 4  0.8716  0.9987
M experience  0.9013  0.9995

Table 5.3: Experiment results by floor and overall experience, comparing the performance of the robot using optimized PD gains for each floor versus using the same gains for all floors.

Figure 5.14 shows an example of one of the experiments CoM Y to illustrate the stability using a fixed-gain PD versus an adaptive-gain PD.

0510 15 20 25 30 35 40 45 50 −4 −2 0246·10^{−2} P=0.98 P=0.98 P=0.98 P=0.98 P=0.98 P=0.99 P=1.01 P=1.04 CoF: 0.33 CoF: 0.26 CoF: 0.20 CoF: 0.11 time (s) CoM y (m) CoM y adaptive PD  CoM y non-variable PDDesired CoM y

Figure 5.14: Comparison of absolute centre of mass position in the Y-axis using an adaptive PD controller vs non-variable PD

From the graph in Figure 5.14, it can be observed that, initially, the pattern is similar for both the adaptive and fixed PD controllers. This can be attributed to the floors being less slippery and the proportional value being the same for both controllers in the beginning. However, as the experiment progresses towards more slippery floors, the adaptive PD controller exhibits a more effective response to the slippage caused by the lower friction of these floors. The controller’s adaptability to the floor on which the robot is walking is attributed to our prior research, which involved the development of an ITShoe for measuring ground reaction force data, and the implementation of a Long-Short Term Memory (LSTM) network to identify the various floor types.

It is important to note that, for the defined performance metric, a lower value indicates better performance. In other words, achieving a smaller task time, lower energy consumption, and less stability error are desired outcomes.

This demonstrates the versatility and effectiveness of the controller used in this work to adapt to different slippery floor conditions.

The experimental results presented here are illustrated in a video available on YouTube^{2}, which provides a better understanding of the problem and the observed behaviours.

5.5 Conclusion

In this study, we have demonstrated the versatility and potential of the controller developed in 1809.02167 by successfully adapting and implementing it to control our humanoid robot. Our findings highlight that the integration of an Embedded Yaw COntroller (EYC) has significantly enhanced the overall performance of the robot while walking on slippery floors. This improvement was particularly evident when the robot was exposed to different types of slippery surfaces, effectively mitigating the challenges posed by the varying conditions. Furthermore, the EYC has also successfully rectified the non-linear paths resulting from the robot’s unconventional building technique, contributing to a more stable and controlled gait. It is worth noting that we assumed the robot’s ability to identify the type of floor it is walking on, as this issue was resolved in a prior study 9598809 using the ITShoe and a LSTM. As part of future work, the plan is to conduct tests on the physical humanoid robot and make necessary adjustments to seamlessly integrate floor detection with the controller adaptation. This would involve transitioning from ROS (Robot Operating System) to YARP (Yet Another Robot Platform) and translating the forces measured by the ITShoe to ankle forces and torques for the robot.

2 https://youtu.be/aWqmWoWTbv8

Part IV

GENERAL DISCUSSION AND CONCLUSIONS

Chapter 6

Discussion and Concluding Remarks

Chapter 6. Discussion and Concluding Remarks

6.1 Discussion

The general discussion of the thesis is structured into three major sections, each contributing to the overall objective of enhancing humanoid locomotion on slippery floors.

The first section (Chapter 2) is dedicated to the development of the ITShoe, a lowcost, wireless, and adaptable solution designed for measuring the net GRFs in humanoid robots. The ITShoe not only provides valuable insights into sensor positioning, but it also offers a simple architecture that streamlines data gathering and manipulation processes.

The primary objective of this section extends beyond addressing the needs of humanoid NAO robots navigating slippery floors. It also focuses on the seamless integration of the ITShoe with a wide range of humanoid robots. The development of the ITShoe was carefully considered with future adaptability in mind, allowing for easy customization by changing its outer design to match the foot design of each specific robot. This approach ensures that the ITShoe can be effortlessly integrated into different humanoid robots, enhancing their locomotion capabilities on real-world surfaces.

Through our experimental results, we demonstrate the practical application of the ITShoe in measuring ground reaction forces. While the precise magnitudes, particularly for horizontal forces that are challenging to quantify, may not be exact, we highlight the significance of observing variations in these forces. Even without resorting to mathematical approaches, visual examination allows us to perceive the differences across various types of floors. However, a significant challenge lies in the lack of a robust system for consistently placing the ITShoe on the robot’s foot, ensuring precise pressure measurements. This issue represents a crucial area for future improvement, as similar-looking slippery floors can have varying effects on robot stability. Thus, detecting even small variations in the total GRFs using the ITShoe becomes essential for enhancing the overall performance and adaptability of humanoid robots on such surfaces.

The second part of the thesis (Chapter 3 and 4) explores and implements learning approaches to interpret the measured ground reaction forces. These approaches harness the power of machine learning techniques to extract meaningful insights and patterns from the force data.

Chapter 3 introduces the initial approach for classifying slippery floors. While we achieved successful classification results using neural networks on test datasets, we encountered challenges when applying the trained models to real robot scenarios with new data. Several factors contributed to this discrepancy. Firstly, the combination of gait parameters led to diverse force curves even within the same floor type. This complexity, compounded by the limited representation of the curves due to insufficient data, made accurate classification difficult. Although this particular research did not yield conclusive results for our specific problem, it paved the way for Chapter 4.

In Chapter 4, we adopted a more suitable network architecture, specifically a LSTM network, which is well-suited for analysing temporal data, such as the evolving force patterns during walking. By augmenting the dataset with a larger volume of data and utilizing a fixed gait, we achieved improved classification results in real-time scenarios. The utilization of the LSTM network, coupled with the increased dataset and controlled gait, proved instrumental in addressing the challenges faced in this section and obtaining more reliable classification outcomes. The real robot was used to conduct experiments on labelled floors with decreasing coefficients of friction, which were measured when the robot ITShoe came into contact with the floor.

The third part (chapter 5) is dedicated to enhancing the adaptability of a humanoid controller by incorporating specific tweaks and adjustments based on the insights gained from the previous chapters. This section presents a solution that optimizes humanoid efficiency on slippery floors. By combining the knowledge acquired from measuring the ground reaction forces and the application of learning approaches, the humanoid controller is fine-tuned to enhance its stability, agility, and overall performance in challenging and slippery environments. Notably, the implementation of an EYC with adaptable parameters based on the classification of different floor types has proven to significantly improve the defined stability criteria outlined in this chapter.

In summary, these three parts collectively contribute to the specific challenge of improving bipedal locomotion on slippery floors.

6.2 Concluding Remarks

In conclusion, this research has made substantial progress in addressing the challenge of locomotion on various surfaces, with a particular focus on slippery floors.

Firstly, the development of the ITshoes wearable force system aligns with the vision set forth in Hypothesis 1, enabling accurate measurements of ground reaction forces. These measurements have provided a robust foundation for subsequent analysis and control strategies, supported by extensive experimentation and data collection. The reliability and validity of the collected data have ensured that informed decisions can be made based on the insights derived from the ground reaction forces.

The development of the ITshoes has taken into account crucial factors such as costeffectiveness, weight, adaptability, and user-friendliness. These considerations have been instrumental in ensuring that the ITshoes have the potential to be widely accessible and seamlessly integrated into various designs of humanoid robots.

Secondly, our research validates Hypothesis 2, as we effectively harnessed machine learning techniques to distinguish between different types of slippery surfaces. Leveraging the normal and tangential ground reaction forces data obtained from the ITshoes, our classification system provides the humanoid robot with real-time knowledge of floor conditions. This achievement empowers the robot to make informed decisions and adapt its locomotion accordingly.

Lastly, our work strongly supports Hypothesis 3, as we have seamlessly integrated the classified floor types into the humanoid controller, thereby significantly enhancing the robot’s locomotion performance. The adapted controller, tailored to the specific characteristics of the NAO robot, enables the robot to navigate different slippery floors with remarkable ease.

In essence, our research has not only addressed the hypotheses outlined at the beginning but has also opened avenues for further advancements in the field of humanoid robotics, demonstrating the potential of wearable force systems and machine learning techniques in enhancing robot locomotion and adaptability.

Furthermore, our work introduces a novel approach involving four floors with varying coefficients of friction and the development of a real-time classification system to adapt the humanoid controller accordingly. Nevertheless, the overarching objective lies in extending this concept to detect transitions between different floor types. Such transitions would signify new frictional challenges and, consequently, necessitate the adaptation of controller parameters. This avenue of exploration represents a promising direction for future research, promising to further advance our understanding of bipedal locomotion on diverse surfaces.

6.3 Future Directions

To enhance the accuracy, reliability, and overall performance of ground reaction force measurements with the ITshoes, as well as the subsequent data information extraction and humanoid controller improvements, it is recommended to focus on the following future directions:

Design a secure attachment mechanism: Develop a mechanism that maintains consistent pressure between the robot’s feet and the ITshoes. This ensures reliable and precise data collection. Incorporate adjustable straps or fasteners to create a firm connection, minimizing slippage or movement during locomotion. Use materials with suitable frictional properties to enhance grip and stability.

Expand the dataset: Acquire more data to improve the accuracy and robustness of the floor classification system. Increase the diversity and quantity of data to enhance machine learning algorithms’ effectiveness and generalization capabilities. Include a range of floors with varying frictional properties within the classified ones to better predict unfamiliar surfaces.

Explore challenging floor conditions: Actively seek out and analyse new and unclassified floor conditions to enhance the system’s capabilities. Gather data from diverse and unexplored floors to improve the classification model iteratively. This helps the system adapt to a wider range of floor conditions.

Implement the developed controller in the real robot: Integrate the controller with the ITshoes’ ground reaction force measurements, replacing simulated sensors with realtime data. Validate the integration through experimentation and testing in real-world scenarios. Evaluate the system’s performance and adaptability to different types of floors and environmental conditions.

By implementing these steps, the accuracy, reliability, and adaptability of the ITshoes and the floor classification system can be significantly enhanced, leading to an improved controller for humanoid locomotion on slippery floors. This advancement will contribute to the development of more capable and agile humanoid robots that can navigate challenging slippery environments with increased stability and efficiency.

6.4 Contributions

In this section, the contributions made in this PhD thesis will be outlined, with each corresponding to a specific chapter (which comprises the actual scientific papers):

• Instrumented system for Humanoid Gait Analysis

The first contribution of this thesis is the development of an innovative instrumented system, as presented and explored in Chapter 2 (scientific article almeida2018novel ), which serves as the foundation for precise measurement and analysis of Ground Reaction Forces on various surfaces, particularly slippery ones.

• Computational Intelligence Techniques for Floor Classification

The second contribution is detailed in Chapters 3 and 4 (scientific articles almeida2020learning ; 9598809 ), where computational intelligence techniques are introduced to interpret force data and classify different types of slippery floors.

• Humanoid Adaptive Controller

The third contribution encompasses the integration of a humanoid controller and additional adaptations and improvements, as detailed in Chapter 5 (scientific article submitted). These enhancements significantly enhance control and responsiveness, especially on challenging surfaces.

Additionally, there is a separate contribution made to research focused on automating the NAO Robot’s gait. In this context, a controller was presented that adjusts the robot’s gait based on the analysis of human walking and its adaptation to slippery surfaces (scientific article gui).

Finally, the last contribution of this thesis centers on the adaptation of humanoid gait on slippery surfaces, marking the culmination of all previous efforts, as elaborated in Chapter 5 (submitted scientific article). This achievement exemplifies the synergistic effect resulting from the integration of the instrumented system, computational intelligence techniques, and the humanoid controller.