\chapter{Introduction}\label{chapter:intro}
\thispagestyle{myheadings}
% -------------------------------------------------------------
% Framework
% -------------------------------------------------------------
\section{Framework}\label{sec:intro_framework}
The work presented in this dissertation  has been developed within the framework of a new concept for robot learning by demonstration, combining teleoperation and kinesthethic teaching around an haptic interface using force feedback.

Robot learning by demonstration is a powerful approach in order to automate the tedious manual programming of robots, to learn locomotion patterns without complex dynamical models and to reduce the complexity of high dimensional search spaces \cite{a._billard_robot_2008, argall_survey_2009}. The demonstrations are typically provided by teleoperating the robot, or by vision and motion sensors recordings of the user doing the task. Recent progresses aim to provide more user-friendly interfaces, such as kinesthetic teaching \cite{s._calinon_learning_2007, p._kormushev_imitation_2011, d._lee_incremental_2011}.

The main focus of this work is centered on the University of Aveiro Humanoid Project (PHUA), that currently consists of a 27 degree-of-freedom (DOF) whole body small-sized humanoid platform based on an hybrid actuation system that combines servomotor powered joints with elastic elements, providing an energy storage/recover mechanism \cite{a._cardoso_desenvolvimento_2006, santos_development_2010}.

The robot's anthropometric proportions and ranges served as inspiration towards a platform that permits walking with straight support legs by incorporating a compliant foot with a passive toe joint, bringing the robot's foot kinematics closer to the humans.

The control of full-body humanoid robots is an extremely complex problem, especially for tasks such as locomotion. This complexity arises from the many degrees-of-freedom involved, the lack of precise models, the non-closed form for robot control, the dependency on the environment conditions, the compliance of actuators, the variable stiffness of links, the backlash of transmissions or the noise in internal sensors, along with many others.

The same problems have been faced in the PHUA project. Although conceived with care and with many
components previously simulated before effective construction, the platform suffers from many of the
limitations mentioned above. Tasks like walking or stepping/squatting, or even manipulation or grasping, so natural in humans, become very difficult in these robots with all their mechanical and controlling limitations.

To tackle the pre-programmed control issues, the project took a new direction towards robot learning, but with a new approach: tele-kinesthetic teaching. Basically, it is teleoperation with an haptic interaction with the robot's environment and information from its state.

Haptic interaction can be defined as manipulation using the human sense of touch. The term ''haptics'' arises from the Greek root \textit{haptikos}, meaning ''able to grasp or perceive''. This kind of interaction with computers implies the ability to use our natural sense of touch to feel and manipulate computed quantities \cite{hollis_haptics_2004}.

The haptic interaction presented in this work is within the scope of an area known as computer haptics, ''algorithms and software associated with generating and rendering the touch and feel of virtual objects'' (analogous to computer graphics), but also slightly crossing into machine haptics, defined as ''the design, construction, and use of machines to replace or augment human touch'' \cite{srinivasan_what_2012}. This approach uses the field of computer haptics to feedback the operators of the real world conditions through the use of machine haptics, which, in this case is not just an haptic interface device, but also the robot itself.

The field of haptic interaction and perception is a new way of human-machine interface, and one that may revolutionize the way we interact with machines, giving them a new ''feel''. The human haptic system, with its tactile, kinesthetic, and motor capabilities together with the associated cognitive processes, presents a uniquely bidirectional information channel to our brains that remains somewhat unexplored.

Given the increasing quantities and types of information that surrounds us, there is a critical need to explore new ways to interact with this ever-flowing new information. In order to be efficient in this interaction, it is essential that we utilize all of our sensorimotor capabilities.

In this work, an approach is suggested where the user provides demonstrations by physically interacting with the humanoid robot through an haptic interface \cite{cruz_tele-kinesthetic_2012}. The proposed methodology enables a natural interface for tele-kinesthetic teaching and sensing, in which the user provides functional guidance and corrections, while being aware about (i.e., able to “feel”) the dynamics of the system, its physical capabilities and/or constraints.

The combination of haptic and visual (visuohaptic) information has been confirmed as a good memorizing mechanism for humans \cite{morris_haptic_2007}, which gives confidence that such an approach can perform well in demonstrations even with inexperienced operators.

Additionally, during the demonstration phase, the sensory information and the commands guiding the execution of a specific task can be recorded. All the data logged from the human-robot interaction can be later used for learning purposes.

This approach goes beyond previous research on teaching by demonstration that is unable to raise the level of bidirectional human-robot interaction. Instead, it refers to a deeper relationship between the user and the robot who share control to reach common goals using the same measures of outcome.

% -------------------------------------------------------------
% objectives
% -------------------------------------------------------------
\section{Objectives}\label{sec:intro_objectives}
The objective of this work is to present new programming and control concepts for humanoid robots by means of an haptic interface, specifically the development of a force feedback haptic control and sensing mechanism, aimed at the robot's perception/actuation data logging. 

It is an objective to document current robot learning methodologies involving interactive kinesthetic learning and teleoperation. As this work is centered around humanoid platforms, it is also critical to document their current state and progresses.

As it was unsure if the platform could perform complex tasks such as walking, stepping or squatting, a dynamic model is proposed as an easy-to-deploy solution, using a toolbox for a commercial software, \textit{MATLAB}.

It is also proposed the development of a user-friendly interface for robot control and an haptic interface, providing the means for expert data logging when used by inexperienced operators. This requires deep knowledge of the haptic device's programming library and the definition of a control structure for the platform. To ensure the operator perceives the current state of the robot, it is required to develop ways of translating that information into force feedback, through the haptic device. The collected data can then be used to obtain a model of the task, using data as the position and force information, among others.

Lastly, it was a primary objective to develop haptic tele-kinesthetic demonstrations that could show the potential of this work's approach.
% -------------------------------------------------------------
% state of the art
% -------------------------------------------------------------
\section{State of the Art}\label{sec:intro_state_of_the_art}

% +++++++++++++ HAPTIC INTERFACES +++++++++++++
\subsection{Principles of Haptic Interaction}\label{sec:sota_haptic_principles}
In the early 20th century, psychophysicists introduced the word ''haptics'' to label the subfield of their studies that addressed human touch-based perception and manipulation. In the 1970s and 1980s, significant research efforts in a completely different field, robotics, also began to focus on manipulation, perception and interaction by touch. 

The idea of using touch as a means of communication grew in the late 1980s through the late 1990s \cite{sherrick_touch_1985, craig_somesthesis_1999}. Since then it became popular and, in the late years, the number of published papers with the word ''haptic'' in them reached thousands a year.

The word ''haptics'' refers to the capability to sense a natural or synthetic mechanical environment through touch. Haptics also includes kinesthesia (or proprioception), the ability to perceive one's body position, movement and weight. It has become common to speak of the ''haptic channel'' to collectively designate the sensory and motor components of haptics. This is because certain anatomical parts (in particular the hand) are organs in which perceiving the world and acting upon it are activities that take place together. Tactile and kinesthetic channels work together to provide humans with means to perceive and act on their environment.

With the exception of input devices, such as the mouse, computer interaction relies on skills similar to those needed for using typewriters. Haptic interfaces, with a principle as exemplified in Figure \ref{fig:sota_haptic_interface}, may be viewed as an approach to address this limitation. It is thus possible to classify haptics into the area of human-computer interfaces. Unlike traditional interfaces that provide visual and auditory information, haptic interfaces generate mechanical signals that stimulate human kinesthetic and touch channels \cite{hayward_haptic_2004}.

%\begin{figure}[H]
%\centering
%\includegraphics[width=\textwidth]{Imagens/haptic_interface2}
%\caption[The principle of haptic interaction.]{The principle of haptic interaction \cite{fulkar_applications_2010}.}
%\label{fig:sota_haptic_interface}
%\end{figure}
%%% ******** TIKZ DEFINITIONS ******** %%%
% Define styles for the objects
\tikzstyle{block} = [draw, rectangle, very thick, minimum height=3em, minimum width=6em, text centered,text width=2cm]
\tikzstyle{pinstyle} = [pin edge={to-,thick,black}]
\tikzstyle{branch} = [circle,inner sep=0pt,minimum size=1mm,fill=black,draw=black]
\tikzstyle{aux} = [coordinate, thick]
\tikzstyle{labelalt} = [draw, coordinate, text centered,text width=2cm]
% We need layers to draw the block diagram
\pgfdeclarelayer{background}
\pgfdeclarelayer{foreground}
\pgfsetlayers{background,main,foreground}
%%% ******************************** %%%
\begin{figure}[H]
\centering
% The block diagram code is probably more verbose than necessary
\begin{tikzpicture}[auto, node distance=2.8cm,>=latex']
    \node [block, name=intertool] {Interaction Tool};
	\node [block,right of=intertool] (sensori) {Sensorimotor System};  
    \node [block,right of=sensori] (brain) {Brain};
    \node [block,left of=intertool] (hapticinter) {Haptic Interface};  
    \node [block,left of=hapticinter] (computer) {Computer};    
    \node [aux,above of=brain, node distance=1cm] (overbrain) {};
    \node [aux,below of=brain, node distance=1cm] (underbrain) {}; 
    \node [aux,above of=computer, node distance=1cm] (overcomp) {}; 
    \node [aux,below of=computer, node distance=1cm] (undercomp) {}; 
    \node [below of=intertool, node distance=1.3cm] (coupled) {\textsl{\small{Mechanically Coupled}}}; 
    \node [aux, right of=coupled, above of=coupled, node distance=0.5cm, xshift=0.8cm,yshift=-0.2cm] (auxone) {};
	\node [aux, left of=coupled, above of=coupled, node distance=0.5cm, xshift=-0.8cm,yshift=-0.2cm] (auxtwo) {};    
    
    \draw [draw,-,very thick] (sensori) |-  node [near end, above] {\small{\textit{Perception}}} (overbrain);
    \draw [->, very thick] (overbrain) --  (brain);
    \draw [->,very thick] (underbrain) -|  node [near start, below] {\small{\textit{Action}}} (sensori);
    \draw [-,very thick] (brain) --  (underbrain);
	\draw [-,very thick] (hapticinter) |-  node [near end, above] {\small{\textit{Sensing}}} (overcomp);
    \draw [->,very thick] (overcomp) --  (computer);
    \draw [->,very thick] (undercomp) -|  node [near start, below] {\small{\textit{Actuation}}} (hapticinter);
    \draw [-,very thick] (computer) --  (undercomp);    
    \draw [-, dashed, very thick] (intertool) -- node [name=rightdash,above] {} (sensori); 
	\draw [-, dashed, very thick] (intertool) -- node [name=leftdash,above] {} (hapticinter);     
	\draw [->, dashed, very thick] (auxone) -- (rightdash);
	\draw [->,dashed, very thick] (auxtwo) -- (leftdash);
   
	\path (brain.east |- overbrain.north)+(-1,+1) node (human) {\textbf{\small Human User}};   
	\path (overcomp.north -| computer.west)+(+2.2,+1) node (haptic) {\textbf{\small Haptic Rendering System}};      
    
	\begin{pgfonlayer}{background}
        \path (brain.east |- overbrain.north)+(+0.3,+0.8) node (a) {};
        \path (undercomp.south -| intertool.south)+(+0,-0.8) node (b) {};
        \path[rounded corners=15pt, draw=black!60, dashed, thick]
            (a) rectangle (b);
        \path (intertool.south |- overcomp.north)+(0,+0.8) node (c) {};
        \path (undercomp.south -| computer.west)+(-0.3,-0.8) node (d) {};
        \path[rounded corners=15pt, draw=black!60, dashed, thick]
            (c) rectangle (d);
    \end{pgfonlayer}
\end{tikzpicture}
\caption[The principle of haptic interaction.]{The principle of haptic interaction (adapted \cite{fulkar_applications_2010}).}
\label{fig:sota_haptic_interface}
\end{figure}

These signals are synthesized through computer algorithms, in a process know as ''\textit{haptic rendering}'', schematically represented on Figure \ref{fig:sota_haptic_rendering}. Haptic rendering algorithms compute the correct interaction forces between the haptic interface representation inside the virtual (or virtualized) environment and the objects populating the environment. Moreover, haptic rendering algorithms ensure that the haptic device correctly renders such forces on the human operator. Normally, haptic rendering is accompanied by simultaneous graphical rendering in what is more properly referred to as a visual/haptic interface. Unlike graphical rendering which can satisfy the eye at update rates of 30 frames per second or even less, haptic rendering must be done at rates around a kilohertz, the human kinesthetic sensing frequency minimum threshold, in order for the rendering to be seamless and natural \cite{salisbury_haptic_2004}.

\begin{figure}[H]
\centering
% \includegraphics[width=\textwidth]{Imagens/haptic_rendering}
\caption[An example of haptic rendering architecture.]{An example of haptic rendering architecture, split into three main blocks. Collision-detection algorithms provide information about contacts $S$ occurring at position $X$ and the objects in the virtual environment. Force-response algorithms return the ideal interaction force $ F_{d}$ while control algorithms return a force $F_r$ to the user \cite{salisbury_haptic_2004}.}
\label{fig:sota_haptic_rendering}
\end{figure}
\vspace*{5mm}
Nowadays, haptics can be found in a great variety of applications, such as video games, military war games, vehicle simulators, touchscreens, arts and creation, computer aided design/engineering/manufacturing (CAD/CAE/CAM), assisted surgery machines and, of course, robotics (telerobotics) \cite{fulkar_applications_2010}.

In video games or touchscreens, haptic sensations are created by actuators, or motors, which create a vibration. Those vibrations are managed and controlled by embedded software, and integrated into device user interfaces and applications. The rumble effect in video game controllers or the touch vibration on smartphone dial pads are both examples of haptic effects \cite{immersion_corporation_haptics_2012}.

For control or virtual reality applications, haptics goes beyond simple rumble or vibration signals. These signals are no other than energy, so for applications that require a deeper human-machine connection, force, torque, vibration and visual signals are combined. In this sense, we can say haptic interface devices behave like small robots that exchange mechanical energy with the user. The term \textit{device-body interface} is frequent to highlight the physical connection between operator and device through which energy is exchanged. Although these interfaces can be in contact with any part of the operator's body, hand interfaces have been the most widely used and developed systems to date \cite{salisbury_haptic_2004}.
\clearpage
\subsection{Haptic Interface Devices}\label{sec:sota_haptic_devices}

Nowadays, a number of different haptic interface commercial and academic solutions coexist, for numerous applications. However, within the devices with relevance for the context of this work, a distinction between haptic interface devices must be introduced: their intrinsic mechanical behaviour.

On one hand, impedance based haptic devices simulate mechanical impedance, i.e., they read position and output force. Simpler to design and much cheaper to produce, these impedance-type architectures are the most common within both academic and commercial worlds.

On the other hand, admittance haptic devices simulate mechanical admittance, reading force and outputting position. Admittance-based devices, such as the 3DOF Haptic Master \cite{van_der_linde_hapticmaster_2012}, represented i Figure \ref{fig:sota_haptics_hapticmaster_freedom6s}, are generally used for applications requiring high forces in a large workspace. This device has a maximum exertable force of 250 N with a sensitivity of 0.01 N.

Additionally, haptic devices can be broadly classified into: exoskeletons and stationary devices; gloves and wearable devices; point-sources and specific task devices. Haptic exoskeletons/gloves are mainly admittance-based devices, not very frequent in telerobotics experiments. So from this point on, the term ''haptic device'' refers to point-source devices, for their preponderance in teleoperation and robotics.

\begin{figure}[H]
\centering
% \includegraphics[width=0.85\textwidth]{Imagens/hapticmaster_freedom6s}
\caption[The FCS HapticMaster and the MTB Freedom 6S devices.]{The FCS Control Systems admittance-based HapticMaster \cite{van_der_linde_hapticmaster_2012} (left) and the MTB Technologies impedance-based Freedom-6S \cite{hayward_haptic_2004} (right) devices.}
\label{fig:sota_haptics_hapticmaster_freedom6s}
\end{figure}\vfill

Still in the 1990s, impedance-based commercial solutions began to emerge, some with intricate mechanics, such as the MTB Technologies Freedom-6S, in Figure \ref{fig:sota_haptics_hapticmaster_freedom6s}, a 6DOF device (extensible to 7 \textit{via} accessories) built with hybrid kinematics. It is characterized by a wide dynamic range and six-axis static and dynamic balancing. Its design was ''wrist partitioned'', i.e., the position and orientation stages each being parallel mechanisms. The position stage is directly driven and the orientation stage is driven through remotizing tendons with a differential tensioning technique that operates with low tension and hence low friction.

Today, commercial solutions for haptic interface devices exist in many different physical dimensions and specifications. One of the most popular is the desktop device, due to its compact form factor and portability. These solutions provide a force feedback system with affordable price, being very popular in the academia. The most common devices are the SensAble PHANToM OMNI and Desktop models, shown in Figure \ref{fig:sota_haptics_phantom_omni_desktop}.

These are sometimes called ''pen-grasping'' joysticks, in the sense that the operator interacts with the device \textit{via} a pen-like end-effector. With a workspace of around 27 dm\textsuperscript{3} and force feedback magnitudes of 3.3 N and 7.4 N, respectively \cite{sensable_technologies_specifications_2009}, these devices are well known for their capstan drive, which allows for their construction. They are mostly found in 3D CAD and 3D moulding applications, for the pen-like interface emulates the tool usage mechanisms, making the learning curve steeper.

\begin{figure}[H]
\centering
\includegraphics[width=72mm]{Imagens/phantom_omni}
\includegraphics[width=58.5mm]{Imagens/phantom_desktop}
\caption[The SensAble PHANToM OMNI and Desktop devices.]{The SensAble PHANToM OMNI (left) and PHANToM Desktop (right) haptic impedance-based desktop devices \cite{_phantom_2008,sensable_technologies_specifications_2009}.}
\label{fig:sota_haptics_phantom_omni_desktop}
\end{figure}

Within the desktop solutions, there is a new subdivision based on a parallel configuration (Delta). These devices became popular for allowing 6DOF force feedback/position reading in a very compact mechanism, but sacrificing workspace dimensions. The Virtuose 6D Desktop from Haption is a 6DOF spherical joystick with 6DOF force feedback, within a volume of 120 mm in diameter and 35° in three directions at the center, with a resolution in position at about 2 mm. Its maximum exertable force is 10 N and its maximum exertable torque is 0.5 Nm \cite{haption_virtuose_2012}. Another example are the Force Dimension products, also based on a parallel kinematics mechanism. The Omega \cite{force_dimension_omega_2012} and Delta \cite{force_dimension_delta_2012} offer greater force feedback, but with a slightly smaller workspace. These devices are represented in Figure \ref{fig:sota_haptics_virtuose_desktop_omega6_delta6}.

\begin{figure}[H]
\centering
\includegraphics[width=0.3\textwidth]{Imagens/virtuose_desktop}
\includegraphics[width=0.3\textwidth]{Imagens/omega6}
\includegraphics[width=0.3\textwidth]{Imagens/delta6}
\caption[The Haption Virtuose 6D Desktop and the Force Dimension Omega and Delta.]{The Haption Virtuose 6D Desktop (left) , the Force Dimension Omega (middle) and Delta (right) haptic impedance-based desktop devices \cite{haption_virtuose_2012, force_dimension_omega_2012, force_dimension_delta_2012}.}
\label{fig:sota_haptics_virtuose_desktop_omega6_delta6}
\end{figure}

When bigger workspaces are required, these devices are not suitable. The Sensable PHANToM Premium line is the company's ultimate line of products. The PHANToM Premium 1.0, 1.5 and 3.0, in Figure \ref{fig:sota_haptics_phantom_premiums}, are shoulder pivoting devices with larger force feedback levels when compared to their desktop variants, but with the exact same mechanism type, allowing for 3DOF force feedback and a 6DOF position reading. The PHANToM Premium 3.0 and the 1.5 model both have a ''HIGH FORCE'' variant that is capable of rendering 6DOF forces/torques.\\

\begin{figure}[H]
        \centering
        \begin{subfigure}[b]{0.35\textwidth}
                \centering
                \includegraphics[width=\textwidth]{Imagens/phantom_premium_1}
                \caption{}
                \label{fig:premium_1}
        \end{subfigure}%
        ~\qquad\qquad %add desired spacing between images, e. g. ~, \quad, \qquad etc. 
          %(or a blank line to force the subfigure onto a new line)
        \begin{subfigure}[b]{0.35\textwidth}
                \centering
                \includegraphics[width=\textwidth]{Imagens/phantom_premium_1_5}
                \caption{}
                \label{fig:premium_1_5}
        \end{subfigure}
        ~\\[2mm] %add desired spacing between images, e. g. ~, \quad, \qquad etc. 
          %(or a blank line to force the subfigure onto a new line)
        \begin{subfigure}[b]{0.4\textwidth}
                \centering
                \includegraphics[width=\textwidth]{Imagens/phantom_premium_3}
                \caption{}
                \label{fig:premium_3}
        \end{subfigure}
        \caption[The SensAble PHANToM Premium 1.0, 1.5 and 3.0 devices.]{The SensAble PHANToM Premium 1.0, 1.5 and 3.0 haptic impedance-based desktop devices \cite{sensable_technologies_specifications_2009-1}.}\label{fig:sota_haptics_phantom_premiums}
\end{figure}

Again from Haption, we can find two models with large workspaces. The Virtuose 6D35-45 offers force-feedback on all 6 degrees-of-freedom together with a large workspace (450 mm). Its maximum exertable force is 35 N and its maximum exertable torque is 3 Nm. Modular in design, it can be purchased as a 3DOF device and later upgraded to 6DOF \cite{haption_virtuose_2012-1}. The Virtuose 6D40-40 is a new generation master arm for teleoperation with force-feedback. It is very similar to the 6D35-45 model in workspace limitations (400 mm workspace), but allows for a much higher force/torque feedback. Its maximum exertable force is 100 N and its maximum exertable torque is 10 Nm \cite{haption_virtuose_2012-2}. Both models are shown in Figure \ref{fig:sota_haptics_virtuose_6d35_6d40}.

\begin{figure}[H]
\centering
\includegraphics[width=0.45\textwidth]{Imagens/virtuose_6d35}\qquad
\includegraphics[width=0.35\textwidth]{Imagens/virtuose_6d40}
\caption[The Haption Virtuose 6D35 and 6D40 devices.]{The Haption Virtuose 6D35 (left) and 6D40 (right) haptic devices \cite{haption_virtuose_2012-1, haption_virtuose_2012-2}.}
\label{fig:sota_haptics_virtuose_6d35_6d40}
\end{figure}

New kinds of haptic devices have appeared in the later years. One of the most promising is the dual-pantograph arrangement device. These devices are 6DOF joysticks with full 6DOF force/torque feedback designed specifically to control industrial serial robots equipped with force sensors. The device is made combining two pantographs (Delta type actuators) driven by motors at its shoulders and another at the waist. The wand type end-effector is connected to both end points of each pantograph through universal joints (U-joints). The yaw about the wand axis gives its extra DOF. However, this kind of joysticks still possess very limited workspaces, so their applicability is still somewhat limited. Figure \ref{fig:sota_haptics_pantographs} depicts two examples of this technology.

\begin{figure}[H]
\centering
\includegraphics[width=0.3\textwidth]{Imagens/new_6dof_haptic_device}\hspace{1cm}
\includegraphics[width=0.35\textwidth]{Imagens/haptic-wand}
\caption[A double pantograph device proposal and the Quanser Haptic Wand device.]{A double pantograph style haptic device proposal (left) \cite{vu_new_2011} and the Quanser Haptic Wand (right) commercial haptic device \cite{quanser_5-dof_2012}.}
\label{fig:sota_haptics_pantographs}
\end{figure}

Another area quickly growing is the dual-device operation, with a great focus in the medical industry. The Force Dimension Sigma, in Figure \ref{fig:sota_haptics_sigma7}, is a dual joystick device that covers the natural range of motion of the human hand and it is compatible with bi-manual teleoperation console design. Along with a full gravity compensation and driftless calibration, it possesses a good level of forces, torques and a high precision on all its 7DOF \cite{force_dimension_sigma_2012}, making it one the most accomplished master devices available.

\begin{figure}[H]
\centering
\includegraphics[width=0.7\textwidth]{Imagens/sigma7}
\caption[The Force Dimension Sigma haptic device.]{The Force Dimension Sigma haptic device in a medical application \cite{force_dimension_sigma_2012}.}
\label{fig:sota_haptics_sigma7}
\end{figure}

Virtual environment (VE) manipulation/navigation is another field were dual configurations have great impact. The DLR (German Aerospace Center) bimanual haptic device, represented in Figure \ref{fig:sota_haptics_dlr}, is a good example of the development of this field \cite{hulin_dlr_2011}. The device is composed of two 6DOF DLR/KUKA Light Weight Robot (LWR) arms and an extra DOF at the handles. The LWRs have similar dimensions to the human arms and can be operated in torque and position control mode at an update rate of 1 kHz, for a full haptic rendering support. The two robot arms are mounted behind the user, so that the intersecting workspace of the robot and human arms becomes maximized. This system has a 150 N maximum exertable force generation. The haptic system is combined with a motion detect helmet and a haptic glove system for maximum VE immersion. Even though it is primary an impedance-based device, it is also prepared for admittance-based control and a hybrid mode, making it one the few functional devices with this capability. It is considered to be one of the most complex and complete VE haptic devices.

\begin{figure}[H]
\centering
\includegraphics[width=0.7\textwidth]{Imagens/dlr}
\caption[The DLR bimanual haptic device.]{The DLR bimanual haptic device \cite{hulin_dlr_2011}.}
\label{fig:sota_haptics_dlr}
\end{figure}\clearpage

% +++++++++++++ END OF HAPTIC INTERFACES +++++++++++++

% +++++++++++++ TELEOP +++++++++++++
\subsection{Teleoperation, Kinesthetic Teaching and Learning from Demonstration}\label{sec:sota_teleop_kines}
The teleoperation of robots is currently a vast, yet still growing, scientific field, with countless publications on the matter. It is a concept that has grown to cover all fields of robotics and into the commercial/industrial worlds. Robot learning is another topic of great interest to the scientific community. New algorithms and techniques are proposed each year, with considerable improvements in every iteration. The use of haptic interfaces for human-machine learning processes study is also well accepted, as the field of haptic guidance is growing \cite{feygin_haptic_2002}.

Teleoperation of robots benefits from the extension of their sensory capabilities with the use of haptic interfaces. Usually, telerobotic applications provide the remote user images and navigation sensor data. With additional force feedback, the remote control performance is improved. Small mobile robots have been successfully manipulated using an haptic interface, as shown on Figure \ref{fig:sota_teleop_mobile_robot}, with the user being aware of the state changes in the robot's environment. Sensors in the car bumpers provide force feedback to the user \cite{rosch_haptic_2002}.

\begin{figure}[H]
\centering
\includegraphics[width=0.58\textwidth]{Imagens/mobile_robot}
\caption[Haptic teleoperation of a mobile robot.]{Haptic teleoperation of a mobile robot \cite{rosch_haptic_2002}.}
\label{fig:sota_teleop_mobile_robot}
\end{figure}

In robotics, learning from demonstration is also becoming common place. Rather than imitating human actions, the focus is to develop an intuitive and interactive method based on user demonstrations to create human-like, autonomous behaviour for a robot. New paradigms have emerged with the use of haptic systems and robot learning. Skill transfer mechanisms have gained a growing interest in the scientific community. For instance, one experiment constructs a reality-based haptic interaction system for knowledge transfer, by linking a human expert skill with robotic movement in real time, in an handwriting task \cite{park_transfer_2010}. The goal was to successfully recreate manipulation motion with a robotic system and transfer haptic forces to an untrained user with an haptic device.

In another experiment, illustrated in Figure \ref{fig:sota_teleop_colab}, a human teacher uses an haptic control strategy to teach a humanoid robot collaborative tasks with a human partner \cite{calinon_learning_2009}. The robot can learn through observation to perform a collaborative manipulation task which is first demonstrated by a user controlling the robot’s hand via an haptic interface. Haptic communication reflects more than pure dynamic information on the task. It also includes communication patterns which result from both users, human partner and teleoperated robot, constantly adapting their hand motion to coordinate in time and space their respective motions.

\begin{figure}[H]
\centering
\includegraphics[width=0.55\textwidth]{Imagens/colab}
\caption[Teleoperating a humanoid robot demonstrating collaborate tasks with human users.]{Teleoperating a humanoid robot to demonstrate and teach an object lifting task collaboratively with a human user \cite{calinon_learning_2009}.}
\label{fig:sota_teleop_colab}
\end{figure}

Studies have shown that the use of force profiles along with position information improves the quality of the robot learning. In one experiment, using a Barret Technology \textit{WAM} manipulator, kinesthetic teaching was used for position recording and an haptic interface was used to trace the force profile of the same tasks, such as ironing a piece of cloth or opening a door, as described in Figure \ref{fig:sota_teleop_comparison}. Feeding the force profile to the learning algorithm improved the robot's response when autonomously performing those tasks \cite{p._kormushev_imitation_2011}.

\begin{figure}[H]
\centering
\includegraphics[width=0.77\textwidth]{Imagens/compare1}
\includegraphics[width=0.77\textwidth]{Imagens/compare2}
\caption[Retrieving position and force profiles from different tasks.]{Retrieving position and force profiles from different tasks, such as ironing a piece of cloth or opening a door, for teaching purposes \cite{p._kormushev_imitation_2011}.}
\label{fig:sota_teleop_comparison}
\end{figure}

Teaching by kinesthetic manipulation with humanoid robots is also a topic of interest in the academia, with several new approaches successfully implemented at each passing year. One experiment placed a humanoid robot writing on a chalk board while standing up \cite{kormushev_upper-body_2011}. The humanoid robot upper body training for imitation learning is done via kinesthetic teaching, while the Reaction Null Space method is used for keeping the balance of the robot. During the user demonstration phase, as shown in Figure \ref{fig:sota_humanoid_kin_teach}(a) and (b), along with the kinematic infromation logging, a force/torque sensor is used to record the exerted forces. During the reproduction, a hybrid position/force controller is used to apply the learned trajectories (position and force-wise) to the end-effector, as seen on Figure\ref{fig:sota_humanoid_kin_teach}(c).

\begin{figure}[H]
        \centering
        \begin{subfigure}[b]{0.32\textwidth}
                \centering
                \includegraphics[height=44mm]{Imagens/humanoid_kine_1}
                \caption{}
                \label{fig:hkine_1}
        \end{subfigure}%
        \quad%add desired spacing between images, e. g. ~, \quad, \qquad etc. 
          %(or a blank line to force the subfigure onto a new line)
        \begin{subfigure}[b]{0.32\textwidth}
                \centering
                \includegraphics[height=44mm]{Imagens/humanoid_kine_2}
                \caption{}
                \label{fig:hkine_2}
        \end{subfigure}%
        ~%add desired spacing between images, e. g. ~, \quad, \qquad etc. 
          %(or a blank line to force the subfigure onto a new line)
        \begin{subfigure}[b]{0.32\textwidth}
                \centering
                \includegraphics[height=44mm]{Imagens/humanoid_kine_3}
                \caption{}
                \label{fig:hkine_3}
        \end{subfigure}
        \caption[Upper-body humanoid kinesthetic teaching.]{Upper-body humanoid kinesthetic teaching while standing \cite{kormushev_upper-body_2011}.}\label{fig:sota_humanoid_kin_teach}
\end{figure}

Learning from demonstration has also been successfully achieved using virtual environments. For instance, a serial robot arm was taught to catch a ball in mid-air by analysing data collected in a virtual world environment, as shown in Figure \ref{fig:sota_teleop_VE}, compiled from user inputs \cite{hamon_virtual_2011}.

\begin{figure}[H]
\centering
\includegraphics[width=0.6\textwidth]{Imagens/virtual_learning}
\caption[Virtual environment used for learning by demonstration.]{Virtual environment used for learning by demonstration with a serial arm catching a ball in mid-air \cite{hamon_virtual_2011}.}
\label{fig:sota_teleop_VE}
\end{figure}

This dissertation implements a keyword not often used: tele-kinesthetic teaching. This method refers to teleoperation/telerobotics with three-dimensional force feedback using an haptic interface. The forces reproduced are those of the controlled object or equivalent, and are not an input for the teleoperation. This is an important feature around this keyword. However, in literature, the most relevant work found around this keyword approximates the user's hand motion with force by synthesizing EMG signals that are transformed into 3D motion and then applied to a hanging ball within a cubic frame \cite{koike_tele-kinesthetic_2006}, as shown in Figure \ref{fig:sota_teleop_telekin}. As the usage of this keyword in this experiment seems to be related to force transfer from human to machine, one must reinforce that the approach suggested in this work implements the opposite strategy.

\begin{figure}[H]
\centering
\includegraphics[width=0.75\textwidth]{Imagens/tele_kin}
\caption[Another tele-kinesthetic interaction environment.]{Another tele-kinesthetic interaction environment based on EMG signals \cite{koike_tele-kinesthetic_2006}.}
\label{fig:sota_teleop_telekin}
\end{figure}

Lastly, a new paradigm in kinesthetic teaching must be presented: industrial kinesthetic teaching. Presented in 2012, the Rethink Robotics Baxter, in Figure \ref{fig:sota_baxter}, is a dual-arm upper-body humanoid-like robot \cite{guizzo_rise_2012}. Composed of two 7DOF serial elastic actuated arms, an array of cameras and sensors, a quad-core computer running Linux, an LCD touchscreen for human-machine interface and interchangeable grippers, it is the first of its kind. This robot was built to be a low cost solution (\$22.000 US) for industrial robotics, but its strength is not on high-precision manipulation. The Baxter possesses two buttons and a knob on each forearm used to navigate the robot user-interface and also to teach kinesthetic frames to the robot, demonstrating how to perform the tasks. This solution does not require a specialized human operator to perform the teaching and the LCD touchscreen provides a more user-friendly interface than the usual code input interfaces.

\begin{figure}[H]
\centering
\includegraphics[width=0.65\textwidth]{Imagens/baxter}
\caption[The Rethink Robotics Baxter robot.]{The Rethink Robotics Baxter industrial kinesthetic teaching robot.}
\label{fig:sota_baxter}
\end{figure}
% +++++++++++++ END OF TELEOP +++++++++++++
% +++++++++++++ PHUA +++++++++++++
\subsection[University of Aveiro Humanoid Project]{University of Aveiro Humanoid Project (PHUA)}\label{subsec:sota_phua}
Started in 2003/2004, this project was the outcome of a joint venture between the Mechanical Engineering Department (DEM) and the Electronics, Telecommunications and Informatics Department (DETI) of the University of Aveiro. Dedicated to the development and integration of hardware and software components, the project made a major step forward with the design and development, in 2009, of a new small-size humanoid platform based on hybrid actuation, as depicted in Figure \ref{fig:sota_phua}. The focus has been on the mechanical design and the selection of actuators and sensors \cite{santos_development_2010, godinho_desenvolvimento_2011, sabino_estrutura_2009, ribeiro_desenvolvimento_2010}. The humanoid robot's height is around 65 cm, and its weight is approximately 6 kg. It is equipped with an array of two 5000 mAh \textit{LiPo} batteries on the back.

\begin{figure}[H]
\centering
\includegraphics[height=95mm]{Imagens/phua_front}\hspace*{3mm}
\includegraphics[height=95mm]{Imagens/phua_side}\hspace*{3mm}
\includegraphics[height=95mm]{Imagens/phua_back}
\caption[Current form of the PHUA robot platform.]{Current form of the PHUA robot platform, with the hybrid actuation system and the complete motor wiring installed.}
\label{fig:sota_phua}
\end{figure}

This robot is a proprietary whole-body humanoid platform with a total of 25 active and 2 passive degrees-of-freedom, as shown in Figure \ref{fig:sota_phua_dof}. The joint actuation is composed of \textit{HITEC\textsuperscript{\textregistered}} analog and digital servomotors, as distributed in the Table \ref{table:sota_phua_dofs}, combined with elastic elements. The robot uses a belt and pulley adjustable transmission system on the legs and torso, providing torque to the joints with higher requirements.

As illustrated in Figure \ref{fig:sota_phua_elastics}, the design of this platform accounts for a hybrid actuation system that combines active joints with adjustable elastic elements (rubber bands), providing a passive mechanism to store and recover energy. Mainly to enhance the system response and also to try lessen the actuator's torque demands, flexible rubber bands are attached between several pair of consecutive links in parallel with the servomotors.

The robot is also equipped with a set of 8 force sensors, 4 on each foot, and a \textit{FireFly\textsuperscript{\textregistered}} \textit{FireWire} artificial vision system in the head \cite{santos_hybrid_2011}. This platform is also equipped with an onboard \textit{PC-104} small form-factor computer.

{\centering
\begin{minipage}[b]{0.44\textwidth}
  \centering
  %figure
  \includegraphics[height=70mm]{Imagens/phua_dofs}
  \captionof{figure}{PHUA kinematic chains.}
  \label{fig:sota_phua_dof}
\end{minipage}
\begin{minipage}[b]{0.54\textwidth}
  \centering
  %table
  \begin{table}[H]
  \centering
	\captionof{table}{PHUA platform's degrees-of-freedom and installed servomotors.}
	\label{table:sota_phua_dofs}
	\begin{tabular}{c|c|c}
	\hline
	\textbf{Joint}&\begin{tabular}{c}\textbf{Number of}\\\textbf{DOFs}\end{tabular}&\textbf{Servomotor}\\
	\hline
	Toe & $1 (\times2)$ & (none)\\
	\hline
	Ankle & $2 (\times2)$ & \textit{HSR-5980SG}\\
	\hline
	Knee & $1 (\times2)$ & \textit{HSR-5980SG}\\
	\hline
	Hip & $3 (\times2)$ & \textit{HSR-5980SG}\\
	\hline
	Trunk & $3$ & \begin{tabular}{c}\textit{HSR-5980SG}$(\times2)$\\
	\textit{HSR-8498SG}$(\times1)$\end{tabular}\\
	\hline
	Shoulder & $3 (\times2)$ & \begin{tabular}{c}\textit{HSR-5980SG}\\
	\textit{HSR-5498SG}\\
	\textit{HS-82MG}\end{tabular}\\
	\hline
	Elbow & $1 (\times2)$ & \textit{HSR-5498SG}\\
	\hline
	Neck & $2$ & \begin{tabular}{c}\textit{HSR-5498SG}\\
	\textit{HS-5056MG}\end{tabular}\\
	\hline 
	Total & 27 & ---\\ \hline
	\end{tabular}
  \end{table}
\end{minipage}
}\\

The robot's anthropometric proportions and ranges allow the platform to walk with straight support legs by incorporating a compliant foot with a passive toe joint, bringing the robot's foot kinematics closer to the humans and allowing for a more natural gait.

\begin{figure}[H]
\centering
\includegraphics[width=0.44\textwidth]{Imagens/knee_elastic}
\includegraphics[width=0.54\textwidth]{Imagens/ankle_dof.jpg}
\caption{Elastic elements placed on the robot's joints and the passive toe joint.}
\label{fig:sota_phua_elastics}
\end{figure}

Table \ref{table:sota_phua_limits} details the robot's joints anthropometric ranges and the total angular course, measured from the home position (in degrees), defined as the robot with all limbs at full stretch downwards, with the torso upright and a zero tilting in the head. At the moment there are unavailable DOF's due to reasons that will be detailed further in Chapter \ref{chapter:setup}, and thus are not represented.\\[5mm]~

  \begin{table}[H]
  \centering
	\captionof{table}{PHUA robot degree-of-freedom range limitations.}
	\label{table:sota_phua_limits}
	\begin{tabular}{c|c|c|c}
	\textbf{DOF} & \textbf{Minimum} & \textbf{Maximum} & \textbf{Total Range}\\
	\hline
	\begin{tabular}{c}
	Head\\Tilt
	\end{tabular}& 0\degup & 10\degup & 10\degup\\
	\hline
	\begin{tabular}{c}
	Shoulder\\Flexion/Extension
	\end{tabular}& \minus 45\degup & 135\degup & 180\degup\\
	\hline
	\begin{tabular}{c}
	Shoulder\\Abduction/Adduction
	\end{tabular}& 0\degup & 180\degup & 180\degup\\
	\hline
	\begin{tabular}{c}
	Elbow\\Flexion/Extension
	\end{tabular}& 0\degup & 120\degup & 120\degup\\
	\hline
	\begin{tabular}{c}
	Torso\\Rotation
	\end{tabular}& \minus 90\degup & 90\degup & 180\degup\\
	\hline
	\begin{tabular}{c}
	Torso\\Flexion/Extension
	\end{tabular}& \minus 15\degup & 90\degup & 105\degup\\
	\hline
	\begin{tabular}{c}
	Torso\\Lateral Flexion/Extension
	\end{tabular}& \minus 90\degup & 90\degup & 180\degup\\
	\hline
	\begin{tabular}{c}
	Ankle\\Inversion/Eversion
	\end{tabular}& \minus 30\degup & 45\degup & 75\degup\\
	\hline
	\begin{tabular}{c}
	Ankle\\Flexion/Extension
	\end{tabular}& \minus 40\degup & 20\degup & 60\degup\\
	\hline
	\begin{tabular}{c}
	Knee\\Flexion/Extension
	\end{tabular}& 0\degup & 130\degup & 130\degup\\
	\hline
	\begin{tabular}{c}
	Hip\\Abduction/Adduction
	\end{tabular}& \minus 40\degup & 45\degup & 85\degup\\
	\hline
	\begin{tabular}{c}
	Hip\\Flexion/Extension
	\end{tabular}& \minus 30\degup & 120\degup & 150\degup\\
	\end{tabular}
  \end{table}
% end of PHUA
%\clearpage
%~\\[50mm]
%\thispagestyle{empty}