Design of walking patterns for zero-momentum point (ZMP)-based biped robots. A computational optimal control approach

Technical Report 2018 52 Pages

Engineering - Robotics


Table of Contents

1 Computational Optimal Control: Theory and Simulation
1.1 Forward dynamics-based optimization
1.2 Inverse dynamics-based optimization
1.2.1 Discretization
1.2.2 Parameter optimization
1.2.3 Motivating example
1.3 The effect of miscellaneous imposed constraints on performance index of biped locomotion during the SSP
1.3.1 Dynamic modeling
1.3.2 Discretization and parameter optimization
1.4 Suboptimal trajectory planning of biped robot during complete gait cycle
1.4.1 Dynamic modeling
1.4.2 Discretization and parameter optimization
1.5 Simulation Results
1.5.1 The effect of constraints on the performance index of biped locomotion during the SSP
1.5.2 Suboptimal trajectory planning of biped locomotion during complete gait cycle

2 Conclusions and Future Work

3 References


Biped robots have gained much attention for decades. A variety of researches has been conducted to make them able to assist or even substitute for humans in performing special tasks. In addition, studying biped robots is important in order to understand the human locomotion and to develop and improve control strategies for prosthetic and orthotic limbs. Some challenges encountered in the design of biped robots are: (1) biped robots have unstable structures due to the passive joint located at the unilateral foot-ground contact. (2) They have different configuration when switching from walking phase to another. During the single support phase, the robot is under-actuated, while turning into an over-actuated system during the double-support phase. (3) Biped robots have many degrees of freedom (DOFs). (4) Biped robots interact with different unknown environments. Therefore, this work is focused on offline computational optimal control strategies for zero-moment point-based biped robots. Computational optimal control has been performed to investigate the effects of some imposed constraints on biped locomotion, such as enforcing swing foot to move level to the ground, hip motion with constant height etc. finite difference approach has been used to transcribe infinite dimensional optimal control problem into finite dimensional suboptimal control problem. Then parameter optimization has been used to get suboptimal trajectory of the biped with the imposing different constraints. In general, any artificially imposed constraint to biped locomotion can lead to increase in value of input control torques. On the other hand, suboptimal trajectory of biped robot during complete gait cycle had been accomplished with different cases such that continuous dynamic response occurs. Enforcing the biped locomotion to move with linear transition of zero-moment point (ZMP) during the DSP can lead to more energy consumption.

1 Computational Optimal Control: Theory and Simulation

One of the challenging problems of biped locomotion is generation of feasible trajectories associated with guaranteed stability and adaptable motion [Vun10]; biped robots have inherent instability in nature. In addition, complete understanding of human motion, which could be lost, can guide the designers to innovate robust biped locomotion. Numerous approaches have been used to generate the motion of the biped robot as detailed in Chapter 2 of [Hay14]. However, there are two efficient methods used for this purpose: the optimization-based gait and center of gravity (COG)-based gait. The latter does not deal with the minimum energy, optimal design, and the different kinematic and dynamic constraints of the biped robot. These problems can be dealt successfully with the optimal control theory [Che09]. Please see [Hay19, Hay18/1, Hay18/2, Hay18/3, Hay17/1, Hay17/2, Hay16, Hay15, Hay14, Hay14/1, Hay14/2, Hay14/3, Hay14/4, Hay13/1, Hay13/2, Hay13/3, Sam08] for more details on dynamics, walking pattern generators and control of biped robots. As mentioned earlier, the optimal control can be classified as: dynamic programming, indirect methods and direct methods. Although, the dynamic programming is less sensitive to the initial guess of the design parameters, it suffers from the curse of dimensionality [Rob05]. The indirect approach represented by Pontryagin’s maximum principle (PMP) demands necessary conditions for optimality, which results in nonlinear, two-boundary value problem [Pan92, Die11]. However, the computational solution may lead to highly nonlinear ODEs. Obtaining necessary conditions of optimality can be intricate for complex dynamic systems such as biped robots [Che09, Seg05]. In addition, the indirect methods are extremely sensitive to the initial guess of the costate equations. Despite this difficulty, [Ros01, Bes02] have investigated the optimal motion of the biped robot during the SSP and during the complete gait cycle respectively using PMP assuming the boundary conditions of the biped robot are known. In light of above, the analyst needs more flexible methods for optimal control problems, represented by the direct methods, by transcribing the infinite dimension problem into finite-dimensional nonlinear programming (static or parameter optimization). This can be implemented by discretization of the controls or the states or both of them, depending on the selected discretization approach, and solving the problem using one of the nonlinear programming algorithms such as sequential quadratic programming (SQP), interior points, genetic algorithm (GA) etc. Despite its ease and robustness, this method can only give suboptimal/approximate solution [Pan92, Die11, Hul96, Goh88].

This report focuses on three important points. First, a systematic brief review of direct optimal control is introduced in Sections ‎1.1 and ‎1.2; the superiority of inverse dynamics-based optimization, its discretization and methods of parameter optimization are focused on. Second, the effect of different constraints on the performance index of the target biped is presented in Section ‎1.3. Whereas, Section ‎1.4 deals with suboptimal trajectory planning of the target biped during the complete gait cycle with. One of clear problems encountered in the dynamic response of the biped robot is the discontinuity of the actuating torques/ground reaction forces at the transition instances during transferring from the SSP to the DSP and vice versa. Therefore, the latter section attempts to solve this problem by tracking desired ground reaction forces adopted from Assumption 4-4 of [Hay14]. Section ‎1.5 describes the simulation results, whereas Section ‎1.6 presents conclusions.

1.1 Forward dynamics-based optimization

In general, formulation of the forward dynamics-based optimal control problem for any nonlinear dynamical system can be described as follows:

Determine the input control in order to minimize the performance index (criterion), ,

illustration not visible in this excerpt

with and are scalar functions of the indicated arguments, is the state vector, and are the time, initial and final time respectively.

Subject to the system differential equations

illustration not visible in this excerpt

where represents the function vector that relates the states to input control. In addition, the following constraints could be available:

Initial constraints:

illustration not visible in this excerpt

where and denote vectors of initial inequality and equality constraints respectively.

Final constraints:

illustration not visible in this excerpt

where and denote vectors of final inequality and equality constraints respectively.

Path constraints:

illustration not visible in this excerpt

where and denote vectors of path inequality and equality constraints respectively.

Box constraints of the input control and the state vector:

illustration not visible in this excerpt

with the subscript refers to lower value, while refers to upper value.

The optimal control problem (Eq. ‎1‑1 to Eq.1‑6) could be converted into parameter optimization problem via discretization of the input control, in case of single and multiple shooting, or both the input control and the state vector in case of collocation method (see Tab. ‎1‑1 which shows the formulations, advantages and disadvantages of the single shooting, the collocation method, and multiple shooting). Then, any technique of nonlinear programming can be used successfully for optimization purposes. For further details, refer to [Die11, Hul96, Goh88].

The formulation of discretized optimal control problem can be described as a nonlinear programming as follows:

Determine: the design vector which may be the control variables or both input control and the states.

illustration not visible in this excerpt

To solve Eq. ‎1‑1 numerically and convert it to Eq. ‎1‑7, any well-known numerical integration approach such as trapezoidal or composite Simpson’s rule etc. can be used successfully.

Subject to:

illustration not visible in this excerpt

Remark ‎1‑1. When applying forward dynamics-based optimization to the multi-body dynamics (robotic system), the following issues should be noticed:

- Eq. ‎1‑2 requires rewriting the equation of motion for biped robot (Eqs- 4-2 and 4-19 of [Hay14]) as follows:

illustration not visible in this excerpt

with and .

Thus, calculation of mass-matrix inverse is required which is often computationally expensive unless a recursive technique is exploited.

- If the multibody dynamic systems move with constrained motion, the equality and inequality constraints may not have explicit expressions for the input control variables. Consequently, these constraints must be differentiated many times until the input control vector appears; for detail, refer to [Bet01].

- To solve the NLP, it is necessary to choose a feasible initial guess for the design variables. As a result, it is not easy to get a good initial guess for the control variables at the forward dynamics-based methods.

Despite the difficulties encountered in the solution of forward dynamics–based optimization, it is adopted as an optimization tool for generating optimal walking patterns of biped robot in [Aze02/2, Rou98].

Remark ‎1 ‑ 2 . After converting the original optimal control problem into NLP, the routine fmincon of the MATLAB Optimization Toolbox can be used easily. In fact, most of the MATLAB routines can be used effectively: ga (genetic algorithm), GlobalSearch, Multistart and the PatternSearch [Mat11]. Becerra [Bec13] made a simple detailed example using fmincon to solve collocation approach.

Remark ‎1‑3. Most books of optimal control concentrate on forward dynamics–based optimization as a direct optimal problem rather than the inverse dynamics-based optimization mentioned later; for detailed comprehension of this interesting topic, see [Die11, Bet01, Ger12].

Tab. ‎1‑1: Formulation, advantages and disadvantages of single shooting, collocation and multiple shooting

illustration not visible in this excerpt

1.2 Inverse dynamics-based optimization

The inverse dynamic equations of robotic systems express the actuating torques/forces in terms of acceleration and states; exactly as described in Chapter 4 of [Hay14]. Thus, the core of inverse dynamics-based optimization is to discretize the system states then the discrete actuating torque vector is obtained using Eqs. 4-2 and 4-19 of [Hay14].

The difference between the inverse dynamics and forward dynamics-based optimization are briefly explained in Tab. ‎1‑2.

Tab. ‎1‑2: Distinctive differences between forward dynamics and inverse dynamics-based optimization

illustration not visible in this excerpt

Stryk [Str97] and Steinbach [Ste97] reported independently that the trajectory optimization obtained from inverse dynamics is faster than that of forward dynamics. Roussel et al. [Rou97] have made a comparative study on the dynamic optimization of point-feet biped robot. The authors have considered the forward dynamics approach using the single-shooting approach with the Euler method as integration method, and the inverse-dynamic approach using the polynomial approximation and the combined polynomial-Fourier series which was used by [Yen87]. They have not considered the piecewise spline and the finite difference-based optimization.

1.2.1 Discretization

In the following, discretization techniques are briefly described. Spline-based discretization

Spline-based optimization has been used extensively in literature. The first reference [Seg05] used a piecewise fourth-order spline function to discretize the problem; the cubic spline functions may result in discontinuities in the third derivative of the approximated joint displacements. However, literature has approved the efficiency of the cubic–spline functions in implementation. In the following, we consider two efficient tools for the solution of inverse-dynamics approach: the piecewise cubic spline functions and the finite difference equations. A detailed study on the spline-based optimization of the biped robot can be found in [Seg05]. To motivate our analysis, let us consider the following simple optimal problem cited from [Pan92].

illustration not visible in this excerpt

Above all, it is recommended to reformulate Eq. ‎1‑28 into second order differential equation; this can facilitate the inverse dynamics-based optimization. Thus, Eq. ‎1‑27 to Eq. ‎1‑30 can be expressed as


illustration not visible in this excerpt

Then, the systems displacement, , is discretized into equidistant segments ( ); see Fig. ‎1‑4. Thus, it can be approximated as

illustration not visible in this excerpt

Fig. ‎1‑4: Discretization of states using piecewise spline functions

Consequently, we have a piecewise spline function of displacement for every interval (segment) with four coefficients determined by using the following connecting and boundary conditions:

· At the inner connecting grids

illustration not visible in this excerpt

All the coefficients of the piecewise spline functions can be determined if the displacements of the target system are known at the grids and the derivatives of these displacements are known at the boundary conditions. From Eq. ‎1‑35, we have unknown coefficients of all splines. These coefficients should satisfy conditions of Eq. ‎1‑36 and four conditions of Eq. ‎1‑37; therefore, all conditions are satisfied. According to above, the design parameters that should be optimized are the displacements of the grid points as well as their derivatives at the boundary conditions only.

Formulation of the piecewise spline–based optimization can be described as


illustration not visible in this excerpt

Minimize using the composite Simpson’s 1/3 Rule [Cha08]:

illustration not visible in this excerpt

Subject to:

illustration not visible in this excerpt

As we see, the constraints are satisfied at only at the grid points rather than all time-axis; therefore, the direct optimal problem is called suboptimal control. In addition, one may ask what is the role of spline function in the discretization; why the analyst directly generate sampled data for displacements. In effect, two fundamental tasks are implemented by spline functions which are: (i) determining the velocity and acceleration of the systems at the inner grid points, and (ii) for constrained motion of robotic systems, the analyst needs only to satisfy the position constraint rather than its first two derivatives to guarantee the continuity of velocity and acceleration; as noted in Eq. ‎1‑36.



ISBN (eBook)
File size
1004 KB
Catalog Number
Walking patterns Zero-momentum point Computational optimal control Biped robots Single support phase Double-support phase Gait cycle



Title: Design of walking patterns for zero-momentum point (ZMP)-based biped robots. A computational optimal control approach