Many lower exoskeletons have been developed for strength augmentation and walking assistance scenarios over the past few decades[1-5]. For strength augmentation related applications, lower exoskeletons are designed to track the pilot's motion with little interaction force between the exoskeleton and the pilot[6-8]. The controller of these lower exoskeletons can be roughly categorized into two categories, namely,sensor-based controller and model-based controller.

For sensor-based controllers, extra sensors are always employed to measure the pilot's information and/or the interaction force between the pilot and the exoskeleton[9-11]. With the measured sensory information, many variations of control strategies can be employed to control the lower exoskeleton, i.e.,impedance control strategies. For example, the Hybrid assistive limb (HAL) exoskeleton system is an impedance control strategy proposed by Y. Sankai based on measuring eletro-myo-graphical (EMG)signals of the pilot[12]. In the impedance control strategy of HAL system, EMG signals are utilized to calculate reference patterns of the pilot which aims at estimating the human-exoskeleton interaction (HEI)between the pilot and the exoskeleton[13]. Furthermore,based on measuring the pilot’s motion with acceleration sensors, the active-impedance control strategy[14] and the fuzzy-based impedance control strategy[15] are proposed to adapt to the changing interaction dynamics among different pilots. However,sensor-based controllers heavily rely on complex sensory systems, which are unstable sometimes and is mostly costly. Hence, sensor-based controllers are, to some extent, limited in most strength augmentation scenarios.

On the other hand, model-based controllers are designed to simplify the sensory system of the exoskeleton, which is only based on the information from exoskeleton itself. Sensitivity amplification control (SAC) is one of the model-based controllers proposed by the berkeley lower extremity exoskeleton(BLEEX)[16]. With a sensitivity factor in the modelbased controller, SAC can estimate the output joint torques based on current states (joint angle, angular velocity and angular acceleration) of the lower exoskeleton. The SAC strategy is able to reduce the interaction force between the pilot and the exoskeleton without measuring it directly, which also reduces the complexity of the exoskeleton sensory system.However, the SAC strategy requires accurate dynamic models of the lower exoskeleton (sensitive to model imperfections and different pilots) which makes the system identification process quite complicated[17].

In this paper, we propose a novel variable virtual impedance control (VVIC) strategy which inherits both advantages of sensor-based controllers and modelbased controllers. On the one hand, it is a model-based control strategy, which reduces the complexity of the exoskeleton sensory system. On the other hand, we apply a reinforcement learning method based on policy improvement and path integrals (PI2) to learn parameters of the virtual impedance model, which circumvents the complicated system identification process. The main contributions of this paper can be summarized as follows：

1) A novel VVIC strategy with a model-based controller named virtual impedance controller is proposed, which reduces the exoskeleton sensory system requirement;

随着养蜂产业的扩大，砂仁、花椒、魔芋、万寿菊等经济作物收成的增长，曼来村打赢脱贫攻坚战信心满满。据统计，截至2017年底，曼来村建档立卡贫困户脱贫出列35户133人；2018年计划脱贫出列97户341人，2019年计划脱贫出列3户8人，实现整村脱贫。

2) To reduce the complicated system identification process, a reinforcement learning method is utilized to learn/optimize parameters of the virtual impedance controller of VVIC strategy;

3) The proposed VVIC strategy is verified on both a single DOF platform and HUALEX system.

The proposed VVIC strategy is firstly validated on a single DOF exoskeleton platform, and then tested on a HUALEX system. Experimental results show that the proposed VVIC strategy is able to adapt different HEI to different pilots when compared with canonical model-based control strategies.

1 Virtual Impedance Control Strategy

This section presents the proposed virtual impedance control strategy. We will firstly introduce the design details of virtual impedance controller in Subsection 1.1. Then, in Subsection 1.2, we analyze the stability of the proposed model-based controller.

9) For all time steps i∈[1,T]：

1.1 Virtual Impedance Controller

For the control of lower exoskeletons for strength augmentation related applications, the pilot always plays as a master role in the human-coupled system,which means that the exoskeleton should follow/track the pilot's motion. A general control system block diagram with the model-based controller for the single degree of freedom (DOF) case is depicted in Fig. 1,where： G represents the transfer function of the lower exoskeleton, C is the designed model-based controller of the lower exoskeleton. Khm is the impedance between the pilot and the exoskeleton. qe and qh indicate the joint angles of lower exoskeleton and the pilot, respectively. Thm is the resulting interaction torque applied by the pilot. Tact is the output torque applied by the lower exoskeleton actuator.

有知情人士透露，美的对事业部的整合还会持续。曾经归属环电的清洁电器，传言已久的冰箱和洗衣机、家用空调和中央空调都是潜在整合对象。下一步更深层次的变革将是机制、模式和人员改革，去掉臃肿、下放权力，加大激励让能者多劳。

Fig. 1 A general control system diagram with the model-based controller for the single DOF case

As shown in Fig. 1, the input torque of the lower exoskeleton is combined with the actuator output torque Tact and pilot's resulting interaction torque Thm.The design goal of lower exoskeleton controller is to reduce the interaction torque, which also means that the exoskeleton can track the pilot's motion as soon as possible.

The traditional impedance controller always be designed as in Eq. (1), the pilot's angles are taken as inputs to the controller：where is the estimated dynamics of the lower exoskeleton. k and d are stiffness and damping parameters of the designed impedance model,respectively. However, for the design of model-based controllers, we do not measure sensory information from the pilot. In the exoskeleton control of strength augmentation scenarios, the exoskeleton will receive the pilot's joint states after several control cycles.Therefore, a virtual impedance model is presented for the model-based exoskeleton controller：

where hk and hd are positive parameters of the virtual impedance model. Hence, the proposed virtual impedance controller can be represented as Eq. (4),which is a model-based controller only based on the states of the lower exoskeleton：

where K h = k kh and Dh = d dhare virtual impedance factors of the proposed virtual impedance controller.

1.2 Stability Analysis

Since the design goal of the lower exoskeleton controller is to reduce the interaction torque, Thm approximates to zero, and the stability of the system can be guaranteed by the stability of q e /q h [18].

From Fig. 1, the open loop system equation can be represented as：

11) Normalize ΔΘ according Eq. (20).

where Vh = D h s2 + K h s . Through the model-based controller in Eq. (6) and the system equation described in Eq. (5), we have：

MMP-9又称明胶酶B，作用底物主要是Ⅳ、Ⅴ、Ⅶ、Ⅹ、Ⅺ型胶原以及明胶、纤维粘连蛋白、弹性蛋白等。研究发现，老年小鼠左心室中MMP-9基因和蛋白的表达均明显升高[27-28]；而敲除MMP-9可通过减少TGF-β诱导的骨膜蛋白和结缔组织生长因子的表达，减少胶原沉积，进而减轻老年小鼠的心脏纤维化，改善心脏舒张功能[28]。还有研究发现，TIMP-1在正常心脏组织中随着年龄的增长表达增加，同时伴随着MMP-1或TGF-β表达的增加，心脏胶原比例失调，心脏纤维化发生，提示TIMPs和MMPs的平衡失调参与了年龄相关性心脏纤维化[29-30]。

then the equation of q e /qh can be obtained as：

According to Eq. (12), if the virtual impedance parameters Dh and Kh are small enough (always positive), the system will still be stable when the dynamic model of lower exoskeleton is not over estimated (J ＜, B ＜ ). Hence, the system is always stable when the dynamic model of lower exoskeleton is not over estimated.

If the dynamics of lower exoskeleton is estimated accurately ( 1G= ), then Eq. (8) can be simplified to：

Since virtual impedance parameters Dh and Kh and the impedance Khm all have positive values, the control system is always stable when the dynamics of lower exoskeleton can be accurately estimated.

Another situation is that we haven't gained accurate dynamic models of the lower exoskeleton. In this case, we consider a single DOF exoskeleton with the second order dynamics and ignore the gravity composition, which indicates that：

将农村生活垃圾纳入城镇垃圾处理体系，推动了城乡垃圾一体化处理。每户配备小型垃圾桶和手推车对垃圾进行分类收集，每村要配置一处垃圾收集站。收集站的垃圾定时由乡镇政府清运，根据距离处理场的远近不同，分批运往中转站或垃圾填埋场。强化畜禽养殖污染治理，提高畜禽粪污收集和处理机械化水平。

where J and B represent the inertial moment and viscous Friction of the lower exoskeleton, respectively.The estimated exoskeleton dynamics can be represented as：

where and represent the estimated inertial moment and viscous friction parameters. From Eq. (10)and Eq. (11), the equation of q e /q h can be represented as：

② 低 PLR组（PLR≤125）患者 114例（44.4%），高 PLR组 （PLR＞125）患者 143例（55.6%）。两组患者的各临床资料之间差异无统计学意义。

2 Virtual Impedance Adaptation through Reinforcement Learning

For the implementation of traditional model-based controllers, i.e. SAC in BLEEX system, the systemidentification process is often employed to obtain system dynamics and human-related parameters of the designed controller (sensitivity factors in SAC)[19].However, the lower exoskeleton is a kind of humancoupled system for different pilots, which requires that the controller needs to recalibrate for different pilots.

In this paper, a model-free reinforcement learning method is employed to learn the optimal virtual impedance parameters of VIC, which aims at adapting with different HEI for different pilots. Combining the learning process and the model-based controller, which is named as the VVIC strategy, we can reduce the system sensor requirement as well as the system identification process. In the reinforcement learning process, a model-free reinforcement learning method named policy improvement and path integral (PI2)algorithm[20-21] is employed to learn the parameters Kh and D h of VIC.

The parameterized policy of PI2 is defined as：

14) Until Trajectory cost R is converged.

Eq. (14) calculates the jth average weight, where n is the number of parameters which is to be learned(n=2 in our case).

CBCT位置验证可纠正摆位误差，提高靶区位置精度，以CBCT验证后的靶区位置精度为标准，利用体表光学监测系统采用Vision RT影像作为参考影像进行体表位置变化比对，在治疗中体表光学监测结果与CBCT位置验证结果一致，与Wikström等[18]报道类似。故体表光学监测可用于头颈部放疗分次内的靶区部位体表位置监测，从而监测分次内靶区位置变化。

where α1 andα2 are positive scale factors. In order to obtain the pilot motion information during the learning process, inclinometers are utilized to measure the pilot's joint angle position qh and angular velocity

With the defined policy and cost function, the learning process of virtual impedance parameters based on PI2 for single DOF algorithm is described in as：

1) Initialize the virtual impedance parameter vector Θ.

2) Initialize basis function W ti according Eq. (14).

3) Repeat.

4) Run K gait cycles of the exoskeleton using stochastic parameters Θ +ϵt at every time step.

5) For all gait cycles k ∈[1,K]：

6) Compute the projection matrix M through Eq.(16).

7) Compute the stochastic cost S through Eq. (17).

8) Compute the probability P through Eq. (18).

In the implementation of VVIC strategy, we define the immediate cost based on the measured sensory information of the pilot. For single DOF case,the immediate cost function is defined as follows：

在新课程改革的大背景下，课程改革标准也对学校体育教学提出了新的要求：“要着力突显学生在体育教学中的主体地位，要提高对学生积极参与精神的重视程度，要在全面了解学生个体差异的前提下因材施教，要保证每一位学生在体育教学中都能受益。”因此，学生可以在体育课的自主性活动中尽情展示自身的特长，挖掘自身的运动潜能，要积极转变学生一味被动接受的局面，使其能够自主探究学习，并在这一过程中找到学习的乐趣，通过不断丰富学生的运动体验来达到唤醒学生创新精神的目的[2]。为了能在活动中最大程度的激发学生们独立探索创新的热情，活动组织者应当以以下几点内容为教学活动设计的参照点：

1)Java程序设计课程需要对程序设计语言(如C或C++或计算概论等)有所了解，有很多学生对于语言理解的不够透彻，在学习过程中十分困难。教师在定制教学大纲时，通过泛雅平台帮助学生们回顾或了解一下相关的程序设计语言，设置一个前导课程。

10) Compute ΔΘti for each time step through Eq.(19).

The proposed virtual impedance controller can be rewritten as：

12) Update Θ ← Θ + ΔΘ .

13) Run one noiseless gait cycle to compute the trajectory cost R through Eq. (21).

3）思考授课过程：之前笔者关注到，在过去培训过程中，授课老师较少与新员工互动。笔者思考可否走入新员工之间进行授课，打破老师远远地站在讲台后面高高在上的固化形象，先从形式上拉近彼此的距离，再通过授课结束后与他们的交流（如满意度调查、留提问时间、结束后个别探讨）来增加实质的接触。

where Θ is a vector of virtual impedance parameters[K h , D h ]T and ϵt indicate the exploration noise. WtT is the basis function with Gaussian kernels ω ：

是指在旅游中利用现代信息技术，建立起综合性强的数据库，通过网络平台的功能，实现信息的交流，可以满足不同的需求、旅游信息化主要是对旅游相关产业链进行连接，从而形成实体资源，信息资源、生产要素资源的分配、组合、加工、传播、销售，为消费者提供优质的服务。改变了传统模式，提升了整体水平，用一种全新模式运行，可以大大提高效率。企业将信息化技术和自身情况结合在一起，可以增强市场竞争力，获得更好的发展机遇，实现可持续发展。由此可见，旅游信息化是时代发展的必然，能够适应社会环境变化的需求。

As shown in Tab. 1, virtual impedance parameters of VIC will be updated every K+1 gait cycles. The updating rule is described in Tab. 1 with Eq. (16) to Eq.(20).

The matrix H in Eq. (16) and Eq. (17) is a positive semi-definite weight matrix. The scale factor λ in Eq.(18) is set within (0,1]. With updated parameter vector Θ, a noiseless gait cycle (without exploration noise ϵt) is taken to determine whether the learning process should be terminated through calculating the trajectory cost R：

where 1/dtρ= (dt indicates time duration of the gait cycle) is a normalization factor, since the duration of each gait cycle are always different in real-time applications.

In the implementation of VVIC strategy, the reinforcement learning process needs to be taken in the case of different pilots, which learns optimal virtual parameters to adapt different HEI to different pilots.Afterwards, with the learned optimal model-based controller, the lower exoskeleton is able to track the pilot's motion as soon as possible based only on joint information of lower exoskeleton.

3 Experiments and Discussions

In this section, the proposed VVIC strategy is validated both on a single DOF case in simulation environment and the HUALEX system. Experimental results and discussions will be introduced in next two subsections.

3.1 Single DOF Case in Simulation Environment

3.1.1 Introduction to the Single DOF Exoskeleton Platform

Fig.2 illustrates the model of single DOF exoskeleton when coupling with the pilot in knee joint.As a human-coupled system, the resultant torque on exoskeleton knee joint is combined with two parts： one is Tact which is provided by the exoskeleton actuator,and another is Thm which is provided by the pilot through compliant connection between the exoskeleton and the pilot.

2.1.1 营养支持:由于疾病所带来的困扰,输尿管异位开口的患者往往胃口不好,进食偏少,所以不少患者有代谢紊乱、营养不良的现象。而手术治疗必然形成一定创伤,或者切除部分组织,这就要求患者保持正常的营养和代谢水平。因此术前患者必须重视增加营养,建议患者多吃高糖、高蛋白、高维生素的饮食。另外还应鼓励患者要多饮水,以增加尿量,减少尿盐析出沉淀,为尽早接受手术做好充分的准备工作。

Fig. 2 Model of single DOF exoskeleton coupling with the pilot in knee joint

The dynamics of single DOF exoskeleton including the pilot is defined as Eq. (22) in the simulation environment：

where the last term mgl⋅sin q e is the gravity composition. Hence, according the control law of proposed VIC in Eq. (4), the controller of single DOF exoskeleton is designed as follows：

where and are estimated inertial moment and viscous friction parameters, respectively. K h and D h are the virtual impedance parameters of proposed VVIC strategy which should be learned to adapt different HEI to different pilots.

3.1.2 Experiments of Simulated Single DOF Exoskeleton

In the experiments of simulated single DOF exoskeleton, different values of the impedance K hm(described in Fig.1) are used to simulate different HEIs to different pilots. Here we choose three different impedance K h m. The estimated dynamic parameters of model-based controller and are set as=0.9 J, =0.9 B with suitable values. Pilot's motion angles are set as periodic sine waves with different frequencies and amplitudes in simulation experiments.

Fig. 3 Learning curves of reinforcement learning process for simulated different pilots

In the learning process of the proposed VVIC strategy, the exoskeleton should take several gait cycles to obtain optimal virtual impedance parameters of the controller. The exoskeleton updates parameters every 4(K=4) gait cycle and spends one gait cycle to calculate trajectory cost R (the parameters Θ will beupdated every 5 gait cycles). Weight parameters 1α and 2α of immediate cost function (described in Eq.(15)) are both chosen as 1 500. Fig. 3 illustrates learning curves of reinforcement learning process for different simulated pilots (relationship between values of impedance parameters hmK is C＞B＞A). As shown in Fig. 3, the learning process will take almost 120 gait cycles (24 updates) to obtain optimal virtual impedance parameters (trajectory cost R converged).

After obtaining the optimal parameters of the VVIC strategy, comparative experiments are carried out to compare the proposed VVIC and traditional SAC algorithm. Fig. 4 shows the control performances of the proposed VVIC strategy and SAC algorithm with pilot A. In the comparison experiments, we choose 11 gait cycles (total 50 gait cycles) with different motion patterns to compare control performances of VVIC strategy and SAC algorithm.Black curves in Fig. 4 represent the interaction force between the pilot and exoskeleton, which is calculated by a spring-damping model in the simulator. As shown in Fig. 4, experimental results show that the proposed VVIC strategy achieves better performance (with less interaction force) than the traditional SAC algorithm.

Fig. 4 Control performances of the proposed VVIC strategy and SAC algorithm

Tab. 1 shows the normalized mean square error(nMSE) of VVIC strategy and SAC algorithm in total 50 gait cycles with three different simulated pilots.Results show that the proposed VVIC strategy achieves better performance when dealing with different HEI to different pilots, e.g. with simulated pilot C, nMSE of the SAC algorithm is almost three times comparing with the proposed VVIC strategy(0.124 rad compare to 0.038 rad).

Table 1 Comparison of VVIC strategy and SAC algorithm for three different simulated pilots in single DOF case

NMSE/rad Simulated pilot A Simulated pilot B Simulated pilot C VVIC 0.024 0.032 0.038 SAC 0.069 0.094 0.124

3.2 Experiments on the HUALEX System

3.2.1 Introduction to the HUALEX System

HUALEX system is designed for the strength augmentation applications. Fig. 5 shows the total HUALEX system with a pilot. In Fig. 5, 1— The pilot;2— The load backpack with the power unit and main controller (rigid connection with the HUALEX spline);3— Semi-rigidly connecting HUALEX to the pilot(waist, thighs, shanks and feet); 4— Active joints with DC servo motors (hip joints and knee joints); 5—Node controllers for active joints; 6— Smart shoes with plantar sensors.

Fig. 5 HUALEX with the pilot

As shown in Fig. 5, four active joints (hips and knees) are designed to provide active torques for strength augmentation. Ankle joints of HUALEX system are energy-storage mechanisms which can store energy in the stance phase and release it in the swing phase. Between the pilot and HUALEX system, many compliant connections are utilized to connect the pilot and HUALEX system in a semi-rigid way.

The control system of HUALEX is combined with one main controller and four node controllers for each active joints. The control algorithm is running on the main controller, and node controllers are aiming to collect sensory information and execute control commands. In the HUALEX system, three kinds of sensors are utilized in the sensory system： 1) Encoders are embedded in each active joint to measure motion information of HUALEX. 2) IMU sensors are utilized to measure motion information of the pilot if necessary.3) Plantar sensors in smart shoes are aiming to judge the walking phases of HUALEX.

3.2.2 Experimental Setup

In experiments of the HULEX system, three different pilots (A： 172 cm/76 kg, B： 176 cm/80 kg, C：180 cm/96 kg) are chosen to operate the HUALEX system in sequence, which indicates that during learning process of VVIC strategy, the learned optimal parameters of VVIC with pilot A will be regarded as initial values of VVIC with pilot B (note that the VVIC parameters of each joint of HUALEX system are learned independently). During the learning process,IMU sensors are utilized to measure the pilot's motion information for obtaining optimal virtual impedance parameters. Besides the virtual impedance parameters of VVIC, parameters of HUALEX dynamics are identified through Solidworks software. After obtaining optimal parameters of VVIC, IMU sensors are remained to capture the pilot's motion information(not use for control) which are aiming to validate the control performance of the proposed VVIC strategy.

3.2.3 Results and Discussions

Fig. 6 shows the learning curves of VVICs in the HUALEX system with different pilots (in left hip and knee joints). As discussed in experimental setup section, pilot A operates the HUALEX system at first so that the learning process of VVIC strategy needs to spend more training gait cycles (almost 140 gait cycles). With better initial values from pilot A, the learning process of pilot B and C can be reduced to almost 80 gait cycles.

After obtaining optimal virtual impedance parameters of the VVIC strategy through thereinforcement learning process, we validate the control performance of proposed VVIC strategy with comparison to the traditional SAC algorithm. The results show that the proposed VVIC strategy achieves good control performances. Moreover, Tab. 2 gives the comparison of the VVIC strategy and SAC algorithm with different pilots (100 gait cycles for each pilot). As shown in Tab. 2, the proposed VVIC strategy achieves better performances in experiments of the HUALEX system with different pilots, e.g. in the right knee joint of pilot C, the nMSE of SAC algorithm is almost three times than that of the VVIC strategy (0.094/rad compare to 0.032/rad).

(2)机组给水温度提高后，低负荷工况锅炉SCR脱硝装置入口烟气温度提升至310 ℃以上，可保证SCR在全负荷范围内处于催化剂的高效区运行。

Fig. 6 Learning curves of VVICs in the HUALEX system with different pilots at joint

Table 2 Comparison of SAC and VVIC strategy in HUALEX with different pilots in total 100 gait cycles

NMSE/rad VVIC|SAC Pilot A Pilot B Pilot C Left hip 0.026 0.078 0.028 0.085 0.025 0.086 Left knee 0.036 0.086 0.038 0.092 0.032 0.09 Right hip 0.024 0.065 0.023 0.068 0.026 0.075 Right knee 0.028 0.087 0.031 0.079 0.032 0.094

4 Conclusions and Future Work

This paper has proposed a novel VVIC strategy to control of a HUALEX system, which aims at adapting different HEI to different pilots. The proposed VVIC strategy is based on a novel VIC, which is a model-based controller with a virtual impedance model.In order to adapt different HEI to different pilots, the PI2 reinforcement learning algorithm is employed to obtain optimal parameters in virtual impedance of VIC.Control performances of the proposed VVIC strategy are validated on a single DOF exoskeleton simulation environment as well as the HUALEX system.Experimental results indicate that the proposed VVIC has better performances compared with the traditional SAC algorithm, and can deal with variation HEI from different pilots.

五山镇地处谷城西北部，跨北纬 32°13'-32°25'，东经111°12'-111°25'，西与房县为邻，北与丹江口市交界，西南毗邻紫金镇，东南连接石花镇。因境内有马鞍山、云雾山、邱家山、百日山、李家山等五座山峰而得名。茶叶是五山镇域经济的主要支撑，目前已形成5.8万亩生态茶园，年产3000多吨、产值过10亿元的“五山玉皇剑”茶，被誉为“湖北十大名茶精品”、中国国际农业博览会名牌产品。

In the future, we will investigate the methods which can learn/update the parameters of VVIC online.In this case, the HUALEX will be able to 'get used to'the pilot during the operation process. Moreover, the estimation to the accurate dynamic models of HUALEX is also important, the accurate dynamic models always achieve better performances for model-based controller in strength augmentation lower exoskeletons.

References

[1]KAZEROONI H, CHU A, STEGER R. That which does not stabilize, will only make us stronger[J]. International Journal of Robotics Research, 2007, 26(1)： 5-89.

[2]SANKAI Y. HAL： Hybrid assistive limb based on cybernics[J]. Robotics Research, 2010： 25-34.

[3]WALSH C J, PALUSKA D, PASCH K, et al. Development of a lightweight, under-actuated exoskeleton for loadcarrying augmentation[C]//IEEE International Conference on Robotics and Automation (ICRA). Florida, USA： IEEE,2006： 3485-3491.

[4]STAUSSE K A, KAZEROONI H. The development and testing of a human machine interface for a mobile medical exoskeleton[C]//IEEE International Conference on Intelligent Robots and Systems (IROS). California, USA：IEEE, 2011： 4911-4916.

[5]ESQUENAZI A, TALATY M, PACKEL A, et al. The rewalk powered exoskeleton to restore ambulatory function to individuals with theracic-level motor-complete spinal cord injury[J]. American Journal of Physical Medicine and Rehabilitation, 2012, 91(11)： 911-921.

[6]HUANG R, CHENG H, CHEN Q, et al. Interative learningfor sensitivity factors of a human-powered augmentation lower exoskeleton[C]//IEEE International Conference on Intelligent Robots and Systems (IROS). Hamburg, Germany：IEEE, 2015： 6409-6415.

[7]WALSH C J, PASCH K, HERR H. An autonomous,under-actuated exoskeleton for load-carrying augmentation[C]//IEEE International Conference on Intelligent Robots and Systems (IROS). Hamburg, Germany： IEEE,2006： 1410-1415.

[8]ZOSS A, KAZEROONI H, CHU A. On the mechanical design of the berkeley lower extremity exoskeleton (BLEEX)[C]//IEEE International Conference on Intelligent Robots and Systems (IROS). Edmonton, Canada： IEEE, 2005：3132-3139.

[9]TRAN H T, CHENG H, LIN X, et al. The relationship between physical human-exoskeleton interaction and dynamic factors： using a learning approach for control applications[J]. Science China Information Science, 2014,57(12)： 1-13.

[10]KAZEROONI H, STEGER R, HUANG L, et al. Hybrid control of the berkeley lower extremity exoskeleton(BLEEX)[J]. International Journal of Robotics Research,2006, 25(6)： 561-573.

[11]KAWAMOTO H, SANKAI Y. Power assist method based on phase sequence and muscle force condition for HAL[J].Advance Robotics, 2005, 19(7)： 717-734.

[12]LEE S, SANKAI Y. Power Assist control for walking aid with hal-3 based on EMG and impedance adjustment around knee joint[C]//International Conference on Intelligent Robots and Systems (IROS). Lausanne,Switzerland： [s.n.], 2002： 1499-1504.

[13]HAYASHI T, KAWAMOTO H, SANKAI Y. Control method of robot suit HAL working as operator's muscle using biologic and dynamical information[C]//IEEE International Conference on Intelligent Robots and Systems (IROS). Edmonton, Canada： IEEE, 2005： 3063-3068.

[14]GABRIEL A O, COLGATE J E, PESHKIN M A, et al.Active-impedance control of a lower-limb assistive exoskeleton[C]//IEEE International Conference on Rehabilitation Robotics. Noordwijk, Netherlands： IEEE,2007： 188-195.

[15]TRAN H T, CHENG H, DUONG M K, et al. Fuuzy-based impedance regulation for control of the coupled human-exoskeleton system[C]//IEEE International Conference on Robotics and Biomimetics. Bali, Indonesia：IEEE, 2014： 986-992.

[16]KAZEROONI H, RACINE J L, HUANG L, et al. On the control of the berkeley lower extremity exoskeleton(BLEEX)[C]//International Conference of Robotics and Automation (ICRA). Barcelona, Spain： [s.n.], 2005：4353-4360.

[17]GHAN J, STEGER R, KAZEROONI H. Control and system identification for the berkeley lower extremity exoskeleton[J]. Advanced Robotics, 2006, 20(9)： 989-1014.

[18]RACINE J L. Control of a lower extremity exoskeleton for human performance amplification[D]. California, USA：University of California, Berkeley, 2003.

[19]GHAN J, KAZEROONI H. System identification for the berkeley lower extremity exoskeleton (BLEEX) [C]//International Conference of Robotics and Automation(ICRA). Florida, USA： [s.n.], 2006： 3477-3484.

[20]THEODOROU E A, BUCHILI J, SCHAAL S. A generalized path integral control aproach to reinforcement learning[J]. Journal of Machine Learning Research,2010,11： 3137-3181.

[21]BUCHLI J, STULP F, THEODOROU E A, et al. Learning variable impedance control[J]. International Journal of Robotics Research, 2011, 30(7)： 820-833.