Motion planning is critical to realize the autonomous operation of mobile robots. As the complexity and randomness of robot application scenarios increase, the planning capability of the classical hierarchical motion planners is challenged. With the development of machine learning, the deep reinforcement learning (DRL)-based motion planner has gradually become a research hotspot due to its several advantageous feature. The DRL-based motion planner is model-free and does not rely on the prior structured map. Most importantly, the DRL-based motion planner achieves the unification of the global planner and the local planner. In this paper, we provide a systematic review of various motion planning methods. Firstly, we summarize the representative and state-of-the-art works for each submodule of the classical motion planning architecture and analyze their performance features. Then, we concentrate on summarizing reinforcement learning (RL)-based motion planning approaches, including motion planners combined with RL improvements, map-free RL-based motion planners, and multi-robot cooperative planning methods. Finally, we analyze the urgent challenges faced by these mainstream RL-based motion planners in detail, review some state-of-the-art works for these issues, and propose suggestions for future research.
In multi-view image localization task, the features of the images captured from different views should be fused properly. This paper considers the classification-based image localization problem. We propose the relational graph location network (RGLN) to perform this task. In this network, we propose a heterogeneous graph construction approach for graph classification tasks, which aims to describe the location in a more appropriate way, thereby improving the expression ability of the location representation module. Experiments show that the expression ability of the proposed graph construction approach outperforms the compared methods by a large margin. In addition, the proposed localization method outperforms the compared localization methods by around 1.7% in terms of meter-level accuracy.
This paper studies a finite-time adaptive fractional-order fault-tolerant control (FTC) scheme for the slave position tracking of the teleoperating cyber physical system (TCPS) with external disturbances and actuator faults. Based on the fractional Lyapunov stability theory and the finite-time stability theory, a fractional-order nonsingular fast terminal sliding mode (FO-NFTSM) control law is proposed to promote the tracking and fault tolerance performance of the considered system. Meanwhile, the adaptive fractional-order update laws are designed to cope with the unknown upper bounds of the unknown actuator faults and external disturbances. Furthermore, the finite-time stability of the closed-loop system is proved. Finally, comparison simulation results are also provided to show the validity and the advantages of the proposed techniques.
For the typical first-order systems with time-delay, this paper explors the control capability of linear active disturbance rejection control (LADRC). Firstly, the critical time-delay of LADRC is analyzed using the frequency-sweeping method and the Routh criterion, and the stable time-delay interval starting from zero is accurately obtained, which reveals the limitations of general LADRC on large time-delay. Then in view of the large time-delay, an LADRC controller is developed and verified to be effective, along with the robustness analysis. Finally, numerical simulations show the accuracy of critical time-delay, and demonstrate the effectiveness and robustness of the proposed controller compared with other modified LADRCs.
The attitude tracking control problem is addressed for hypersonic vehicles under actuator faults that may cause an uncertain time-varying control gain matrix. An adaptive compensation scheme is developed to ensure system stability and asymptotic tracking properties, including a kinematic control signal and a dynamic control signal. To deal with the uncertainties of the control gain matrix, a new positive definite one is constructed. The minimum eigenvalue of such a new control gain matrix is estimated. Simulation results of application to an X-33 vehicle model verify the effectiveness of the proposed minimum eigenvalue based adaptive fault compensation scheme.
In this paper, the formation control problem of second-order nonholonomic mobile robot systems is investigated in a dynamic event-triggered scheme. Event-triggered control protocols combined with persistent excitation (PE) conditions are presented. In event-detecting processes, an inactive time is introduced after each sampling instant, which can ensure a positive minimum sampling interval. To increase the flexibility of the event-triggered scheme, internal dynamic variables are included in event-triggering conditions. Moreover, the dynamic event-triggered scheme plays an important role in increasing the lengths of time intervals between any two consecutive events. In addition, event-triggered control protocols without forward and angular velocities are also presented based on approximate-differentiation (low-pass) filters. The asymptotic convergence results are given based on a nested Matrosov theorem and artificial sampling methods.
Since the dynamical system and control system of the missile are typically nonlinear, an effective acceleration tracking autopilot is designed using the dynamic surface control (DSC) technique in order to make the missile control system more robust despite the uncertainty of the dynamical parameters and the presence of disturbances. Firstly, the nonlinear mathematical model of the tail-controlled missile is decomposed into slow acceleration dynamics and fast pitch rate dynamics based on the naturally existing time scale separation. Secondly, the controller based on DSC is designed after obtaining the linear dynamics characteristics of the slow and fast subsystems. An extended state observer is used to detect the uncertainty of the system state variables and aerodynamic parameters to achieve the compensation of the control law. The closed-loop stability of the controller is derived and rigorously analyzed. Finally, the effectiveness and robustness of the design is verified by Monte Carlo simulation considering different initial conditions and parameter uptake. Simulation results illustrate that the missile autopilot based DSC controller achieves better performance and robustness than the other two well-known autopilots. The method proposed in this paper is applied to the design of a missile autopilot, and the results show that the acceleration tracking autopilot based on the DSC controller can ensure accurate tracking of the required commands and has better performance.
Spacecraft orbit evasion is an effective method to ensure space safety. In the spacecraft’s orbital plane, the space non-cooperate target with autonomous approaching to the spacecraft may have a dangerous rendezvous. To deal with this problem, an optimal maneuvering strategy based on the relative navigation observability degree is proposed with angles-only measurements. A maneuver evasion relative navigation model in the spacecraft’s orbital plane is constructed and the observabi-lity measurement criteria with process noise and measurement noise are defined based on the posterior Cramer-Rao lower bound. Further, the optimal maneuver evasion strategy in spacecraft’s orbital plane based on the observability is proposed. The strategy provides a new idea for spacecraft to evade safety threats autonomously. Compared with the spacecraft evasion problem based on the absolute navigation, more accurate evasion results can be obtained. The simulation indicates that this optimal strategy can weaken the system’s observability and reduce the state estimation accuracy of the non-cooperative target, making it impossible for the non-cooperative target to accurately approach the spacecraft.
Spacecraft require a large-angle manoeuvre when performing agile manoeuvring tasks, therefore a control moment gyroscope (CMG) is employed to provide a strong moment. However, the control of the CMG system easily falls into singularity, which renders the actuator unable to output the required moment. To solve the singularity problem of CMGs, the control law design of a CMG system based on a cooperative game is proposed. First, the cooperative game model is constructed according to the quadratic programming problem, and the cooperative strategy is constructed. When the strategy falls into singularity, the weighting coefficient is introduced to carry out the strategy game to achieve the optimal strategy. In theory, it is proven that the cooperative game manipulation law of the CMG system converges, the sum of the CMG frame angular velocities is minimized, the energy consumption is small, and there is no output torque error. Then, the CMG group system is simulated. When the CMG system is near the singular point, it can quickly escape the singularity. When the CMG system falls into the singularity, it can also escape the singularity. Considering the optimization of angular momentum and energy consumption, the feasibility of the CMG system steering law based on a cooperative game is proven.
To meet the requirements of safety, concealment, and timeliness of trajectory planning during the unmanned aerial vehicle (UAV) penetration process, a three-dimensional path planning algorithm is proposed based on improved holonic particle swarm optimization (IHPSO). Firstly, the requirements of terrain threat, radar detection, and penetration time in the process of UAV penetration are quantified. Regarding radar threats, a radar echo analysis method based on radar cross section (RCS) and the spatial situation is proposed to quantify the concealment of UAV penetration. Then the structure-particle swarm optimization (PSO) algorithm is improved from three aspects. First, the conversion ability of the search strategy is enhanced by using the system clustering method and the information entropy grouping strategy instead of random grouping and constructing the state switching conditions based on the fitness function. Second, the unclear setting of iteration numbers is addressed by using particle spacing to create the termination condition of the algorithm. Finally, the trajectory is optimized to meet the intended requirements by building a predictive control model and using the IHPSO for simulation verification. Numerical examples show the superiority of the proposed method over the existing PSO methods.
Considering packet losses, time-varying delay, and parameter uncertainty in the switched fuzzy system, this paper designs a robust fault detection filter at any switching rate and analyzes the H∞ performance of the system. Firstly, the Takagi-Sugeno (T-S) fuzzy model is used to establish a global fuzzy model for the uncertain nonlinear time-delay switched system, and the packet loss process is modeled as a mathematical model satisfying Bernoulli distribution. Secondly, through the average dwell time method and multiple Lyapunov functions, the exponentially stable condition of the nonlinear network switched system is given. Finally, specific parameters of the robust fault detection filter can be obtained by solving linear matrix inequalities (LMIs). The effectiveness of the method is verified by simulation results.
The utilization of traffic information received from intelligent vehicle highway systems (IVHS) to plan velocity and split output power for multi-source vehicles is currently a research hotspot. However, it is an open issue to plan vehicle velocity and distribute output power between different supply units simultaneously due to the strongly coupling characteristic of the velocity planning and the power distribution. To address this issue, a flexible predictive power-split control strategy based on IVHS is proposed for electric vehicles (EVs) equipped with battery-supercapacitor system (BSS). Unlike hierarchical strategies to plan vehicle velocity and distribute output power separately, a monolayer model predictive control (MPC) method is employed to optimize them online at the same time. Firstly, a flexible velocity planning strategy is designed based on the signal phase and time (SPAT) information received from IVHS and then the Pontryagin’s minimum principle (PMP) is adopted to formulate the optimal control problem of the BSS. Then, the flexible velocity planning strategy and the optimal control problem of BSS are embedded into an MPC framework, which is online solved using the shooting method in a fashion of receding horizon. Simulation results verify that the proposed strategy achieves a superior performance compared with the hierarchical strategy in terms of transportation efficiency, battery capacity loss, energy consumption and computation time.
The rapid increase of the scale and the complexity of the controlled plants bring new challenges such as computing power and storage for conventional control systems. Cloud computing is concerned as a powerful solution to handle complex large-scale control missions by using sufficient computing resources. However, the computing ability enables more complex devices and more data to be involved and most of the data have not been fully utilized. Meanwhile, it is even impossible to obtain an accurate model of each device in the complex control systems for the model-based control algorithms. Therefore, motivated by the above reasons, we propose a data-driven predictive cloud control system. To achieve the proposed system, a practical data-driven predictive cloud control testbed is established and together a cloud-edge communication scheme is developed. Finally, the simulations and experiments demonstrate the effectiveness of the proposed system.
This paper focuses on the solution to the dynamic affine formation control problem for multiple networked under-actuated quad-rotor unmanned aerial vehicles (UAVs) to achieve a configuration that preserves collinearity and ratios of distances for a target configuration. In particular, it is investigated that the quad-rotor UAVs are steered to track a reference linear velocity while maintaining a desired three-dimensional target formation. Firstly, by integrating the properties of the affine transformation and the stress matrix, the design of the target formation is convenient and applicable for various three-dimensional geometric patterns. Secondly, a distributed control method is proposed under a hierarchical framework. By introducing an intermediary control input for each quad-rotor UAV in the position loop, the necessary thrust input and the desired attitude are extracted. In the attitude loop, the desired attitude represented by the unit quaternion is tracked by the designed torque input. Both conditions of linear velocity unavailability and mutual collision avoidance are also tackled. In terms of Lyapunov theory, it is prooved that the overall closed-loop error system is asymptotically stable. Finally, two illustrative examples are simulated to validate the effectiveness of the proposed theoretical results.
The coupling efficiency of hollow-core fiber changes with temperature, which leads to the decrease of the finesse ( F ) of fiber resonator and limits the performance of the resonant fiber optic gyroscope (R-FOG) system. Negative-curvature anti-resonant fiber (ARF) can maintain single-mode characteristics under the condition of large mode field diameter, achieve efficient and stable fiber coupling, and significantly improve the consistency of the F of the spatial coupling resonator in variable temperature environment. A new type of ARF with a mode field diameter (MFD) of 25 μm is used to fabricate a fiber resonator with a length of 5.14 m. In the range of 25 °C?75 °C, the averageF is 31.45. The ARF resonator is used to construct an R-FOG system that shows long-term bias stability (3600 s) of 3.1 °/h at room temperature, 4.6 °/h at 75 °C. To our knowledge, this is the best reported index of hollow-core fiber resonator and R-FOG system within the temperature variation range of 50 °C test.
With the development of space rendezvous and proximity operations (RPO) in recent years, the scenarios with non-cooperative spacecraft are attracting the attention of more and more researchers. A method based on the costate normalization technique and deep neural networks is presented to generate the optimal guidance law for free-time orbital pursuit-evasion game. Firstly, the 24-dimensional problem given by differential game theory is transformed into a three-parameter optimization problem through the dimension-reduction method which guarantees the uniqueness of solution for the specific scenario. Secondly, a close-loop interactive mechanism involving feedback is introduced to deep neural networks for generating precise initial solution. Thus the optimal guidance law is obtained efficiently and stably with the application of optimization algorithm initialed by the deep neural networks. Finally, the results of the comparison with another two methods and Monte Carlo simulation demonstrate the efficiency and robustness of the proposed optimal guidance method.
The fault-tolerant consensus problem for leader-following nonlinear multi-agent systems with actuator faults is mainly investigated. A new super-twisting sliding mode observer is constructed to estimate the velocity and undetectable fault information simultaneously. The time-varying gain is introduced to solve the initial error problem and peak value problem, which makes the observation more accurate and faster. Then, based on the estimated results, an improved sliding mode fault-tolerant consensus control algorithm is designed to compensate the actuator faults. The protocol can guarantee the finite-time consensus control of multi-agent systems and suppress chattering. Finally, the effectiveness and the superiority of the observer and control algorithm are proved by some simulation examples of the multi-aircraft system.
In the applications of joint control and robot movement, the joint torque estimation has been treated as an effective technique and widely used. Researches are made to analyze the kinematic and compliance model of the robot joint with harmonic drive to acquire high precision torque output. Through analyzing the structures of the harmonic drive and experiment apparatus, a scheme of the proposed joint torque estimation method based on both the dynamic characteristics and unscented Kalman filter (UKF) is designed and built. Based on research and scheme, torque estimation methods in view of only harmonic drive compliance model and compliance model with the Kalman filter are simulated as guidance and reference to promote the research on the torque estimation technique. Finally, a promoted torque estimation method depending on both harmonic drive compliance model and UKF is designed, and simulation results compared with the measurements of a commercial torque sensor, have verified the effectiveness of the proposed method.
The guidance strategy is an extremely critical factor in determining the striking effect of the missile operation. A novel guidance law is presented by exploiting the deep reinforcement learning (DRL) with the hierarchical deep deterministic policy gradient (DDPG) algorithm. The reward functions are constructed to minimize the line-of-sight (LOS) angle rate and avoid the threat caused by the opposed obstacles. To attenuate the chattering of the acceleration, a hierarchical reinforcement learning structure and an improved reward function with action penalty are put forward. The simulation results validate that the missile under the proposed method can hit the target successfully and keep away from the threatened areas effectively.
In this paper, the optimal control of non-linear switching system is investigated without knowing the system dynamics. First, the Hamilton-Jacobi-Bellman (HJB) equation is derived with the consideration of hybrid action space. Then, a novel data-based hybrid Q-learning (HQL) algorithm is proposed to find the optimal solution in an iterative manner. In addition, the theoretical analysis is provided to illustrate the convergence and optimality of the proposed algorithm. Finally, the algorithm is implemented with the actor-critic (AC) structure, and two linear-in-parameter neural networks are utilized to approximate the functions. Simulation results validate the effectiveness of the data-driven method.
The state estimation of a maneuvering target, of which the trajectory shape is independent on dynamic characteristics, is studied. The conventional motion models in Cartesian coordinates imply that the trajectory of a target is completely determined by its dynamic characteristics. However, this is not true in the applications of road-target, sea-route-target or flight route-target tracking, where target trajectory shape is uncoupled with target velocity properties. In this paper, a new estimation algorithm based on separate modeling of target trajectory shape and dynamic characteristics is proposed. The trajectory of a target over a sliding window is described by a linear function of the arc length. To determine the unknown target trajectory, an augmented system is derived by denoting the unknown coefficients of the function as states in mileage coordinates. At every estimation cycle except the first one, the interaction (mixing) stage of the proposed algorithm starts from the latest estimated base state and a recalculated parameter vector, which is determined by the least squares (LS). Numerical experiments are conducted to assess the performance of the proposed algorithm. Simulation results show that the proposed algorithm can achieve better performance than the conventional coupled model-based algorithms in the presence of target maneuvers.
An impact angle constrained fuzzy adaptive fault tolerant integrated guidance and control method for Ski-to-Turn (STT) missiles subject to unsteady aerodynamics and multiple disturbances is proposed. Unsteady aerodynamics appears when flight vehicles are in a transonic state or confronted with unstable airflow. Meanwhile, actuator failures and multisource model uncertainties are introduced. However, the boundaries of these multisource uncertainties are assumed unknown. The target is assumed to execute high maneuver movement which is unknown to the missile. Furthermore, impact angle constraint puts forward higher requirements for the interception accuracy of the integrated guidance and control (IGC) method. The impact angle constraint and the precise interception are established as the object of the IGC method. Then, the boundaries of the lumped disturbances are estimated, and several fuzzy logic systems are introduced to compensate the unknown nonlinearities and uncertainties. Next, a series of adaptive laws are developed so that the undesirable effects arising from unsteady aerodynamics, actuator failures and unknown uncertainties could be suppressed. Consequently, an impact angle constrained fuzzy adaptive fault tolerant IGC method with three loops is constructed and a perfect hit-to-kill interception with specified impact angle can be implemented. Eventually, the numerical simulations are conducted to verify the effectiveness and superiority of the proposed method.
A compensation implementation scheme of the advanced targeting process based on the fine tracking system is proposed in this paper. Based on the working process of the quantum positioning system (QPS) and its acquisition, tracking and pointing (ATP) system, the advanced targeting subsystem of the ATP system is designed. Based on six orbital parameters of the quantum satellite Mozi, the advanced targeting azimuth angle and pitch angle are transformed into the dynamic tracking center of the fine tracking system in the ATP system. The deviation of the advanced targeting process is analyzed. In the Simulink, the simulation experiment of the ATP system considering the deviation compensation of the advanced targeting is carried out, and the results are analyzed.
Once the spoofer has controlled the navigation system of unmanned aerial vehicle (UAV), it is hard to effectively control the error convergence to meet the threshold condition only by adjusting parameters of estimation if estimation of the spoofer on UAV has continuous observation error. Aiming at this problem, the influence of the spoofer’s state estimation error on spoofing effect and error convergence conditions is theoretically analyzed, and an improved adaptively robust estimation algorithm suitable for steady-state linear quadratic estimator is proposed. It enables the spoofer’s estimator to reliably estimate UAV status in real time, improves the robustness of the estimator in responding to observation errors, and accelerates the convergence time of error control. Simulation experiments show that the mean value of normalized innovation squared (NIS) is reduced by 88.5%, and the convergence time of NIS value is reduced by 76.3%, the convergence time of true trajectory error of UAV is reduced by 42.3%, the convergence time of estimated trajectory error of UAV is reduced by 67.4%, the convergence time of estimated trajectory error of the spoofer is reduced by 33.7%, and the convergence time of broadcast trajectory error of the spoofer is reduced by 54.8% when the improved algorithm is used. The improved algorithm can make UAV deviate from preset trajectory to spoofing trajectory more effectively and more subtly.
Driven by the improvement of the smart grid, the active distribution network (ADN) has attracted much attention due to its characteristic of active management. By making full use of electricity price signals for optimal scheduling, the total cost of the ADN can be reduced. However, the optimal day-ahead scheduling problem is challenging since the future electricity price is unknown. Moreover, in ADN, some schedulable variables are continuous while some schedulable variables are discrete, which increases the difficulty of determining the optimal scheduling scheme. In this paper, the day-ahead scheduling problem of the ADN is formulated as a Markov decision process (MDP) with continuous-discrete hybrid action space. Then, an algorithm based on multi-agent hybrid reinforcement learning (HRL) is proposed to obtain the optimal scheduling scheme. The proposed algorithm adopts the structure of centralized training and decentralized execution, and different methods are applied to determine the selection policy of continuous scheduling variables and discrete scheduling variables. The simulation experiment results demonstrate the effectiveness of the algorithm.
This paper studies the fixed-time output-feedback control for a class of linear systems subject to matched uncertainties. To estimate the uncertainties and system states, we design a composite observer which consists of a high-order sliding mode observer and a Luenberger observer. Then, a robust output-feedback controller with fixed-time convergence guarantee is constructed. Rigorous theoretical proof shows that with the proposed controller, the system states can converge to zero in fixed-time free of the initial conditions. Finally, simulation comparison with existing algorithms is given. Simulation results verify the effectiveness of the proposed controller in terms of its fixed-time convergence and perfect disturbance rejection.
There exist many two-level group consensus problems with different psychological behaviors of decision makers. To deal with these group consensus problems and reach a stable consensus, based on the principles and methods of grey system, utility theory and group consensus, we use grey utility function to describe and reflect decision makers’ opinion preferences in different subgroups and different levels, and then we construct a two-level group consensus method with a moderator, and exploit it to solve the negotiation problems of the natural gas subsidy.
Foot-mounted pedestrian navigation system (PNS) is a common solution to pedestrian navigation using micro-electro mechanical system (MEMS) inertial sensors. The inherent problems of inertial navigation system (INS) by the traditional algorithm, such as the accumulated errors and the lack of observation of heading and altitude information, have become obstacles to the application and development of the PNS. In this paper, we introduce a heuristic heading constraint method. First of all, according to the movement characteristics of human gait, we use the generalized likelihood ratio test (GLRT) detector and introduce a time threshold to classify the human gait, so that we can effectively identify the stationary state of the foot. In addition, based on zero velocity update (ZUPT) and zero angular rate update (ZARU), the cumulative error of the inertial measurement unit (IMU) is limited and corrected, and then a heuristic heading estimation is used to constrain and correct the heading of the pedestrian. After simulation and experiments with low-cost IMU, the method is proved to reduce the localization error of end-point to less than 1% of the total distance, and it has great value in application.
Input variables selection (IVS) is proved to be pivotal in nonlinear dynamic system modeling. In order to optimize the model of the nonlinear dynamic system, a fuzzy modeling method for determining the premise structure by selecting important inputs of the system is studied. Firstly, a simplified two stage fuzzy curves method is proposed, which is employed to sort all possible inputs by their relevance with outputs, select the important input variables of the system and identify the structure. Secondly, in order to reduce the complexity of the model, the standard fuzzy c-means clustering algorithm and the recursive least squares algorithm are used to identify the premise parameters and conclusion parameters, respectively. Then, the effectiveness of IVS is verified by two well-known issues. Finally, the proposed identification method is applied to a realistic variable load pneumatic system. The simulation experiments indicate that the IVS method in this paper has a positive influence on the approximation performance of the Takagi-Sugeno (T-S) fuzzy modeling.
Projects on unmanned aerial vehicle (UAV) swarms have been initiated in a big way in the last few years, especially from 2015 to 2016. As a result, the number of related works on UAV swarms has been on the rise, with the rate of growth dramatically accelerating since 2017. This research conducts a bibliometric analysis of robotics swarms and UAV swarms to answer the following questions: (i) Disciplines mentioned in the UAV swarms research. (ii) The future development trends and hotspots in the UAV swarms research. (iii) Tracking related outcomes in the UAV swarms research.
