1. Introduction
Autonomous vehicles (AVs), as one of the most promising topics in both academia and industry in recent decades, are expected to change the future of the world in various ways [
1]. AVs can benefit the next generation of transportation systems due to their safety, mobility, and efficiency advantages [
2]. Moreover, AVs are effective test platforms for various advanced technologies, such as artificial intelligence, automation control, machine learning, and so on [
1]. Based on efforts from around the world, related technologies have been studied in depth [
1,
3]. Considerable related applications have been tested in controlled environments and applied in commercial vehicles [
3]. For example, Google’s AVs have driven more than 1.2 million miles on public roads [
4]. All Tesla vehicles in production are equipped with all the hardware needed for fully autonomous capability and newer applications will gradually be activated [
5]. Although preliminary successes have been achieved with AVs, there are still many intractable issues from both a technical and a societal perspective [
2,
6].
Generally, the architecture of AVs can be divided into three layers, i.e., a perception layer, a planning layer, and a trajectory control layer [
1]. The perception layer aggregates real-time data from multiple sensors to estimate the surrounding scenarios. The trajectory control layer generates commands to control the vehicle according to the planned trajectory and velocity. As the core module, planning layer usually consists of three modules, i.e., route planning, behavior decision-making, and path planning. Route and path planning module generate proper global routes and safe local trajectories. The behavior decision-making module provides safe and reasonable abstract driving actions (e.g., “change lane to left”). The state-of-the-art decision-making system of AVs should have a forward-looking ability, which means that the surrounding vehicles’ future driving behavior should be predicted accurately [
7]. Without the prediction function, emergency incidents may happen, such as the collision between MIT’s “Talos” AV and Cornell’s “Skynet” AV during the 2007 urban challenge, which was held by the Defense Advanced Research Projects Agency (DAPRA) [
3].
As mentioned above, driving behavior prediction plays a significant role in AVs’ decision-making system. However, as latent information, the surrounding vehicles’ future behaviors cannot be directly measured by any available sensors or obtained by inter-vehicle communication technologies (e.g., Connected Vehicles (CV)-related technologies have not been widely deployed) [
8].
In order to predict surrounding vehicles’ driving behavior, AVs should monitor the variation in the target vehicle’s movement and understand the current scenario from the view of the target vehicle. Then, the future behavior can be predicted by prediction models. Usually, three types of information are applied, i.e., vehicle kinematics, the relationships between target vehicle and surrounding entities (e.g., other vehicles, lane lines), and a priori knowledge. Vehicle kinematics refers to the target vehicle’s movement states, such as the lateral velocity and acceleration. The interactions with road elements and other vehicles are described by relative distance, velocity, and direction from the target vehicle to a given entity. These relationships can be characterized by continuous values (e.g., 10 meters) or logical sentences (e.g., “
isCloseTo”). At each time step, for a given target vehicle, a traffic scenario can be defined by these relationships. Each relationship can be considered as one scenario-dependent characteristic. A diversity of scenarios can be defined for the variation of these characteristics. The interactions between the target vehicle and road entities or other vehicles are regulated by a priori knowledge (e.g., traffic rules and commonsense knowledge of driving) [
9]. For example, it can be believed that a vehicle moving in the leftmost lane has a very low probability of turning left, as otherwise it will crash into the curb.
Without an inborn ability to perceive and interpret traffic scenarios accurately predicting driving behavior for typical urban roadways is quite challenging for AVs [
10]. Most existing models for driving behavior prediction were based on one or multiple types of information mentioned in the previous paragraph.
Table 1 presents related up-to-date studies [
11,
12,
13,
14,
15,
16,
17,
18,
19,
20] that predict surrounding vehicles’ driving behaviors. In the table, existing related studies are categorized into three groups according to the features applied in the study.
Vehicle kinematics and relations with road entities were considered by almost all existing studies. These features can be applied directly by TTx (Time to something) models to predict the behavior of the target vehicle. For example, Lefèvre et al. used TTC (Time to Line Crossing) to predict whether the vehicle will depart from the current lane or not [
16]. These features can also be pre-learned by models and then we can find the most similar behavior during the on-line prediction phase by pattern matching. The Hidden Markov Model (HMM) and Support Vector Machines (SVM) with Bayesian Filter (BF) are usually applied to learn the pattern of each behavior [
12,
20]. Besides vehicle kinematics and relations with road entities, interactions between the target vehicle and its surrounding vehicles also have a significant influence on target vehicles’ future behavior. A Dynamic Bayesian Network (DBN) was applied widely to develop interaction-aware prediction models. The interaction-aware models were also believed to be the most comprehensive according to the study of Lefèvre et al. [
9]. However, their applications in AVs are limited because of the expensive computation [
9]. In order to relieve the curse of dimensionality, Bahram et al. [
14] proposed a long-term behavior prediction framework in which model-based and supervised-learning-based subtasks were combined. A complex spatial–temporal cost map with consideration of traffic rules was constructed to determine the interactions. However, in this study, just 60 lane changes of other traffic participants were considered and it is difficult to construct a cost map. All the previous studies tried to build a driving behavior prediction approach with consideration of as many clues as possible, but most previous models fail to consider all the clues. In particular, a priori knowledge is never even considered.
Most mentioned models were designed for one specific scenario, such as lane-changing models for a non-junction segment, turn left/right models for a junction area, etc. However, AVs drive through dynamically changing environments in which a diversity of scenarios show up over time. Multiple scenario-specific models should be used and appropriate models should be activated according to the scenario characteristics, which means that a reasonable adaptation approach should be designed. Aiming to solve the adaptation issue, Gadepally et al. proposed a Hybrid State System and Hidden Markov Model (HSS + HMM) to graft and prune “meta-states” (a group of candidate models) [
17,
18]. The different scenarios were specified by Finite State Machine (FSM) with scenario-dependent features. However, this model was applied for driving behavior estimation (estimating behavior after it has actually happened) and not for prediction. The FSM will become untraceable for large-scale scenarios due to its inherent disadvantages. Another study [
19] proposed a hierarchy scenario model tree (SMT) to specify different scenarios and used single-layer perceptron (SLP) to learn scenario-specified behaviors. For both of these related studies, the traffic rules and drivers’ driving experience were not considered and so it is difficult to integrate them into their proposed models, such as “vehicles on a smaller road yield to vehicles on the main road”. Furthermore, simply scenario representation methods were applied and only the relationships between two vehicles were considered. However, the target vehicles’ future behavior is influenced by multitudinous surrounding entities.
Based on the literature review and the comparison of existing methods, the main objects of this study are summarized as:
• Improving the adaptability of existing models.
In the real world, AVs should drive through a diversity of complex urban scenarios. A reliable activation strategy that can select appropriate candidate prediction models for a given scenario will extend the application scope of the traditional prediction models.
• Combining priori knowledge to improve the quality of prediction.
Priori knowledge information is helpful for driving behavior prediction [
9]. For example, it is believed that a moving vehicle has quite a low probability of changing lanes to the right when there is another vehicle abreast in the right lane. Based on the literature review, it is still not possible to describe these relationships clearly without increasing the computation complexity.
Aiming at the aforementioned goals, this study proposed a scenario-adaptive approach to predict driving behaviors on urban roadways. The main contributions of this study are as follows:
• A novel ontology-based model was proposed to represent the traffic scenarios.
This study developed a novel ontology-based model to represent the spatial state of surrounding entities and their inter-relationships. Based on this model, the scenario can be expressed explicitly and semantic inter-relationships can be represented, which makes it more understandable than traditional scenario description approaches.
• A rule-based reasoning system was developed to improve the adaptability of the driving behavior prediction models.
Based on the structured description of traffic scenarios, adaptation rules were edited. In the rules, candidate scenario-specific models, input features, and priori probabilities for candidate models were specified based on the scenario characteristics.
• The future behavior of the target vehicle was determined by both a priori and a posteriori knowledge.
This is the first study to combine a priori probability and a posteriori probability for behavior prediction. HMM-based probabilistic models were trained off-line and tested on-line to get the a posteriori probability for each possible behavior. A priori probability is provided by a rule-based reasoning system.
In the following sections, the problem formulation and an overview of the proposed approach are presented. The ontology model to describe traffic scenarios and the HMMs to learn continuous features are presented. Then, the approach to predict the future behavior of a target vehicle is given. Finally, the proposed approach is evaluated in real traffic scenarios on urban roadways. The results of the experiments are presented and discussed. We finally conclude this paper with contributions, limitations, and future work in
Section 4.
4. Conclusions
In this study, a scenario-adaptive driving behavior prediction approach was proposed for autonomous driving on urban roadways. The scenario-dependent characteristics (road elements, traffic participants, and their inter-relationships) were represented by the proposed ontology model. The continuous features of each driving behavior were learned by HMMs with field data. Based on a priori knowledge (traffic rules, driving experience), a rule-based reasoning method is designed to specify candidate models, input features, and a priori probabilities for different scenarios. The future behavior of each target vehicle was predicted with consideration of both the a priori probabilities and the testing results of trained HMMs (a posteriori probabilities).
Experiments were conducted to evaluate the proposed approach. Based on the experimental results, the driving scenarios on the urban roadways can be described and interpreted by the proposed ontology model. For various urban traffic scenarios, the candidate models, input features, and a priori probabilities can be generated. The target vehicle’s lane-changing behaviors can be predicted on average 1.85 s (for LCL) or 2.10 s (for LCR) in advance. Compared with the state-of-the-art driving behavior prediction model, the proposed approach can extend the prediction time horizon by up to 56% (0.76 s) on average, and the precision can be improved by 26% for long-term predictions. The proposed ontology model can improve an autonomous vehicle’s scenario understanding ability greatly with the considering of semantic information. The evolution of the scenario can be predicted by the proposed approach, which can serve as the basis for safer decision-making algorithms for AVs.
Even though the proposed approach has advantages for driving behavior prediction, there are still some limitations, such as the time-consuming and tedious work to edit the SWRL rules. In future work, we will apply more advanced methods to generate these rules, such as learning rules from the training data. Furthermore, more field data will be collected and the proposed framework will be verified for more scenarios (e.g., highway entrance, T-junctions, uncontrolled intersections, etc.) with more driving behaviors.