Complex Sequential Modeling

Wu, Guojun

Etd

Complex Sequential Modeling

Público

Traditional sequential models such as Recurrent neural networks (RNNs) have achieved significant success in learning complex patterns for sequential input data. At each time step, an RNN stores the previous hidden state vector, and upon receiving the current input vector, linearly transforms the tuple and passes it through a non-linearity to update the state vectors over T time steps. Subsequently, RNNs output the predictions as a function of the hidden states. The model parameters (i.e. state/input/prediction parameters) are learned by minimizing an empirical loss. However, for real world applications involving sequential data like sensor data and human behavior data, plain RNNs usually tends to produce poor results. First, vanishing and exploding gradients often occur in training multi-layer RNNs which is widely used to solve problems with complex patterns. Secondly, recurrent neural networks can not model the problems which current state is related to future expectations, such as problems involving human decision-making process. The main theme of my dissertation is to design, develop and evaluate efficient models to mitigate those issues. I focus on designing RNN models with theoretically guaranteed training stability and applying inverse reinforcement learning model to human decision problems in my dissertation. The dissertation contains following five research themes. 1) Human Decision Modeling with Sequential Human Decision Data. In this paper, I make the first attempt to model passengers' preferences of making various transit choices using Markov Decision Process (MDP). Moreover, we develop a novel inverse preference learning algorithm to infer the passengers' preferences and predict the future human behavior changes, e.g., ridership, of a new urban transit plan before its deployment. I validate our proposed framework using a unique real-world dataset (from Shenzhen, China) with three subway lines opened during the data timespan. With the data collected from both before and after the transit plan deployments, Our evaluation results demonstrated that the proposed framework can predict the ridership with only 19.8% relative error, which is 23%-51% lower than other baseline approaches. 2) Altering Human Decision-Making Process via Reward Advancement. Many real world human behaviors can be characterized as sequential decision making processes, such as urban travelers' choices of transport modes and routes。 Differing from choices controlled by machines, which in general follows perfect rationality to adopt the policy with highest reward, studies have revealed that human agents make sub-optimal decisions under bounded rationality. Such behaviors can be modeled using maximum causal entropy (MCE) principle.In this work, I define and investigate a novel reward transformation problem (namely, reward advancement): Recovering the range of additional reward functions that transform the agent's policy from $\pi_o$ to a predefined target policy under MCE principle. I show that given an MDP and a target policy $\pi_t$, there are infinite many additional reward functions that can achieve the desired policy transformation. Moreover, I propose an algorithm to further extract the additional rewards with minimum ``cost'' to implement the policy transformation. I demonstrated the correctness and accuracy of our reward advancement solution using both synthetic data and a large-scale (6 months) passenger-level public transit data from Shenzhen, China. 3) Using Sequential Models in General Human Prediction Problems.In this work, I aim to develop a joint framework of combining inverse reinforcement learning (IRL) with deep learning (DL) regression model, called IRL-DL, to predict drivers' future behavior in ride-hailing platforms. Specifically, I formulate the dynamic evolution of each driver as a sequential decision-making problem and then employ IRL as representation learning to learn the preference vector of each driver. Then, I integrate drivers' preference vector with their static features (e.g., age, gender) and other attributes to build a regression model (e.g., LTSM-neural network) to predict drivers' future behavior. I use an extensive driver data set obtained from a ride-sharing platform to verify the effectiveness and efficiency of our IRL-DL framework, and results show that our IRL-DL framework can achieve consistent and remarkable improvements over models without drivers' preference vectors. 4) Training Stability in Learning Recurrent Models.In this work, I study the training stability in deep recurrent neural networks (RNNs), and propose a novel network, namely, deep incremental RNN (DIRNN). In contrast to the literature, I prove that DIRNN is essentially a Lyapunov stable dynamical system where there is no vanishing or exploding gradient in training. To demonstrate the applicability in practice, I also propose a novel implementation, namely TinyRNN, that sparsifies the transition matrices in DIRNN using weighted random permutations to reduce the model sizes. 5) Lightweight Convolutional Neural Network via Recurrent Convolution. In this work, we aim to address the problem of learning lightweight networks by proposing a novel CSR-Conv layer that replaces traditional linear convolution with channel-split recurrent convolution. The hidden state transition in the vanilla RNNs leads to deeper networks, given backbones, to compensate for the performance loss while reducing the model sizes. Essentially our CSR-Conv can be viewed as the generalization of linear convolution. We show that the model size of a lightweight network decreases with respect to the number of the duplicate networks.

Creator