Mohamad H Danesh

Selected Publications

2022

1. CoRL

   LEADER: Learning Attention Over Driving Behaviors For Planning Under Uncertainty

   Mohamad H Danesh, Panpan Cai, and David Hsu

   In 6th Annual Conference on Robot Learning, 2022

   Abs arXiv Bib PDF Code

   Uncertainty on human behaviors poses a significant challenge to autonomous driving in crowded urban environments. The partially observable Markov decision processes (POMDPs) offer a principled framework for planning under uncertainty, often leveraging Monte Carlo sampling to achieve online performance for complex tasks. However, sampling also raises safety concerns by potentially missing critical events. To address this, we propose a new algorithm, LEarning Attention over Driving bEhavioRs (LEADER), that learns to attend to critical human behaviors during planning. LEADER learns a neural network generator to provide attention over human behaviors in real-time situations. It integrates the attention into a belief-space planner, using importance sampling to bias reasoning towards critical events. To train the algorithm, we let the attention generator and the planner form a min-max game. By solving the min-max game, LEADER learns to perform risk-aware planning without human labeling.

   @inproceedings{danesh2022leader,
     title = {LEADER: Learning Attention Over Driving Behaviors For Planning Under Uncertainty},
     author = {Danesh, Mohamad H and Cai, Panpan and Hsu, David},
     booktitle = {6th Annual Conference on Robot Learning},
     year = {2022},
     organization = {PMLR},
     webpage = {https://sites.google.com/view/leader-paper/home},
     talk = {https://youtu.be/56LzTZfwY2Q?t=2859},
     oral = {true},
     awards = {best paper awards finalist},
   }

2021

1. ICML

   Re-understanding Finite-State Representations of Recurrent Policy Networks

   Mohamad H Danesh, Anurag Koul, Alan Fern, and 1 more author

   In International Conference on Machine Learning, 2021

   Abs arXiv Bib PDF Code Slides

   We introduce an approach for understanding control policies represented as recurrent neural networks. Recent work has approached this problem by transforming such recurrent policy networks into finite-state machines (FSM) and then analyzing the equivalent minimized FSM. While this led to interesting insights, the minimization process can obscure a deeper understanding of a machine’s operation by merging states that are semantically distinct. To address this issue, we introduce an analysis approach that starts with an unminimized FSM and applies more-interpretable reductions that preserve the key decision points of the policy. We also contribute an attention tool to attain a deeper understanding of the role of observations in the decisions. Our case studies on 7 Atari games and 3 control benchmarks demonstrate that the approach can reveal insights that have not been previously noticed.

   @inproceedings{danesh2021re,
     title = {Re-understanding Finite-State Representations of Recurrent Policy Networks},
     author = {Danesh, Mohamad H and Koul, Anurag and Fern, Alan and Khorram, Saeed},
     booktitle = {International Conference on Machine Learning},
     pages = {2388--2397},
     year = {2021},
     organization = {PMLR},
   }

2. ICML UDL

   Out-of-Distribution Dynamics Detection: RL-Relevant Benchmarks and Results

   Mohamad H Danesh, and Alan Fern

   In International Conference on Machine Learning, UDL Workshop, 2021

   Abs arXiv Bib PDF Slides

   We study the problem of out-of-distribution dynamics (OODD) detection, which involves detecting when the dynamics of a temporal process change compared to the training-distribution dynamics. This is relevant to applications in control, reinforcement learning (RL), and multi-variate time-series, where changes to test time dynamics can impact the performance of learning controllers/predictors in unknown ways. This problem is particularly important in the context of deep RL, where learned controllers often overfit to the training environment. Currently, however, there is a lack of established OODD benchmarks for the types of environments commonly used in RL research. Our first contribution is to design a set of OODD benchmarks derived from common RL environments with varying types and intensities of OODD. Our second contribution is to design a strong OODD baseline approach based on recurrent implicit quantile networks (RIQNs), which monitors autoregressive prediction errors for OODD detection. Our final contribution is to evaluate the RIQN approach on the benchmarks to provide baseline results for future comparison.

   @inproceedings{danesh2021out,
     title = {Out-of-Distribution Dynamics Detection: RL-Relevant Benchmarks and Results},
     author = {Danesh, Mohamad H and Fern, Alan},
     booktitle = {International Conference on Machine Learning, UDL Workshop},
     year = {2021},
     organization = {PMLR},
     baseline = {https://github.com/modanesh/recurrent_implicit_quantile_networks},
     benchmark = {https://github.com/modanesh/anomalous_rl_envs},
     oral = {true}
   }

3. Stochastic Block-ADMM for Training Deep Networks

   Saeed Khorram, Xiao Fu, Mohamad H Danesh, and 2 more authors

   arXiv preprint arXiv:2105.00339, 2021

   Abs arXiv Bib PDF

   In this paper, we propose Stochastic Block-ADMM as an approach to train deep neural networks in batch and online settings. Our method works by splitting neural networks into an arbitrary number of blocks and utilizes auxiliary variables to connect these blocks while optimizing with stochastic gradient descent. This allows training deep networks with non-differentiable constraints where conventional backpropagation is not applicable. An application of this is supervised feature disentangling, where our proposed DeepFacto inserts a non-negative matrix factorization (NMF) layer into the network. Since backpropagation only needs to be performed within each block, our approach alleviates vanishing gradients and provides potentials for parallelization. We prove the convergence of our proposed method and justify its capabilities through experiments in supervised and weakly-supervised settings.

   @article{khorram2021stochastic,
     title = {Stochastic Block-ADMM for Training Deep Networks},
     author = {Khorram, Saeed and Fu, Xiao and Danesh, Mohamad H and Qi, Zhongang and Fuxin, Li},
     journal = {arXiv preprint arXiv:2105.00339},
     year = {2021},
   }