Adversarial Machine Learning by Aneesh Sreevallabh Chivukula & Xinghao Yang & Bo Liu & Wei Liu & Wanlei Zhou

Author:Aneesh Sreevallabh Chivukula & Xinghao Yang & Bo Liu & Wei Liu & Wanlei Zhou
Language: eng
Format: epub
ISBN: 9783030997724
Publisher: Springer International Publishing

5.3 Adversarial Reinforcement Learning

Reinforcement machine learning is the study of intelligent agents and their actions in a simulated environment such that a notion of cumulative reward is maximized in the interactions between the agent and the environment. Instead of input/output labels required in supervised machine learning, reinforcement learning’s focus is to find a balance between exploration and exploitation of patterns. Reinforcement learning can be interpreted as sampling-based methods to solve optimal control problems. The goal of reinforcement learning is to learn a policy that maximizes the expected cumulative reward and minimizes long-term regret. An intelligent agent in reinforcement learning has to randomly select actions without reference to an estimated probability distribution. Associative reinforcement learning tasks combine supervised learning with reinforcement learning. In game theoretical modelling, reinforcement learning can be used to produce error estimates on the optimization with reference to bounded rationality.

Chen et al. [118] review adversarial attacks taxonomy on reinforcement learning. The adversarial examples are classified into implicit adversarial examples that add imperceptible adversarial manipulations to mislead the learner and dominant adversarial examples which add physical world perturbations to change the local information available to reinforcement learning. The adversarial attack scenarios are classified into misclassification attacks to target a neural network performing reinforcement learning and Targeted attacks to target a particular class label in training that is misclassified into the target class label selected by the adversary. The learning model trained according to reinforcement learning policies is called the target agent. Q-Learning is a popular training algorithm for reinforcement learning. It proposed updates to a Q-value representing cumulative reward of the target agent. Through an iterative learning process, the target agent maximizes the Q-value by finding a best path to the goal. It can be represented by utility functions that evaluate strength and weakness of actions in a particular state. Deep Q-Network is a deep learning enhancement to Q-Learning. It gives rise to deep reinforcement learning with deep learning network’s loss functions defining the Q-value utilities. The (asynchronous advantage actor-critic) A3C algorithm utilizes the actor-critic framework to improve the training process in deep reinforcement learning. Trust Region Policy Optimization (TRPO) is able to control the changes in reinforcement learning policies from an information-theoretic KL divergence the old and the new policies. The subsequent literature review by Chen et al. [118] shows that the fast gradient sign method (FGSM) can be adapted to reinforcement learning systems and adversarial examples can be crafted for Q-learning paths from the gradient of the maximum Q-value for each point on the path. A policy induction attack is summarized for Deep Q-Networks. Adversarial defense mechanisms are proposed due to adversarial training variants and learning objective regularizations in the adversarial loss functions for deep reinforcement learning. In such attack settings, complete blackbox threat models are quite rare. Variations of adversarial training and regularization terms in the objective function, modifying network structure such as defensive distillation, and deep generative modelling that produces adversarial examples are the most common defense mechanisms. Application domains for such adversarial machine learning

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.