MIT press provides another excellent book in creative commons.
Algorithms for decision making: free download book
I plan to buy it and I recommend you do. This book provides a broad introduction to algorithms for decision making under uncertainty.
The book takes an agent based approach
An agent is an entity that acts based on observations of its environment. Agents
may be physical entities, like humans or robots, or they may be nonphysical entities,
such as decision support systems that are implemented entirely in software.
The interaction between the agent and the environment follows an observe-act cycle or loop.
- The agent at time t receives an observation of the environment
- Observations are often incomplete or noisy;
- Based in the inputs, the agent then chooses an action at through some decision process.
- This action, such as sounding an alert, may have a nondeterministic effect on the environment.
- The book focusses on agents that interact intelligently to achieve their objectives over time.
- Given the past sequence of observations and knowledge about the environment, the agent must choose an action at that best achieves its objectives in the presence of various sources of uncertainty including:
- outcome uncertainty, where the effects of our actions are uncertain,
- model uncertainty, where our model of the problem is uncertain,
3. state uncertainty, where the true state of the environment is uncertain, and - interaction uncertainty, where the behavior of the other agents interacting in the environment is uncertain.
The book is organized around these four sources of uncertainty.
Making decisions in the presence of uncertainty is central to the field of artificial intelligence
Table of contents is
Introduction
Decision Making
Applications
Methods
History
Societal Impact
Overview
PROBABILISTIC REASONING
Representation
Degrees of Belief and Probability
Probability Distributions
Joint Distributions
Conditional Distributions
Bayesian Networks
Conditional Independence
Summary
Exercises
viii contents
Inference
Inference in Bayesian Networks
Inference in Naive Bayes Models
Sum-Product Variable Elimination
Belief Propagation
Computational Complexity
Direct Sampling
Likelihood Weighted Sampling
Gibbs Sampling
Inference in Gaussian Models
Summary
Exercises
Parameter Learning
Maximum Likelihood Parameter Learning
Bayesian Parameter Learning
Nonparametric Learning
Learning with Missing Data
Summary
Exercises
Structure Learning
Bayesian Network Scoring
Directed Graph Search
Markov Equivalence Classes
Partially Directed Graph Search
Summary
Exercises
Simple Decisions
Constraints on Rational Preferences
Utility Functions
Utility Elicitation
Maximum Expected Utility Principle
Decision Networks
Value of Information
Irrationality
Summary
Exercises
SEQUENTIAL PROBLEMS
Exact Solution Methods
Markov Decision Processes
Policy Evaluation
Value Function Policies
Policy Iteration
Value Iteration
Asynchronous Value Iteration
Linear Program Formulation
Linear Systems with Quadratic Reward
Summary
Exercises
Approximate Value Functions
Parametric Representations
Nearest Neighbor
Kernel Smoothing
Linear Interpolation
Simplex Interpolation
Linear Regression
Neural Network Regression
Summary
Exercises
Online Planning
Receding Horizon Planning
Lookahead with Rollouts
Forward Search
Branch and Bound
Sparse Sampling
Monte Carlo Tree Search
Heuristic Search
Labeled Heuristic Search
Open-Loop Planning
Summary
Exercises
Policy Search
Approximate Policy Evaluation
Local Search
Genetic Algorithms
Cross Entropy Method
Evolution Strategies
Isotropic Evolutionary Strategies
Summary
Exercises
Policy Gradient Estimation
Finite Difference
Regression Gradient
Likelihood Ratio
Reward-to-Go
Baseline Subtraction
Summary
Exercises
Policy Gradient Optimization
Gradient Ascent Update
Restricted Gradient Update
Natural Gradient Update
Trust Region Update
Clamped Surrogate Objective
Summary
Exercises
Actor-Critic Methods
Actor-Critic
Generalized Advantage Estimation
Deterministic Policy Gradient
Actor-Critic with Monte Carlo Tree Search
Summary
Policy Validation
Performance Metric Evaluation
Rare Event Simulation
Robustness Analysis
Trade Analysis
Adversarial Analysis
Summary
Exercises
MODEL UNCERTAINTY
Exploration and Exploitation
Bandit Problems
Bayesian Model Estimation
Undirected Exploration Strategies
Directed Exploration Strategies
Optimal Exploration Strategies
Exploration with Multiple States
Summary
Exercises
Model-Based Methods
Maximum Likelihood Models
Update Schemes
Exploration
Bayesian Methods
Bayes-adaptive MDPs
Posterior Sampling
Summary
Exercises
Model-Free Methods
Incremental Estimation of the Mean
Q-Learning
Sarsa
Eligibility Traces
Reward Shaping
Action Value Function Approximation
Experience Replay
Summary
Exercises
Imitation Learning
Behavioral Cloning
Dataset Aggregation
Stochastic Mixing Iterative Learning
Maximum Margin Inverse Reinforcement Learning
Maximum Entropy Inverse Reinforcement Learning
Generative Adversarial Imitation Learning
Summary
Exercises
PART IV STATE UNCERTAINTY
19 Beliefs 373
Belief Initialization
Discrete State Filter
Linear Gaussian Filter
Extended Kalman Filter
Unscented Kalman Filter
Particle Filter
Particle Injection
Summary
Exercises
20 Exact Belief State Planning 399
Belief-State Markov Decision Processes
Conditional Plans
Alpha Vectors
Pruning
Value Iteration
Linear Policies
Summary
Exercises
Offline Belief State Planning
Fully Observable Value Approximation
Fast Informed Bound
Fast Lower Bounds
Point-Based Value Iteration
Randomized Point-Based Value Iteration
Sawtooth Upper Bound
Point Selection
Sawtooth Heuristic Search
Triangulated Value Functions
Summary
Exercises
Online Belief State Planning
Lookahead with Rollouts
Forward Search
Branch and Bound
Sparse Sampling
Monte Carlo Tree Search
Determinized Sparse Tree Search
Gap Heuristic Search
Summary
Exercises
Controller Abstractions
Controllers
Policy Iteration
Nonlinear Programming
Gradient Ascent
Summary
Exercises
PART V MULTIAGENT SYSTEMS
Multiagent Reasoning
Simple Games
Response Models
Dominant Strategy Equilibrium
Nash Equilibrium
Correlated Equilibrium
Iterated Best Response
Hierarchical Softmax
Fictitious Play
Gradient Ascent
Summary
Exercises
Sequential Problems
Markov Games
Response Models
Nash Equilibrium
Fictitious Play
Gradient Ascent
Nash Q-Learning
Summary
Exercises
State Uncertainty
Partially Observable Markov Games
Policy Evaluation
Nash Equilibrium
Dynamic Programming
Summary
Exercises
Collaborative Agents
Decentralized Partially Observable Markov Decision Processes
Subclasses
Dynamic Programming
Iterated Best Response
Heuristic Search
Nonlinear Programming
Summary
Exercises
APPENDICES
Mathematical Concepts
Measure Spaces
Probability Spaces
Metric Spaces
Normed Vector Spaces
Positive Definiteness
Convexity
Information Content
Entropy
Cross Entropy
Relative Entropy
Gradient Ascent
Taylor Expansion
Monte Carlo Estimation
Importance Sampling
Contraction Mappings
Graphs
Probability Distributions
Computational Complexity
Asymptotic Notation
Time Complexity Classes
Space Complexity Classes
Decideability
Neural Representations
Neural Networks
Feedforward Networks
Parameter Regularization
Convolutional Neural Networks
Recurrent Networks
Autoencoder Networks
Adversarial Networks
Search Algorithms
Search Problems
Search Graphs
Forward Search
Branch and Bound
Dynamic Programming
Heuristic Search
Problems
Hex World
2048
Cart-Pole
Mountain Car
Simple Regulator
Aircraft Collision Avoidance
Crying Baby
Machine Replacement
Catch
F.10 Prisoners Dilemma
Rock-Paper-Scissors
Travelers Dilemma
Predator-Prey Hex World
Multi-Caregiver Crying Baby
Collaborative Predator-Prey Hex World
Julia
Types
Functions
Control Flow
Packages
Convenience Functions
Book link
