# Sensors and Partially Observable Markov Decision Processes (POMDP)

## Partially observable environments

### Hidden Markov Model (HMM) recap

### Belief states

Inc initial knowledge of environment, page on storing observations

eg if position is right or left, at start belief state is both, if we try to move right, we are definitely right.

what if actions differ between states? can consider only actions available to all states if cost of illegal action is high

goal test: need to test all belief states

## Partially Observable Markov Decision Processes

### Partially Observable Markov Decision Processes (POMDP)

### Policies for POMDPs

### Value functions for POMDPs

max of linear terms?

### Belief Markov Decision Processes

## Identifying policies

### Value iteration for POMDPs

Exponential in complexity for actions, observations?

### Policy iteration for POMDPs

## POMDPs with infinite states

### POMDPs with infinite states