Sensors and Partially Observable Markov Decision Processes (POMDP)

Partially observable environments

Hidden Markov Model (HMM) recap

Belief states

Inc initial knowledge of environment, page on storing observations

eg if position is right or left, at start belief state is both, if we try to move right, we are definitely right.

what if actions differ between states? can consider only actions available to all states if cost of illegal action is high

goal test: need to test all belief states

Partially Observable Markov Decision Processes

Partially Observable Markov Decision Processes (POMDP)

Policies for POMDPs

Value functions for POMDPs

max of linear terms?

Belief Markov Decision Processes

Identifying policies

Value iteration for POMDPs

Exponential in complexity for actions, observations?

Policy iteration for POMDPs

POMDPs with infinite states

POMDPs with infinite states