IAI Questions

Later reward probably have higher utility than sooner reward

Discounting reward helps algorithm converge

Discounting factor is between 0 and 1

Exploration - risk of getting negative reward

Markov - action outcomes depend only on the current state

Policy - single plan that can achieve goal states

Exploitation - makes the decision that maximizes the long term sum of rewards

Optimal policy is a consequence of optimal value function

We can solve it in cubic time by standard linear algebra solvers.

Optimal value functions can be decomposed.

They specify the best possible performance over all policies.

Maximum change in utility of state is higher than threshold

Maximum change in utility of state is lower than threshold

After N iterations

Minimum change in utility of state is higher than threshold

It gains efficiency in policy improvement phase.

The optimal policy can converge before the values converge.

Does not calculate and converges value function

Policy is extracted using one-step look ahead.

Each node in search tree is a path in state space graph.

Each state occurs only once in state space graph.

Connected node represent action results.

The initial state is root node of search tree.

There are multiple goal states.

Predecessor function is infeasible to compute.

The max depth of the search is infinite.

Successor function is efficiently computable.

The first goal node opened may be sub-optimal, but finds optimal path if heuristic is admissible and consistent.

It avoids expanding paths that are already expensive.

It reduce space complexity from exponential to quadratic by using a good heuristic.

We can weight the heuristic to find optimal solution faster.

Cost of the optimal solution in the relaxed problem is an admissible heuristic in the original problem.

The removal of restrictions removes edges from state space graph.

Optimal solution in the original problem is also a solution for the relaxed problem.

Relaxed problems have fewer restrictions than original problems.

We can combine admissible heuristics.

We can compare heuristics and choose heuristic with higher branching factor.

We can learn heuristic by identifying which features are predictive of path cost.

We can derive heuristic from the cost of a subproblem.

It interleaves the goal achievement.

It can lead to Sussman anomaly.

It is efficient since it does not maintains goal stack.

It is useful when multiple goals interact, since it increases efficiency by separating goals.

We can build conditional from its reverse.

Posterior is proportional to likelihood time prior.

Likelihood represents the agent's beliefs given the evidence.

Posterior and likelihood are a conditional.