An agent perceives its environment through sensors and acts on it through actuators. Define an agent by the mapping from percept histories to actions:
That mapping is the agent's *policy*. Everything else — how it's stored, computed, or learned — is implementation detail.
Environment taxonomy
The properties of the environment determine what kind of agent design works:
- Fully vs partially observable: can the agent see the full state, or only a partial percept (a chess board vs poker hand)?
- Deterministic vs stochastic: does the same action in the same state always produce the same next state?
- Episodic vs sequential: do current decisions affect future ones?
- Static vs dynamic: does the world change while the agent deliberates?
- Discrete vs continuous: are states, actions, or time integer-valued or real-valued?
- Single vs multi-agent: are other rational agents acting in the same environment?
- Known vs unknown: does the agent know the transition and reward model?
Chess: fully observable, deterministic, sequential, static, discrete, multi-agent, known. Self-driving: partially observable, stochastic, sequential, dynamic, continuous, multi-agent, partially known.
Agent architectures
In rough order of sophistication:
- Simple reflex: action depends only on the current percept.
- Model-based reflex: maintains internal state to track unobserved parts of the world.
- Goal-based: searches over future states to find one that satisfies a goal.
- Utility-based: each state has a utility; agent maximizes expected utility over uncertainty.
- Learning agent: improves the above components from experience.
Almost every interesting AI problem maps onto utility-based or learning architectures.