Trace actions, rewards and returns
Calculate a finite discounted return and distinguish it from a value estimate. Explain why an action changes both outcomes and future learning data.
Text lesson
MLE PathChecking your access…
Also in Technical membership bundle, Career membership bundle, Reinforcement learning learning bundle.
Shared topics and concepts are exploration links, not universal prerequisites.
39 of 39 modules
Calculate a finite discounted return and distinguish it from a value estimate. Explain why an action changes both outcomes and future learning data.
Text lesson
Calculate a Monte Carlo target and a TD(0) target from the same run. Identify the information and estimation error each target uses.
Text lesson
Calculate a legal-action target, TD error and update as three separate quantities. Explain why a negative continuation stays negative.
Text lesson
Choose a bootstrap mask from task semantics rather than an ambiguous stop flag. Detect reset-state contamination in a stored transition.
Text lesson
Calculate Q-learning and sampled SARSA targets on the same transition. Explain off-policy learning without inferring unlimited coverage or convergence.
Text lesson
Calculate a clipped contribution for either advantage sign. Distinguish an objective contribution from a constraint on the entire policy.
Text lesson
Compute grouped advantages with an explicit variance convention. Separate old, current and reference policies and token versus sequence weighting.
Text lesson
Compute a one-step advantage estimate and identify its critic error. Evaluate the clipped PPO surrogate for positive and negative advantage.
Text lesson
Trace the deterministic actor gradient through the critic and distinguish TD3 target smoothing. Calculate an entropy-regularized SAC target and explain the temperature tradeoff.
Text lesson
Explain what replay changes in the training data stream and what it cannot create. Check the target-network and gradient boundary in a DQN fitting step.
Text lesson
Calculate a Double DQN target and a mean-centered dueling head. Explain sampling correction and the distinction between expected return and its distribution.
Text lesson
Write and check a transition contract before interpreting a learning curve. Distinguish an environment semantic check from reproducibility or task-success evidence.
Text lesson
Distinguish a bandit decision from a sequential state-changing control task. Calculate epsilon-greedy probabilities, a UCB index and a posterior-sampling decision.
Text lesson
Explain why low demonstration error can become high rollout error. Trace DAgger data collection and state the expert-access and execution assumptions.
Text lesson
Compute a Bellman expectation backup and contrast it with an optimality backup. Diagnose a state description that omits task-relevant history.
Text lesson
Calculate compounding rollout error in a simple model. Choose a bounded validation and rollout strategy without treating model predictions as new ground truth.
Text lesson
Identify unsupported actions and distribution shift in a fixed dataset. Compute a simple importance-weighted estimate and state its support and logging assumptions.
Text lesson
Determine the direction of a score-function policy update from a sampled return. Explain why an action-independent baseline preserves the expected gradient.
Text lesson
Predict and update a finite belief state from an action and observation. Explain why identical current observations can require different decisions after different histories.
Text lesson
Identify a reward shortcut using a separate task-success measure. Calculate potential-based shaping and design a comparison that exposes seed and selection effects.
Text lesson
Trace version-specific cutoff repair and policy/replay observation identity. Predict zero-based optimizer and target events without training.
Text lesson
Adapt true terminals and timeouts without shifting or inventing valid experience. Preserve declared legal discrete action cardinality.
Text lesson
Trace reset and five-field step results through a deterministic custom Env. Separate transition identity, reset boundaries and declared observation representation.
Text lesson
Route final and reset observations under each pinned autoreset mode. Predict per-slot shapes and reset-only rows from supplied actions.
Text lesson