Write a reviewable evaluation contract
Distinguish a target population and estimand from a convenient score. Write primary benefit, guardrail and evidence rules that a reviewer can apply.
Text lesson
MLE PathChecking your access…
Also in Technical membership bundle, Career membership bundle, Interview Accelerator bundle, ML Residency bundle, AI Engineer Interviews learning bundle, Experimentation and A/B testing learning bundle, LLMs A to Z learning bundle, RAG and search learning bundle.
Shared topics and concepts are exploration links, not universal prerequisites.
41 of 41 modules
Distinguish a target population and estimand from a convenient score. Write primary benefit, guardrail and evidence rules that a reviewer can apply.
Text lesson
Choose group/time/selection boundaries and reconstruct decision-time information.
Text lesson
Compute metric denominators, base-rate effects and cost-sensitive operating choices.
Text lesson
Build a reliability table, distinguish calibration from discrimination and validate post-hoc calibration.
Text lesson
Compute explicit ranking measures and diagnose reference, cutoff and judgment mismatches.
Text lesson
Specify the treatment effect, distinguish assignment from receipt, and assess confounding/interference and decision uncertainty.
Text lesson
Calculate recall from a complete or justified positive-population audit and distinguish it from flag precision.
Text lesson
Translate a flag-quality policy to precision and separate the observed constraint from a future guarantee.
Text lesson
Classify fixed versus learned transforms and place fitting inside each training partition.
Text lesson
Trace all score-informed choices and evaluate the selected procedure without reusing final evidence.
Text lesson
Reconstruct historical aggregates using event time, availability time and versioned values.
Text lesson
Compute per-query RR before averaging, include no-hit queries and explain first-hit limitations.
Text lesson
Measure distinct relevant-item coverage with a fixed judgment denominator and explicit aggregation.
Text lesson
Calculate a confidence-frequency gap and distinguish bin-level overconfidence from ranking quality.
Text lesson
Match assignment, observation, estimand and dependence units in a randomized evaluation.
Text lesson
Derive unbiasedness of a difference in means under complete random assignment. Distinguish assignment effects, treatment receipt, observation units and target population.
Text lesson
Derive two-arm Bernoulli variance from independent sample means. Distinguish pooled-null testing from unpooled estimation uncertainty.
Text lesson
Derive power as two shifted-Normal tail probabilities and explain alpha and beta. Derive a per-arm sample-size approximation and identify when it does not apply.
Text lesson
Compute an allocation-specific chi-square SRM screen and explain its assumptions. Trace validity failures across assignment, observation and analysis.
Text lesson
Prove Bonferroni FWER control using the union bound without independence. Apply a prespecified finite-look correction and identify undeclared testing opportunities.
Text lesson
Distinguish ratio of sums from mean of per-unit ratios and handle zero denominators. Derive the covariance-preserving delta-method variance at the independent-unit level.
Text lesson
Distinguish direct, spillover and persistent-policy effects under interference. Derive an equal-cluster correlation variance factor and state its limits.
Text lesson
Calculate DiD, Wald and cutoff contrasts while stating assumptions and local target populations.
Text lesson
Calculate how an explicit departure from parallel trends changes an additive DiD conclusion. Define a counterfactual trend difference b, justify its scale and plausible range, and distinguish sensitivity from sampling confidence.
Text lesson