How to Learn Optimization

Learn to formulate linear programs, understand the geometry of feasible regions (polytopes), and master the simplex algorithm. Study duality theory, sensitivity analysis, and the economic interpretation of dual variables.

Study unconstrained optimization (gradient descent, Newton's method), constrained optimization (Lagrange multipliers, KKT conditions), and the special properties of convex problems that make them efficiently solvable.

Explore optimization over discrete variables. Study formulation techniques, branch and bound, cutting planes, and classic combinatorial problems like the traveling salesman problem, knapsack, and assignment problems.

Master dynamic programming principles (Bellman's optimality, memoization) and network flow problems (shortest paths, maximum flow, minimum cost flow). Apply these to resource allocation and sequential decision problems.

Study stochastic gradient descent, Adam, and other first-order methods used to train machine learning models. Understand convergence guarantees, regularization as constrained optimization, and hyperparameter tuning.

Learn population-based and trajectory-based metaheuristics: genetic algorithms, simulated annealing, particle swarm optimization, and tabu search. Understand when exact methods are impractical and heuristics are preferred.

Explore multi-objective optimization, stochastic and robust optimization, large-scale optimization with decomposition methods, and real-world case studies in supply chain, engineering design, and finance.