Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning (Operations Research/Computer Science Interfaces Series)

This book introduces to the reader the evolving area of simulation-based optimization, also known as simulation optimization. The book should serve as an accessible introduction to this topic and requires a background only in elementary mathematics. It brings the reader up to date on cutting-edge advances in simulation-optimization methodologies, including dynamic controls, also called Reinforcement Learning (RL) or Approximate Dynamic Programming (ADP), and static optimization techniques, e.g., Simultaneous Perturbation, Nested Partitions, Backtracking Adaptive Search, Response Surfaces, and Meta-Heuristics. Special features of this book include:

Stochastic Control Optimization:

• An Accessible Introduction to Reinforcement Learning Techniques for Solving Markov Decision Processes (MDPs), with Step-by-Step Descriptions of Numerous Algorithms, e.g., Q-Learning, SARSA, R-SMART, Actor-Critics, Q-P-Learning, and Classical Approximate Policy Iteration
• A Detailed Discussion on Dynamic Programing for Solving MDPs and Semi-MDPs (SMDPs), Including Steps for Value Iteration and Policy Iteration
• An Introduction to Function Approximation with Reinforcement Learning
• An In-Depth Treatment of Reinforcement Learning Methods for SMDPs, Average Reward Problems, Finite Horizon Problems, and Two Time Scales
• Computer Programs (available online)
• A Gentle Introduction to Convergence Analysis of Simulation Optimization Methods via Banach Fixed Point Theory and Ordinary Differential Equations (ODEs)

Stochastic Static Optimization:

• A Step-by-Step Description of Stochastic Adaptive Search Algorithms, e.g., Simultaneous Perturbation, Nested Partitions, Backtracking Adaptive Search, Stochastic Ruler, and Meta-Heuristics, e.g., Simulated Annealing, Tabu Search, and Genetic Algorithms 
• A Clear and Simple Introduction to the Methodology of Neural Networks 

The book ends with a chapter on case studies that explain how these methods can be applied in real-world settings; an online repository of computer programs that can be downloaded from a website is also available.

The book was written for students and researchers in the fields of engineering (industrial, electrical, and computer), computer science, operations research, management science, and applied mathematics.  An attractive feature of this book is its accessibility to readers new to this topic.

這本書向讀者介紹了一個不斷發展的領域，即「基於模擬的優化」，也被稱為「模擬優化」。這本書旨在作為這個主題的易於理解的入門指南，只需要基礎的數學背景。它使讀者了解到模擬優化方法的最新進展，包括動態控制，也稱為強化學習（RL）或近似動態規劃（ADP），以及靜態優化技術，例如同時擾動、嵌套分割、回溯自適應搜索、響應曲面和元啟發式算法。本書的特點包括：

隨機控制優化：
- 介紹解決馬爾可夫決策過程（MDPs）的強化學習技術，包括多種算法的逐步描述，例如Q學習、SARSA、R-SMART、Actor-Critics、Q-P-Learning和經典的近似策略迭代。
- 詳細討論解決MDPs和半MDPs（SMDPs）的動態規劃，包括價值迭代和策略迭代的步驟。
- 介紹使用強化學習的函數逼近。
- 深入探討解決SMDPs、平均報酬問題、有限時間範圍問題和兩個時間尺度的強化學習方法。
- 電腦程式（可在線上獲得）。
- 通過Banach不動點理論和常微分方程的收斂分析，對模擬優化方法的收斂性進行了詳細介紹。

隨機靜態優化：
- 逐步描述了隨機自適應搜索算法，例如同時擾動、嵌套分割、回溯自適應搜索、隨機尺規和元啟發式算法，例如模擬退火、禁忌搜索和遺傳算法。
- 對神經網絡方法的方法論進行了清晰而簡單的介紹。

本書以一章關於案例研究結束，解釋了這些方法如何應用於實際情境；同時提供了一個可以從網站上下載的電腦程式的在線存儲庫。

這本書是為工程（工業、電氣和計算機）、計算機科學、運籌學、管理科學和應用數學領域的學生和研究人員而寫的。這本書的一個吸引人之處是它對於新手讀者的易理解性。