Optimization for Machine Learning (Hardcover)

The interplay between optimization and machine learning is one of the most important developments in modern computational science. Optimization formulations and methods are proving to be vital in designing algorithms to extract essential knowledge from huge volumes of data. Machine learning, however, is not simply a consumer of optimization technology but a rapidly evolving field that is itself generating new optimization ideas. This book captures the state of the art of the interaction between optimization and machine learning in a way that is accessible to researchers in both fields.Optimization approaches have enjoyed prominence in machine learning because of their wide applicability and attractive theoretical properties. The increasing complexity, size, and variety of today's machine learning models call for the reassessment of existing assumptions. This book starts the process of reassessment. It describes the resurgence in novel contexts of established frameworks such as first-order methods, stochastic approximations, convex relaxations, interior-point methods, and proximal methods. It also devotes attention to newer themes such as regularized optimization, robust optimization, gradient and subgradient methods, splitting techniques, and second-order methods. Many of these techniques draw inspiration from other fields, including operations research, theoretical computer science, and subfields of optimization. The book will enrich the ongoing cross-fertilization between the machine learning community and these other fields, and within the broader optimization community.

優化和機器學習之間的相互作用是現代計算科學中最重要的發展之一。優化的形式和方法在設計從大量數據中提取關鍵知識的算法方面被證明是至關重要的。然而，機器學習不僅僅是優化技術的使用者，而且是一個快速發展的領域，它本身正在產生新的優化思想。本書以一種對兩個領域的研究人員都易於理解的方式，捕捉了優化和機器學習之間的交互作用的最新狀態。由於其廣泛的應用性和吸引人的理論性質，優化方法在機器學習中受到重視。如今機器學習模型的複雜性、規模和多樣性不斷增加，需要重新評估現有的假設。本書開始了這個重新評估的過程。它描述了已建立框架（如一階方法、隨機逼近、凸松弛、內點方法和近端方法）在新的背景下的復興。它還關注了新的主題，如正則化優化、魯棒優化、梯度和次梯度方法、分裂技術和二階方法。其中許多技術受到其他領域的啟發，包括運籌學、理論計算機科學和優化的子領域。本書將豐富機器學習社區與這些其他領域以及更廣泛的優化社區之間的交流和互相影響。