This textbook is the second edition of the linear algebra and optimization book that was published in 2020. The exposition in this edition is greatly simplified as compared to the first edition. The second edition is enhanced with a large number of solved examples and exercises. A frequent challenge faced by beginners in machine learning is the extensive background required in linear algebra and optimization. One problem is that the existing linear algebra and optimization courses are not specific to machine learning; therefore, one would typically have to complete more course material than is necessary to pick up machine learning. Furthermore, certain types of ideas and tricks from optimization and linear algebra recur more frequently in machine learning than other application-centric settings. Therefore, there is significant value in developing a view of linear algebra and optimization that is better suited to the specific perspective of machine learning. It is common for machine learning practitioners to pick up missing bits and pieces of linear algebra and optimization via "osmosis" while studying the solutions to machine learning applications. However, this type of unsystematic approach is unsatisfying because the primary focus on machine learning gets in the way of learning linear algebra and optimization in a generalizable way across new situations and applications. Therefore, we have inverted the focus in this book, with linear algebra/optimization as the primary topics of interest, and solutions to machine learning problems as the applications of this machinery. In other words, the book goes out of its way to teach linear algebra and optimization with machine learning examples. By using this approach, the book focuses on those aspects of linear algebra and optimization that are more relevant to machine learning, and also teaches the reader how to apply them in the machine learning context. As a side benefit, the reader will pick up knowledge of several fundamental problems in machine learning. At the end of the process, the reader will become familiar with many of the basic linear-algebra- and optimization-centric algorithms in machine learning. Although the book is not intended to provide exhaustive coverage of machine learning, it serves as a "technical starter" for the key models and optimization methods in machine learning. Even for seasoned practitioners of machine learning, a systematic introduction to fundamental linear algebra and optimization methodologies can be useful in terms of providing a fresh perspective. The chapters of the book are organized as follows. 1-Linear algebra and its applications: The chapters focus on the basics of linear algebra together with their common applications to singular value decomposition, matrix factorization, similarity matrices (kernel methods), and graph analysis. Numerous machine learning applications have been used as examples, such as spectral clustering, kernel-based classification, and outlier detection. The tight integration of linear algebra methods with examples from machine learning differentiates this book from generic volumes on linear algebra. The focus is clearly on the most relevant aspects of linear algebra for machine learning and to teach readers how to apply these concepts. 2-Optimization and its applications: Much of machine learning is posed as an optimization problem in which we try to maximize the accuracy of regression and classification models. The "parent problem" of optimization-centric machine learning is least-squares regression. Interestingly, this problem arises in both linear algebra and optimization and is one of the key connecting problems of the two fields. Least-squares regression is also the starting point for support vector machines, logistic regression, and recommender systems. Furthermore, the methods for dimensionality reduction and matrix factorization also require the development of optimization methods. A general view of optimization in computational graphs is discussed together with its applications to backpropagation in neural networks. The primary audience for this textbook is graduate level students and professors. The secondary audience is industry. Advanced undergraduates might also be interested, and it is possible to use this book for the mathematics requirements of an undergraduate data science course.
這本教科書是2020年出版的線性代數與優化書籍的第二版。與第一版相比,這一版的內容大幅簡化。第二版增強了大量已解決的範例和練習題。機器學習初學者常面臨的一個挑戰是需要廣泛的線性代數和優化背景。一個問題是,現有的線性代數和優化課程並不專門針對機器學習;因此,通常需要完成比學習機器學習所需的更多課程內容。此外,某些來自優化和線性代數的概念和技巧在機器學習中比其他以應用為中心的環境中更頻繁地出現。因此,發展一種更適合機器學習特定視角的線性代數和優化觀點具有重要價值。
機器學習從業者通常會在研究機器學習應用的解決方案時,通過「滲透」的方式學習缺失的線性代數和優化知識。然而,這種不系統的方法令人不滿,因為對機器學習的主要關注妨礙了在新情境和應用中以可概括的方式學習線性代數和優化。因此,我們在這本書中顛倒了重點,將線性代數/優化作為主要的興趣主題,而將機器學習問題的解決方案視為這些工具的應用。換句話說,這本書特意以機器學習範例來教授線性代數和優化。通過這種方法,書中專注於與機器學習更相關的線性代數和優化方面,並教導讀者如何在機器學習的背景下應用這些概念。作為附帶好處,讀者將獲得有關機器學習中幾個基本問題的知識。在這個過程結束時,讀者將熟悉許多以線性代數和優化為中心的機器學習算法。雖然這本書並不打算提供機器學習的全面覆蓋,但它作為機器學習中關鍵模型和優化方法的「技術入門」是有用的。即使對於經驗豐富的機器學習從業者,對基本線性代數和優化方法的系統介紹也能提供新的視角。
本書的章節組織如下。
1-線性代數及其應用: 這些章節專注於線性代數的基本概念及其在奇異值分解、矩陣分解、相似度矩陣(核方法)和圖分析中的常見應用。許多機器學習應用被用作範例,例如光譜聚類、基於核的分類和異常檢測。線性代數方法與機器學習範例的緊密結合使這本書與一般的線性代數書籍區別開來。重點顯然是機器學習中最相關的線性代數方面,並教導讀者如何應用這些概念。
2-優化及其應用: 機器學習的許多問題被表述為優化問題,我們試圖最大化回歸和分類模型的準確性。以優化為中心的機器學習的「母問題」是最小二乘回歸。有趣的是,這個問題在線性代數和優化中都會出現,並且是這兩個領域的關鍵連接問題之一。最小二乘回歸也是支持向量機、邏輯回歸和推薦系統的起點。此外,降維和矩陣分解的方法也需要開發優化方法。書中討論了計算圖中優化的一般觀點及其在神經網絡中的反向傳播應用。
本教科書的主要讀者是研究生和教授。次要讀者是業界。高年級本科生也可能感興趣,並且可以將這本書用於本科數據科學課程的數學要求。
以上中文內容由 AI 翻譯