Elements of Dimensionality Reduction and Manifold Learning

Dimensionality reduction, also known as manifold learning, is an area of machine learning used for extracting informative features from data for better representation of data or separation between classes. This book presents a cohesive review of linear and nonlinear dimensionality reduction and manifold learning. Three main aspects of dimensionality reduction are covered: spectral dimensionality reduction, probabilistic dimensionality reduction, and neural network-based dimensionality reduction, which have geometric, probabilistic, and information-theoretic points of view to dimensionality reduction, respectively. The necessary background and preliminaries on linear algebra, optimization, and kernels are also explained to ensure a comprehensive understanding of the algorithms.
The tools introduced in this book can be applied to various applications involving feature extraction, image processing, computer vision, and signal processing. This book is applicable to a wide audience who would like to acquire a deep understanding of the various ways to extract, transform, and understand the structure of data. The intended audiences are academics, students, and industry professionals. Academic researchers and students can use this book as a textbook for machine learning and dimensionality reduction. Data scientists, machine learning scientists, computer vision scientists, and computer scientists can use this book as a reference. It can also be helpful to statisticians in the field of statistical learning and applied mathematicians in the fields of manifolds and subspace analysis. Industry professionals, including applied engineers, data engineers, and engineers in various fields of science dealing with machine learning, can use this as a guidebook for feature extraction from their data, as the raw data in industry often require preprocessing.
The book is grounded in theory but provides thorough explanations and diverse examples to improve the reader's comprehension of the advanced topics. Advanced methods are explained in a step-by-step manner so that readers of all levels can follow the reasoning and come to a deep understanding of the concepts. This book does not assume advanced theoretical background in machine learning and provides necessary background, although an undergraduate-level background in linear algebra and calculus is recommended.

降維，也被稱為流形學習，是機器學習的一個領域，用於從數據中提取有信息的特徵，以更好地表示數據或區分不同類別。本書提供了線性和非線性降維和流形學習的一個有組織的綜述。涵蓋了降維的三個主要方面：譜降維、概率降維和基於神經網絡的降維，分別從幾何、概率和信息論的角度來看待降維。同時，本書還解釋了線性代數、優化和核方法的必要背景和基礎，以確保對算法有全面的理解。

本書介紹的工具可以應用於涉及特徵提取、圖像處理、計算機視覺和信號處理等各種應用。本書適用於廣大讀者，他們希望深入了解從數據中提取、轉換和理解結構的各種方法。目標讀者包括學術界的研究人員、學生和行業專業人士。學術研究人員和學生可以將本書作為機器學習和降維的教材。數據科學家、機器學習科學家、計算機視覺科學家和計算機科學家可以將本書作為參考書。對於統計學家在統計學習領域和應用數學家在流形和子空間分析領域的專業人士也有幫助。行業專業人士，包括應用工程師、數據工程師和從事機器學習相關科學領域的工程師，可以將本書作為從數據中提取特徵的指南，因為工業界的原始數據通常需要預處理。

本書基於理論，但提供了詳盡的解釋和多樣的例子，以提高讀者對高級主題的理解。高級方法以逐步方式解釋，以便各級讀者都能理解推理過程並深入理解概念。本書不假設讀者具有機器學習的高級理論背景，並提供必要的背景知識，儘管建議具有線性代數和微積分的本科水平背景。

Benyamin Ghojogh:

Benyamin Ghojogh received the B.Sc. degree in electrical engineering from the Amirkabir University of Technology, Tehran, Iran, in 2015, the M.Sc. degree in electrical engineering from the Sharif University of Technology, Tehran, Iran, in 2017, and the Ph.D. in electrical and computer engineering (in the area of pattern analysis and machine intelligence) from the University of Waterloo, Waterloo, ON, Canada, in 2021. He was a postdoctoral fellow, focusing on machine learning, at the University of Waterloo, in 2021. His research interests include machine learning, dimensionality reduction, manifold learning, computer vision, data science, and deep learning.

Mark Crowley:

Mark Crowley has a PhD in Computer Science from the University of British Columbia and was a postdoctoral fellow at the Oregon State University. He is now an Associate Professor in the Department of Electrical and Computer Engineering at the University of Waterloo and regularly teaches undergraduate and graduate courses on software programming, artificial intelligence, and data analysis. He is a member of the Waterloo Artificial Intelligence Institute. He carries out research to find dependable and transparent ways to augment human decision making in complex domains, especially in the presence of spatial structure, streaming data, and uncertainty. His research group focuses on developing new algorithms within the fields of reinforcement learning, deep learning, and manifold learning. This often involves collaboration with industry and policy makers in diverse fields such as sustainable forest management, ecology, autonomous driving, physical chemistry, and medical imaging.

Fakhri Karray:

Fakhreddine (Fakhri) Karray is the Loblaws Research Chair in Artificial Intelligence in the department of electrical and computer engineering at the University of Waterloo, Canada. He is the founding co-director of the University of Waterloo AI Institute. He is currently serving as the Provost and Professor of Machine Learning at the Mohamed bin Zayed University of Artificial Intelligence, a first of its kind graduate level, research based artificial intelligence university. Fakhri's research interests are in the areas of advances in machine learning, operational AI, cognitive machines, natural human-machine interaction, autonomous and intelligent systems. Applications of his research include virtual care systems, cognitive and self-aware machines/robots/vehicles, predictive analytics in supply chain management and intelligent transportation systems. Recent work of Fakhri and his research team on deep learning-based driver behavior recognition and prediction has been featured on The Washington Post, Wired Magazine, Globe and Mail, CBC radio and Canada's Discovery Channel. He was honored in 2021 by the IEEE Vehicular Technology Society (VTS) for his novel work on improving traffic flow prediction using weather Information in connected cars through deep learning and tools of AI and received the Society's 2021 Best Land Transportation Paper Award.

Fakhri is the co-author of a textbook on applied artificial intelligence: Soft Computing and Intelligent Systems Design (Pearson Education Publishing, 2004). He has published extensively in the general field of pattern analysis and machine intelligence and is the author of 20 US registered patents. He is the Associate Editor (AE) of flagship journals in the field of AI and intelligent systems, including the IEEE Transactions on Cybernetics, the IEEE Transactions on Neural Networks and Learning Systems and the IEEE Computational Intelligence Magazine. He served as the AE and Guest Editor for the IEEE Transactions on Mechatronics, the IEEE Computational Intelligence Magazine and IEEE Access (special issue on IoMT). He also serves on several editorial boards of AI-related journals and has served as the General Chair/Program Chair for several international conferences in the field of intelligent systems. Fakhri is the co-founder and Chief Scientist of Yourika.ai, a provider of AI based online learning systems. He is a Fellow of the IEEE, a Fellow of the Canadian Academy of Engineering, a Fellow of the Engineering Institute of Canada and a Fellow of the Kavli Frontiers of. He received his PhD from the University of Illinois Urbana-Champaign, USA, and completed his undergraduate engineering degree at the National Engineering School of Tunis, Tunisia.

Ali Ghodsi:

Ali Ghodsi is a Professor of Statistics and Computer Science at the University of Waterloo in Ontario, Canada, and a member of the Waterloo Artificial Intelligence Institute. His current research sweeps across a broad swath of AI encompassing machine learning, deep learning, and dimensionality reduction. He regularly teaches courses on these topics. He studies theoretical frameworks and develops new machine-learning algorithms for analyzing large-scale data sets, with applications in natural language processing, bioinformatics, pattern recognition, computer vision, and sequential decision making. Dr. Ghodsi's work has been published extensively in high-quality proceedings and journals, and he is the co-author of several US patents. His popular lectures on YouTube have more than one million views.

Benyamin Ghojogh:
Benyamin Ghojogh於2015年獲得伊朗德黑蘭阿米爾卡比爾科技大學電機工程學士學位，2017年獲得伊朗德黑蘭夏里夫科技大學電機工程碩士學位，並於2021年獲得加拿大滑鐵盧大學電機與計算機工程博士學位（專攻模式分析和機器智能領域）。他於2021年在滑鐵盧大學擔任機器學習的博士後研究員。他的研究興趣包括機器學習、降維、流形學習、計算機視覺、數據科學和深度學習。

Mark Crowley:
Mark Crowley擁有英屬哥倫比亞大學的計算機科學博士學位，並在俄勒岡州立大學擔任博士後研究員。他現在是滑鐵盧大學電機與計算機工程系的副教授，定期教授軟件編程、人工智能和數據分析的本科和研究生課程。他是滑鐵盧人工智能研究所的成員。他的研究旨在找到可靠和透明的方法來增強複雜領域中人類決策，特別是在空間結構、流動數據和不確定性存在的情況下。他的研究小組致力於在強化學習、深度學習和流形學習領域開發新的算法。這通常涉及與可持續林業管理、生態學、自動駕駛、物理化學和醫學成像等不同領域的行業和政策制定者的合作。

Fakhri Karray:
Fakhreddine（Fakhri）Karray是加拿大滑鐵盧大學電機與計算機工程系的Loblaws人工智能研究主席。他是滑鐵盧大學人工智能研究所的創始聯合主任。他目前擔任穆罕默德·賓·扎耶德人工智能大學的教務長和機器學習教授，這是一所首個以研究為基礎的研究生級人工智能大學。Fakhri的研究興趣涉及機器學習、運營AI、認知機器、自然人機交互、自主和智能系統。他的研究應用包括虛擬護理系統、認知和自我感知的機器/機器人/車輛、供應鏈管理中的預測分析和智能交通系統。Fakhri和他的研究團隊最近在基於深度學習的駕駛行為識別和預測方面的工作被華盛頓郵報、有線雜誌、環球郵報、加拿大廣播公司和加拿大探索頻道報導。他在2021年因他在使用天氣信息通過深度學習和AI工具改善連接車輛中的交通流預測方面的創新工作而獲得IEEE車輛技術學會（VTS）的榮譽，並獲得該學會的2021年最佳陸運論文獎。

Fakhri是應用人工智能教科書《軟計算和智能系統設計》（Pearson Education Publishing，2004年）的合著者。他在模式分析和機器智能的廣泛領域發表了大量論文，並擁有20項美國註冊專利。他是人工智能和智能系統領域的頂級期刊IEEE Transactions on Cybernetics、IEEE Transactions on Neural Networks and Learning Systems和IEEE Computational Intelligence Magazine的副編輯（AE）。他曾擔任IEEE Transactions on Mechatronics、IEEE Computational Intelligence Magazine和IEEE Access（IoMT特刊）的AE和客座編輯。他還擔任多個人工智能相關期刊的編輯委員會成員，並曾擔任多個國際會議的總主席/程序主席。