Finding Ghosts in Your Data: Anomaly Detection Techniques with Examples in Python (Paperback)

Discover key information buried in the noise of data by learning a variety of anomaly detection techniques and using the Python programming language to build a robust service for anomaly detection against a variety of data types. The book starts with an overview of what anomalies and outliers are and uses the Gestalt school of psychology to explain just why it is that humans are naturally great at detecting anomalies. From there, you will move into technical definitions of anomalies, moving beyond I know it when I see it to defining things in a way that computers can understand.
The core of the book involves building a robust, deployable anomaly detection service in Python. You will start with a simple anomaly detection service, which will expand over the course of the book to include a variety of valuable anomaly detection techniques, covering descriptive statistics, clustering, and time series scenarios. Finally, you will compare your anomaly detection service head-to-head with a publicly available cloud offering and see how they perform.
The anomaly detection techniques and examples in this book combine psychology, statistics, mathematics, and Python programming in a way that is easily accessible to software developers. They give you an understanding of what anomalies are and why you are naturally a gifted anomaly detector. Then, they help you to translate your human techniques into algorithms that can be used to program computers to automate the process. You'll develop your own anomaly detection service, extend it using a variety of techniques such as including clustering techniques for multivariate analysis and time series techniques for observing data over time, and compare your service head-on against a commercial service.

What You Will Learn

• Understand the intuition behind anomalies
• Convert your intuition into technical descriptions of anomalous data
• Detect anomalies using statistical tools, such as distributions, variance and standard deviation, robust statistics, and interquartile range
• Apply state-of-the-art anomaly detection techniques in the realms of clustering and time series analysis
• Work with common Python packages for outlier detection and time series analysis, such as scikit-learn, PyOD, and tslearn
• Develop a project from the ground up which finds anomalies in data, starting with simple arrays of numeric data and expanding to include multivariate inputs and even time series data

Who This Book Is For

For software developers with at least some familiarity with the Python programming language, and who would like to understand the science and some of the statistics behind anomaly detection techniques. Readers are not required to have any formal knowledge of statistics as the book introduces relevant concepts along the way.

透過學習各種異常檢測技術並使用Python程式語言，您可以發現數據中被淹沒在噪音中的關鍵信息，並建立一個強大的異常檢測服務，以應對各種數據類型。本書首先概述了異常和離群值的定義，並使用格式塔心理學來解釋為什麼人類天生擅長檢測異常。從那裡，您將進入異常的技術定義，超越了「我看到就知道」的程度，以一種電腦可以理解的方式來定義事物。

本書的核心內容是使用Python建立一個強大且可部署的異常檢測服務。您將從一個簡單的異常檢測服務開始，隨著書的進展，逐步擴展到包括各種有價值的異常檢測技術，包括描述統計、聚類和時間序列場景。最後，您將將您的異常檢測服務與一個公開可用的雲服務進行對比，並觀察它們的表現。

本書中的異常檢測技術和示例結合了心理學、統計學、數學和Python編程，使軟件開發人員能夠輕鬆理解。它們讓您了解異常是什麼，以及為什麼您天生就是一個有天賦的異常檢測者。然後，它們幫助您將您的人類技巧轉化為可以用於編程計算機自動化處理的算法。您將開發自己的異常檢測服務，使用各種技術擴展它，例如包括聚類技術進行多變量分析和時間序列技術觀察隨時間變化的數據，並將您的服務與商業服務進行直接對比。

您將學到什麼
- 理解異常的直覺
- 將您的直覺轉化為異常數據的技術描述
- 使用統計工具檢測異常，例如分佈、變異數和標準差、魯棒統計和四分位距
- 在聚類和時間序列分析領域應用最先進的異常檢測技術
- 使用常見的Python套件進行異常值檢測和時間序列分析，例如scikit-learn、PyOD和tslearn
- 從頭開始開發一個項目，該項目可以在數據中找到異常，從簡單的數值數組開始，擴展到包括多變量輸入甚至時間序列數據

本書適合對象
本書適合具有至少一定程度的Python編程語言熟悉度的軟件開發人員，並希望了解異常檢測技術背後的科學和一些統計知識。讀者不需要具備任何統計學的正式知識，因為本書會在適當的時候介紹相關概念。

​Kevin Feasel is a Microsoft Data Platform MVP and CTO at Faregame Inc, where he specializes in data analytics with T-SQL and R, forcing Spark clusters to do his bidding, fighting with Kafka, and pulling rabbits out of hats on demand. He is the lead contributor to Curated SQL, president of the Triangle Area SQL Server Users Group, and author of PolyBase Revealed. A resident of Durham, North Carolina, he can be found cycling the trails along the triangle whenever the weather is nice enough.

Kevin Feasel是一位微軟數據平台MVP，也是Faregame Inc的首席技術官。他專注於使用T-SQL和R進行數據分析，並擅長控制Spark集群、使用Kafka進行數據處理，以及根據需求隨時展現驚人的能力。他是Curated SQL的主要貢獻者，也是Triangle Area SQL Server Users Group的主席，並且是PolyBase Revealed的作者。他居住在北卡羅來納州的杜倫市，在天氣好的時候，你可以在三角地區的自行車道上看到他騎著自行車。