公安大數據應用基礎（第2版）

大數據理論模塊
第 1 章 大數據與人工智能技術理論 ····························································································2
1.1 大數據的基本概念 ············································································································2
1.1.1 大數據的定義·············································································································2
1.1.2 大數據的本質·············································································································2
1.1.3 大數據的分類·············································································································4
1.1.4 大數據的特征·············································································································5
1.1.5 大數據的功能·············································································································6
1.1.6 大數據處理流程 ·········································································································6
1.1.7 大數據處理範式 ·········································································································7
1.2 大數據的技術演進 ············································································································7
1.2.1 大數據的發展現狀······································································································7
1.2.2 大數據的發展趨勢······································································································8
1.2.3 人工智能與大模型······································································································9
1.3 大數據的應用··················································································································10
1.3.1 企業內部大數據 ·······································································································10
1.3.2 在線社交網絡大數據·································································································11
1.3.3 健康醫療大數據 ·······································································································11
1.3.4 金融大數據··············································································································12
1.3.5 公安大數據··············································································································12
1.4 常用的數據挖掘工具 ······································································································13
1.4.1 Tableau····················································································································13
1.4.2 Excel ·······················································································································14
1.4.3 SPSS Modeler ···········································································································14
1.4.4 Python ·····················································································································14
數據分析與挖掘模塊
第 2 章 SPSS Modeler 軟件·······································································································16
2.1 SPSS Modeler 軟件概述·································································································16
2.1.1 SPSS Modeler 界面····································································································16
2.1.2 數據流的基本管理和執行··························································································18
2.1.3 數據流的其他管理····································································································20
2.1.4 SPSS Modeler 應用案例·····························································································22
2.2 SPSS Modeler 數據的讀入······························································································27
2.2.1 變量的類型··············································································································27
2.2.2 讀數據·····················································································································28
2.2.3 生成實驗方案數據····································································································34
2.2.4 數據合並 ·················································································································36
2.3 SPSS Modeler 數據的基本分析 ······················································································40
2.3.1 數據質量 ·················································································································40
2.3.2 基本描述分析···········································································································46
2.3.3 變量分布探索···········································································································49
2.3.4 二分類型相關性研究·································································································51
2.3.5 兩總體的平均值比較·································································································58
2.3.6 變量的重要性分析····································································································64
第 3 章 數據清洗··························································································································69
3.1 數據清洗概述··················································································································69
3.1.1 數據清洗的概念 ·······································································································69
3.1.2 數據清洗的對象 ·······································································································69
3.1.3 數據清洗的一般步驟·································································································70
3.1.4 數據清洗的常用方式·································································································71
3.1.5 數據清洗的基本方法·································································································71
3.2 Excel 數據清洗的基本操作 ····························································································72
3.2.1 重復值的處理···········································································································72
3.2.2 缺失值及異常值的處理 ·····························································································76
3.3 Excel 數據加工的基本操作 ····························································································80
3.3.1 字段分列 ·················································································································80
3.3.2 字段合並 ·················································································································81
3.3.3 字段匹配 ·················································································································82
3.3.4 數據分組 ·················································································································83
3.4 Excel 數據透視表············································································································83
3.4.1 數據透視表應用 ·······································································································83
3.4.2 數據透視表的實用技巧 ·····························································································87
第 4 章 時間序列分析··················································································································91
4.1 時間序列··························································································································91
4.1.1 時間序列概述···········································································································91
4.1.2 時間序列的預測步驟·································································································92
4.2 移動平均法······················································································································92
4.2.1 一次移動平均法 ·······································································································93
4.2.2 二次移動平均法 ·······································································································96
4.3 指數平滑法······················································································································98
4.3.1 一次指數平滑法 ·······································································································99
4.3.2 二次指數平滑法 ·····································································································100
4.3.3 三次指數平滑法 ·····································································································106
第 5 章 分類預測：決策樹········································································································112
5.1 決策樹概述····················································································································112
5.1.1 什麼是決策樹·········································································································112
5.1.2 決策樹的幾何理解··································································································113
5.1.3 決策樹的核心問題··································································································113
5.2 SPSS Modeler 中的 C5.0 算法及應用···········································································115
5.2.1 C5.0 決策樹的分割點······························································································115
5.2.2 C5.0 決策樹的剪枝過程···························································································116
5.2.3 C5.0 決策樹的推理規則集 ·······················································································117
5.2.4 C5.0 決策樹的應用 ·································································································118
5.3 SPSS Modeler 中的 C＆RT 算法及應用········································································125
5.3.1 C＆RT 的生長過程··································································································126
5.3.2 C＆RT 的剪枝過程··································································································127
5.3.3 C＆RT 的應用·········································································································129
5.4 SPSS Modeler 中的 CHAID 算法及應用······································································131
5.4.1 CHAID 算法的最佳分組變量 ···················································································132
5.4.2 CHAID 算法的剪枝過程··························································································132
5.4.3 Exhaustive CHAID 算法···························································································133
5.4.4 CHAID 算法的應用·································································································133
5.5 SPSS Modeler 中的 QUEST 算法及應用······································································134
5.5.1 QUEST 算法的最佳分割點 ······················································································134
5.5.2 QUEST 算法的應用 ································································································135
5.6 決策樹算法的評估和註意事項·····················································································136
實驗 運用 4 種決策樹算法預測數據··················································································137
第 6 章 分類預測：人工神經網絡 ····························································································145
6.1 人工神經網絡概述 ········································································································145
6.1.1 人工神經網絡的概念和種類·····················································································145
6.1.2 人工神經網絡中的節點 ···························································································147
6.1.3 建立人工神經網絡的一般步驟 ·················································································149
6.2 SPSS Modeler 中的 B-P 反向傳播網絡 ········································································151
6.2.1 感知器模型············································································································151
6.2.2 B-P 反向傳播網絡···································································································154
6.2.3 B-P 反向傳播算法···································································································156
6.2.4 B-P 反向傳播網絡的建立·························································································158
6.3 SPSS Modeler 中的徑向基函數網絡 ············································································161
6.3.1 徑向基函數網絡 ·····································································································161
6.3.2 徑向基函數網絡中的隱藏層節點和輸出節點 ·····························································162
6.3.3 徑向基函數網絡的學習過程·····················································································163
6.4 人工神經網絡的應用 ····································································································164
第 7 章 分類預測：Logistic 回歸分析······················································································178
7.1 二項 Logistic 回歸方程 ·································································································178
7.1.1 二項 Logistic 回歸方程概述 ·····················································································178
7.1.2 二項 Logistic 回歸方程中系數的含義········································································180
7.2 二項 Logistic 回歸分析的應用 ·····················································································182
7.3 多項 Logistic 回歸分析的應用 ·····················································································186
第 8 章 關聯分析························································································································187
8.1 簡單關聯規則分析 ········································································································187
8.1.1 簡單關聯規則的基本概念························································································188
8.1.2 簡單關聯規則的有效性和實用性··············································································189
8.2 Apriori 算法···················································································································192
8.2.1 尋找頻繁項集·········································································································192
8.2.2 依據頻繁項集產生簡單關聯規則··············································································194
8.3 Apriori 算法的應用 ·······································································································195
8.4 序列關聯規則分析 ········································································································202
8.4.1 序列關聯規則的基本概念························································································202
8.4.2 序列關聯規則的時間約束························································································203
8.5 Sequence 算法················································································································204
8.5.1 產生頻繁序列集 ·····································································································204
8.5.2 依據頻繁序列集生成序列關聯規則 ··········································································205
8.6 Sequence 算法的應用····································································································206
第 9 章 聚類分析························································································································210
9.1 聚類分析概述················································································································210
9.2 K-Means 聚類算法及應用·····························································································211
9.2.1 K-Means 聚類算法對“親疏程度”的衡量·································································211
9.2.2 K-Means 聚類過程··································································································211
9.2.3 K-Means 聚類算法的應用························································································213
9.3 兩步聚類算法及應用 ····································································································221
9.3.1 兩步聚類算法對“親疏程度”的衡量 ·······································································221
9.3.2 兩步聚類過程·········································································································222
9.3.3 兩步聚類算法的應用·······························································································224
9.4 Kohonen 網絡聚類算法及應用·····················································································226
9.4.1 Kohonen 網絡聚類算法的原理 ·················································································226
9.4.2 Kohonen 網絡聚類過程 ···························································································227
9.4.3 Kohonen 網絡聚類算法的應用 ·················································································229
9.5 基於聚類分析的離群值探索及應用·············································································232
9.5.1 多維空間基於聚類的診斷方法 ·················································································232
9.5.2 多維空間基於聚類的診斷方法的應用 ·······································································234
數據可視化模塊
第10章 數據可視化 ·················································································································240
10.1 數據可視化入門··········································································································240
10.1.1 i2 Analyst’s Notebook 8 軟件···················································································240
10.1.2 基本概念··············································································································241
10.1.3 數據接口··············································································································245
10.2 基本功能······················································································································245
10.2.1 基本操作··············································································································245
10.2.2 搜索查找··············································································································247
10.3 功能演練······················································································································250
10.3.1 話單關系分析·······································································································250
10.3.2 人員物品動態關系 ································································································257
10.3.3 銀行賬戶交易分析 ································································································261
10.3.4 話單 ABC 分析 ·····································································································268
10.3.5 盜竊案旅業分析····································································································274
10.3.6 人員活動軌跡·······································································································277