人工智能數據素養

數據作為一種新型生產要素，在未來的社會發展過程中將扮演越來越重要的角色，提升數據素養將有助於促進中國人工智能後備人才的高質量發展。本書以人工智能下的大數據時代為背景，從數據素養、數據分析基礎、統計分析、機器學習多個維度全面系統地介紹瞭如何探索數據、整理數據並分析數據。本書沒有給出晦澀難懂的數學公式，也不涉及復雜煩瑣的程序代碼，而是在闡述基本原理的基礎上，輔以簡潔的Python 程序，讓讀者能夠快速入門，提升個人的數據綜合素養。

第1 章 人工智能下的大數據時代 ...................................................................................... 001
1.1 大數據時代和人工智能 ····································································.001
1.1.1 一切皆為數據 ······································································.001
1.1.2 數據高速增長時代 ································································.002
1.1.3 利用人工智能掘金大數據························································.003
1.2 人工智能三要素 ·············································································.004
1.2.1 數據——AI 之源 ···································································.005
1.2.2 算法——AI 之核 ···································································.006
1.2.3 算力——AI 之驅 ···································································.007
1.3 數據素養 ······················································································.007
1.3.1 何為數據素養 ······································································.007
1.3.2 數據素養為何重要 ································································.010
1.3.3 如何提升數據素養 ································································.011
1.4 本章小結 ······················································································.012
第2 章 Python 數據分析基礎 ............................................................................................. 013
2.1 Python 基礎 ··················································································.013
2.1.1 Python 簡介 ·········································································.013
2.1.2 Python 數據類型 ···································································.017
2.1.3 常用的操作、函數和方法························································.021
2.1.4 列表、元組、字典 ································································.024
2.1.5 順序結構 ············································································.027
2.1.6 分支結構 ············································································.027
2.1.7 循環結構 ············································································.030
2.2 Python 數據分析環境 ······································································.032
2.2.1 使用pip 安裝數據分析相關庫 ··················································.032
2.2.2 安裝Anaconda ·····································································.033
2.3 Python 數據分析相關庫 ···································································.033
2.3.1 NumPy 庫 ··········································································.033
2.3.2 Matplotlib 庫 ·······································································.034
2.3.3 SciPy 庫·············································································.035
2.3.4 Pandas 庫 ···········································································.036
2.3.5 xlrd 庫 ···············································································.036
2.3.6 PyMySQL 庫 ·······································································.037
2.3.7 其他數據分析相關庫 ·····························································.037
2.4 本章小結 ·····················································································.038
第3 章 Jupyter 環境的使用 ................................................................................................. 039
3.1 Jupyter Notebook 概述 ·····································································.039
3.1.1 Jupyter Notebook 簡介及優點 ···················································.039
3.1.2 Jupyter Notebook 開發環境的搭建 ·············································.039
3.1.3 使用pip 命令安裝 ································································.044
3.2 認識Jupyter Notebook ·····································································.044
3.2.1 認識Files、Running、Clusters 頁面 ··········································.044
3.2.2 認識Jupyter Notebook 的主頁面 ···············································.046
3.3 新建、運行、保存Jupyter Notebook 文件 ·············································.048
3.3.1 新建一個Jupyter Notebook ······················································.048
3.3.2 運行代碼 ···········································································.049
3.3.3 重命名Jupyter Notebook 文件 ··················································.049
3.3.4 保存Jupyter Notebook 文件 ·····················································.050
3.4 處理不同類型的數據 ······································································.050
3.4.1 處理txt 文件 ·······································································.050
3.4.2 處理CSV 文件 ····································································.052
3.4.3 處理Excel 文件 ···································································.053
3.4.4 處理sql 文件 ······································································.053
3.5 在Markdown 中使用LaTeX 輸入數學公式 ············································.054
3.5.1 使用LaTeX 輸入一個數學公式 ·················································.054
3.5.2 LaTeX 的2 種公式格式 ··························································.055
3.5.3 常用數學公式的寫法 ·····························································.056
3.6 Jupyter Notebook 應用實例解析 ··························································.058
3.6.1 實例1：能力六維雷達圖的繪制 ···············································.058
3.6.2 實例2：詞頻統計 ·································································.059
3.7 本章小結 ······················································································.060
第4 章 探索數據 .................................................................................................................. 062
4.1 走進數據的世界 ·············································································.062
4.1.1 定義數據 ············································································.062
4.1.2 數據的分類 ·········································································.063
4.1.3 深挖數據的4 種能力 ·····························································.065
4.1.4 善用指標分析問題 ································································.067
4.2 數據的評估 ···················································································.069
4.2.1 指標真的可靠嗎 ···································································.069
4.2.2 統計數據會“說謊” ·····························································.071
4.3 數據怎麽用 ···················································································.072
4.3.1 數據清洗 ············································································.072
4.3.2 數據的標準化 ······································································.076
4.4 本章小結 ······················································································.078
第5 章 描述統計 .................................................................................................................. 079
5.1 數據集中趨勢 ················································································.079
5.1.1 均值的定義與應用 ································································.079
5.1.2 中位數的定義與應用 ·····························································.081
5.1.3 眾數的定義與應用 ································································.083
5.1.4 案例分析 ············································································.085
5.2 數據離散程度 ················································································.087
5.2.1 極差的定義與應用 ································································.088
5.2.2 方差的定義與應用 ································································.090
5.3 本章小結 ·····················································································.091
第6 章 推斷統計 ................................................................................................................... 092
6.1 基礎知識要點 ···············································································.092
6.1.1 排列與組合 ········································································.092
6.1.2 隨機事件及其概率 ································································.095
6.2 概率分佈及其特徵 ·········································································.095
6.2.1 二項分佈 ···········································································.096
6.2.2 正態分佈 ···········································································.098
6.3 統計量 ························································································.104
6.3.1 總體與樣本 ········································································.105
6.3.2 參數估計 ···········································································.109
6.3.3 假設檢驗 ···········································································.112
6.4 本章小結 ·····················································································.116
第7 章 數據可視化 ............................................................................................................... 117
7.1 什麽是數據可視化 ·········································································.117
7.1.1 數據可視化的定義和意義 ·······················································.117
7.1.2 數據可視化的發展歷史 ··························································.118
7.2 圖形對象與元素 ············································································.119
7.2.1 如何建立坐標系···································································.121
7.2.2 如何設置坐標軸的文本和圖例 ·················································.122
7.3 可視化色彩的運用原理 ···································································.123
7.3.1 RGB 顏色模式 ·····································································.123
7.3.2 HSL 顏色模式 ·····································································.124
7.3.3 顏色搭配的技巧和案例 ··························································.124
7.4 圖表的基本類型 ············································································.126
7.4.1 如何繪制柱形圖···································································.126
7.4.2 如何繪制散點圖···································································.127
7.4.3 如何繪制餅形圖···································································.129
7.4.4 如何繪制折線圖 ···································································.130
7.5 數據分析及可視化案例 ····································································.132
7.5.1 數據可視化經典案例 ·····························································.132
7.5.2 非結構化數據的可視化案例 ·····················································.132
7.6 常見的數據可視化流程 ····································································.133
7.7 本章小結 ······················································································.133
第8 章 NumPy 數組 ............................................................................................................ 134
8.1 NumPy 庫簡介 ···············································································.134
8.2 NumPy 數組的生成 ·········································································.134
8.2.1 生成一般數組 ······································································.135
8.2.2 生成特殊數組 ······································································.136
8.2.3 生成隨機數組 ······································································.138
8.3 NumPy 數組基礎 ············································································.140
8.3.1 NumPy 數組的基本屬性 ··························································.140
8.3.2 數組索引：獲取單個元素························································.141
8.3.3 數組切片：獲取子數組 ··························································.141
8.4 NumPy 數組重塑 ············································································.143
8.4.1 NumPy 數組的變形 ·······························································.144
8.4.2 NumPy 數組的轉置和換軸 ·······················································.144
8.4.3 NumPy 數組的拼接與分裂 ·······················································.146
8.5 NumPy 庫中的線性代數 ···································································.147
8.5.1 矩陣乘法 ············································································.147
8.5.2 行列式 ···············································································.148
8.5.3 求線性方程的解 ···································································.148
8.6 通用函數 ······················································································.150
8.6.1 一元通用函數 ······································································.150
8.6.2 二元通用函數 ······································································.150
8.6.3 廣播 ··················································································.152
8.7 常用的數據分析函數 ·······································································.154
8.7.1 條件函數 ············································································.154
8.7.2 聚合函數 ···········································································.154
8.7.3 快速排序 ···········································································.156
8.7.4 唯一值與其他集合邏輯 ··························································.157
8.8 本章小結 ·····················································································.158
第9 章 時間序列數據 ........................................................................................................... 159
9.1 時間序列的定義及分類 ···································································.159
9.1.1 時間序列的定義···································································.159
9.1.2 時間序列的分類···································································.159
9.2 時間序列的描述性分析 ···································································.162
9.2.1 圖形描述 ···········································································.162
9.2.2 增長率分析 ········································································.163
9.3 時間序列的預測 ············································································.164
9.3.1 確定時間序列成分 ································································.165
9.3.2 選擇預測方法 ·····································································.166
9.3.3 預測方法評估 ·····································································.167
9.4 平穩時間序列的預測 ······································································.169
9.4.1 簡單平均法 ········································································.169
9.4.2 移動平均法 ········································································.170
9.4.3 指數平滑法 ········································································.171
9.5 趨勢型和復合型時間序列的預測 ························································.172
9.5.1 線性趨勢預測 ·····································································.172
9.5.2 非線性趨勢預測···································································.173
9.5.3 復合型時間序列的分解預測 ····················································.174
9.6 使用Python 處理時間序列數據 ··························································.174
9.6.1 時間序列數據處理工具的選擇 ·················································.175
9.6.2 時間序列數據的導入 ·····························································.175
9.6.3 時間序列數據預處理 ·····························································.176
9.6.4 時間序列數據處理 ································································.177
9.7 本章小結 ·····················································································.185
第10 章 文本數據 ................................................................................................................ 186
10.1 文本數據的導入 ···········································································.186
10.1.1 文本數據與自然語言處理 ······················································.186
10.1.2 分詞 ················································································.187
10.2 文本數據的處理 ···········································································.189
10.2.1 文本特徵初探 ·····································································.189
10.2.2 文本信息的提取 ··································································.194
10.2.3 文本向量化 ········································································.198
10.3 文本分析的應用 ···········································································.200
10.3.1 文本分類 ···········································································.200
10.3.2 文本情感分析 ·····································································.201
10.4 本章小結 ····················································································.204
第11 章 回歸分析 ................................................................................................................ 205
11.1 叩響人工智能之門 ·········································································.205
11.1.1 人工智能與機器學習 ····························································.205
11.1.2 工欲善其事，必先利其器 ······················································.208
11.1.3 算法，該“出道”了 ····························································.208
11.2 萬朝歸宗：線性回歸 ······································································.209
11.2.1 一元之道 ···········································································.210
11.2.2 從一元到多元 ·····································································.216
11.2.3 學習和工作中的線性回歸 ······················································.218
11.3 回歸增強術 ·················································································.221
11.3.1 非線性回歸 ········································································.221
11.3.2 可分類的回歸 ·····································································.224
11.3.3 能降維的回歸 ·····································································.227
11.4 本章小結 ····················································································.228
第12 章 聚類分析 ................................................................................................................ 229
12.1 數據之眼看聚類 ···········································································.229
12.1.1 什麽是聚類 ·······································································.229
12.1.2 人工智能的未來：無監督學習 ···············································.231
12.1.3 距離產生美 ·······································································.232
12.2 K 均值聚類 ·················································································.234
12.2.1 K 均值聚類的思想 ······························································.234
12.2.2 抽絲剝繭K 均值 ·································································.234
12.2.3 鳶尾花的K 均值聚類 ···························································.238
12.3 案例：數據下的省、區、市 ····························································.240
12.3.1 提出問題 ··········································································.240
12.3.2 數據獲取與處理 ·································································.241
12.3.3 建模分析與結果 ·································································.242
12.4 本章小結 ···················································································.244
第13 章 數據素養綜合案例 ................................................................................................ 245
13.1 綜合案例1：利用人工智能爬取大數據，輕松掌握股市動態 ·························.245
13.1.1 認識人工智能網絡爬蟲 ························································.245
13.1.2 爬取股市大數據，分析需求 ··················································.246
13.1.3 爬取股市大數據案例 ···························································.246
13.2 綜合案例2：人工智能數據——體型分析 ···········································.251
13.2.1 K 最鄰近分類算法原理·························································.251
13.2.2 使用KNN 分類算法對體型進行分類的案例 ·······························.253
13.3 其他案例集 ·················································································.254
13.3.1 計算生肖和星座 ·································································.254
13.3.2 猜數游戲 ··········································································.255
13.3.3 二維列表排序 ····································································.255
13.3.4 學生信息錄入 ····································································.255
13.3.5 打印迴文素數（合數） ························································.256
13.3.6 數據庫加密 ·······································································.256
13.3.7 計算圓台的體積和錶面積 ·····················································.256
13.4 本章小結 ···················································································.257