深度學習與神經網絡

全書分為7 個章節。第1 章緒論，梳理了人工智能不同技術流派的特點、深度學習的發展及前沿技術；第2 章介紹相關預備知識，包括線性代數、概率論、優化理論以及機器學習的基礎知識；第3 章從前饋神經網絡的基礎模型——感知器出發，介紹前饋神經網絡的基本結構以及涉及的激活函數、梯度下降、反向傳播等內容；第4 章，介紹深度模型的優化問題，討論了神經網絡優化中常見的病態問題；第5 章介紹深度學習中的正則化方法，包括範數懲罰、數據集增強與噪聲註入、提前停止等；第6 章介紹了捲積神經網絡，以及捲積神經網絡在電腦視覺領域的具體應用；第7 章通過實際案例介紹循環神經網絡與捲積神經網絡的結合應用。

第1 章 緒論····················································································.1
1.1 人工智能·············································································.2
1.1.1 人工智能技術的發展歷程···············································.3
1.1.2 人工智能技術的流派·····················································.9
1.2 深度學習與神經網絡概述······················································.11
1.2.1 深度學習與神經網絡技術的發展歷程······························.11
1.2.2 深度學習與神經網絡的前沿技術····································.16
1.3 深度學習系統架構·······························································.17
1.4 深度學習框架·····································································.19
1.5 深度學習的應用··································································.20
1.5.1 電腦視覺·······························································.20
1.5.2 語音語義··································································.21
1.5.3 自然語言處理····························································.22
1.6 人工智能潛在的安全風險······················································.22
1.6.1 數據層面的風險·························································.23
1.6.2 算法模型層面的風險···················································.23
1.6.3 智能計算框架層面的風險·············································.23
1.6.4 基礎軟硬件層面的風險················································.24
1.6.5 應用服務層面的風險···················································.24
本章小結··················································································.24
第2 章 預備知識············································································.25
2.1 相關數學基礎·····································································.25
2.1.1 線性代數··································································.25
2.1.2 概率論·····································································.27
2.1.3 優化理論··································································.32
2.2 機器學習基礎·····································································.34
2.2.1 機器學習算法的基本流程·············································.35
2.2.2 機器學習常用評價指標················································.36
2.2.3 典型機器學習算法······················································.41
2.3 實驗環境基礎·····································································.49
2.3.1 GPU 驅動的安裝配置··················································.49
2.3.2 依賴環境的安裝配置···················································.52
2.3.3 深度學習框架的安裝配置·············································.54
2.3.4 集成開發環境的安裝配置·············································.56
本章小結··················································································.57
第3 章 前饋神經網絡······································································.58
3.1 感知器··············································································.58
3.1.1 單層感知器·······························································.58
3.1.2 多層感知器·······························································.61
3.1.3 前饋神經網絡的基本結構·············································.62
3.2 激活函數···········································································.63
3.2.1 Sigmoid 函數·····························································.63
3.2.2 ReLU 函數································································.65
3.2.3 Tanh 函數·································································.67
3.2.4 Softmax 函數·····························································.68
3.3 誤差反向傳播·····································································.69
3.3.1 梯度下降法·······························································.69
3.3.2 鏈式法則··································································.70
3.3.3 反向傳播··································································.72
本章小結··················································································.75
第4 章 深度模型的優化···································································.76
4.1 神經網絡的優化問題····························································.76
4.1.1 局部最優和振盪陷阱···················································.76
4.1.2 梯度爆炸和梯度消失···················································.78
4.2 常見的優化算法··································································.80
4.2.1 梯度下降優化算法······················································.80
4.2.2 二階優化算法····························································.81
4.3 自適應學習率算法·······························································.85
4.3.1 AdaGrad 算法····························································.85
4.3.2 RMSprop 算法···························································.86
4.3.3 Adam 算法································································.86
4.4 參數初始化方法··································································.87
4.4.1 隨機初始化·······························································.87
4.4.2 Xavier 初始化····························································.88
4.4.3 He 初始化·································································.89
本章小結··················································································.90
第5 章 深度學習中的正則化·····························································.91
5.1 範數懲罰···········································································.91
5.1.1 L1 正則化··································································.91
5.1.2 L2 正則化··································································.93
5.2 數據集增強與噪聲註入·························································.94
5.2.1 數據集增強·······························································.94
5.2.2 噪聲註入··································································.95
5.3 提前停止···········································································.97
5.4 Dropout ·············································································.97
5.5 批歸一化···········································································.99
本章小結·················································································.102
第6 章 捲積神經網絡······································································103
6.1 捲積神經網絡的發展歷程·····················································.103
6.2 捲積神經網絡的基本組成·····················································.104
6.2.1 捲積層····································································.105
6.2.2 池化層····································································.106
6.2.3 全連接層·································································.108
6.3 常見捲積神經網絡結構························································.108
6.3.1 VGG 網絡································································.109
6.3.2 GoogLeNet 網絡························································.111
6.3.3 ResNet 網絡·····························································.117
6.4 深度生成網絡····································································.122
6.4.1 生成對抗網絡···························································.122
6.4.2 深度捲積生成對抗網絡···············································.123
6.5 圖像分類案例····································································.134
6.5.1 步驟1：搭建環境······················································.135
6.5.2 步驟2：導入依賴庫···················································.137
6.5.3 步驟3：獲取數據······················································.137
6.5.4 步驟4：定義AlexNet 網絡··········································.138
6.5.5 步驟5：模型初始化···················································.140
6.5.6 步驟6：模型訓練······················································.140
6.6 目標檢測案例····································································.143
6.6.1 步驟1：環境配置和模型下載·······································.144
6.6.2 步驟2：主函數解析···················································.151
6.6.3 步驟3：終端指令運行················································.152
本章小結·················································································.153
第7 章 循環神經網絡·····································································.154
7.1 循環神經網絡的基本原理·····················································.154
7.1.1 循環神經網絡的原理··················································.154
7.1.2 雙向循環神經網絡·····················································.157
7.2 循環神經網絡在實際中的應用···············································.158
7.2.1 文本生成·································································.159
7.2.2 語音識別·································································.159
7.2.3 機器翻譯·································································.160
7.2.4 生成圖像描述···························································.161
7.2.5 視頻動作檢測···························································.162
7.2.6 信號分類·································································.162
7.3 長短期記憶網絡及其他門控循環神經網絡································.163
7.3.1 長短期記憶網絡························································.164
7.3.2 其他門控循環神經網絡···············································.166
7.4 深度學習在文本和序列中的應用············································.167
7.4.1 文本數據處理···························································.167
7.4.2 文本分類和情感分析··················································.180
7.4.3 機器翻譯·································································.180
7.4.4 命名實體識別···························································.182
7.5 捲積神經網絡與循環神經網絡···············································.183
7.5.1 捲積神經網絡與循環神經網絡的對比·····························.183
7.5.2 捲積神經網絡與循環神經網絡的組合應用·······················.184
7.6 案例：深度學習的詩歌生成··················································.185
7.6.1 步驟1：導入依賴庫···················································.186
7.6.2 步驟2：讀取數據······················································.187
7.6.3 步驟3：構造數據集···················································.188
7.6.4 步驟4：構造模型······················································.190
7.6.5 步驟5：訓練過程······················································.192
7.6.6 步驟6：生成文本······················································.195
7.7 案例：基於LSTM 算法的股票預測·········································.196
7.7.1 步驟1：導入依賴庫···················································.196
7.7.2 步驟2：獲取並處理數據·············································.197
7.7.3 步驟3：構建預測數據序列··········································.199
7.7.4 步驟4：構建LSTM 網絡············································.201
7.7.5 步驟5：訓練網絡······················································.202
7.7.6 步驟6：預測測試集···················································.203
7.8 案例：基於深度學習的文本分類············································.204
7.8.1 步驟1：項目入口······················································.205
7.8.2 步驟2：訓練模塊······················································.207
7.8.3 步驟3：驗證和測試函數·············································.210
7.8.4 步驟4：數據預處理模塊·············································.211
7.8.5 步驟5：定義模型······················································.215
7.8.6 步驟6：分類結果展示················································.218
本章小結·················································································.219