金融數據分析和數據挖掘案例實戰

第 1 篇　原理篇
第 1 章　數據科學思維..............................................................2
1.1　數據科學的工作範式 ........................................................................................2
1.2　數據分析方法和流程 ........................................................................................4
1.2.1　發現問題 ...............................................................................................................6
1.2.2　近因分析 ...............................................................................................................8
1.2.3　根因分析 ...............................................................................................................9
1.2.4　做出預測 .............................................................................................................10
1.2.5　制定方案 .............................................................................................................12
1.2.6　驗證方案 .............................................................................................................14
1.3　數據挖掘方法論 ..............................................................................................15
1.3.1 CRISP-DM 方法論 .............................................................................................15
1.3.2 SEMMA 方法論 .................................................................................................16
1.4　金融行業數據挖掘場景 ..................................................................................18
第 2 篇　技術篇
第 2 章　某銀行貸款產品精準營銷模型........................................24
2.1　數據介紹 ..........................................................................................................24
2.2　商業分析 ..........................................................................................................29
2.2.1　發現問題 .............................................................................................................29
2.2.2　診斷問題 .............................................................................................................30
2.2.3　明確目標 .............................................................................................................31
2.2.4　定性分析 .............................................................................................................31
2.3　數據理解 ..........................................................................................................35
2.3.1　建立特征體系 .....................................................................................................35
2.4　數據準備 ..........................................................................................................39
2.4.1　提取被解釋變量 .................................................................................................39
2.4.2　提取靜態特征和時點特征 .................................................................................40
2.4.3　提取時期特征 .....................................................................................................40
2.4.4　提取預測用數據寬表 .........................................................................................41
2.5　建模和評估 ......................................................................................................42
2.5.1　定量客戶畫像與數據清洗 .................................................................................42
2.5.2　建立邏輯回歸模型 .............................................................................................45
2.5.3　評估模型 .............................................................................................................47
2.6　模型運用的準備工作 ......................................................................................48
2.7　流程回顧 ..........................................................................................................49
第3章　多維特征的客戶細分...................................................51
3.1　客戶細分 ..........................................................................................................51
3.1.1　客戶細分定義 .....................................................................................................51
3.1.2　客戶細分類型 .....................................................................................................51
3.1.3　案例：銀行多維度客戶畫像 .............................................................................54
3.2　預處理 ..............................................................................................................57
3.2.1　填補缺失值 .........................................................................................................57
3.2.2　修訂錯誤值 .........................................................................................................58
3.2.3　處理離散變量 .....................................................................................................58
3.2.4　正態化與標準化 .................................................................................................61
3.3　維度分析 ..........................................................................................................64
3.4　聚類 ..................................................................................................................72
3.5　簇特征的解釋 ..................................................................................................75
第4章　信用風險預測模型......................................................81
4.1　信貸全生命周期風險管理 ..............................................................................81
4.1.1　貸前階段 .............................................................................................................81
4.1.2　貸中階段 .............................................................................................................83
4.1.3　貸後階段 .............................................................................................................84

4.2　ABC卡簡介 .....................................................................................................86
4.2.1　信用評分卡簡介 .................................................................................................86
4.2.2　ABC卡的應用 ...................................................................................................87
第5章　貸前信用風險預測模型（A卡）.....................................90
5.1　智能信貸審批基本框架 ..................................................................................90
5.1.1　申請人識別 .........................................................................................................91
5.1.2　信貸準入 .............................................................................................................92
5.1.3　申請評分卡 .........................................................................................................97
5.1.4　全樣本建模與抽樣建模 ...................................................................................106
5.2　特征工程 ........................................................................................................107
5.2.1　數據來源 ...........................................................................................................107
5.2.2　數據加工 ...........................................................................................................109
5.3　模型構建與評估 ............................................................................................121
5.3.1　Logistic回歸模型.............................................................................................121
5.3.2　評分刻度與分值分配 .......................................................................................123
5.3.3　模型評估 ...........................................................................................................125
5.4　模型監控 ........................................................................................................129
5.4.1　前端監控 ...........................................................................................................129
5.4.2　後端監控 ...........................................................................................................134
5.5　拒絕推斷 ........................................................................................................138
5.5.1　外部數據推斷 ...................................................................................................138
5.5.2　模型推斷 ...........................................................................................................139
5.5.3　拒絕推斷結果的驗證 .......................................................................................142
5.6　案例 1：某消費金融公司申請評分卡構建 .................................................143
5.6.1　場景介紹 ...........................................................................................................143
5.6.2　數據清洗 ...........................................................................................................143
5.6.3　特征初篩 ...........................................................................................................148
5.6.4　分箱與 WoE 編碼 .............................................................................................149
5.6.5　相關性分析與特征聚類 ...................................................................................151
5.6.6　逐步回歸 ...........................................................................................................151
5.6.7　模型評估 ...........................................................................................................153
5.6.8　評分卡的制作 ...................................................................................................155
5.6.9　模型文檔 ...........................................................................................................158
5.7　案例 2：制作 Vintage 報告 ...........................................................................159
5.7.1　Vintage 相關業務報表 .....................................................................................159
5.7.2　Vintage 報告的制作 .........................................................................................160
5.8　申請評分卡應用 ............................................................................................166
5.8.1　模型及決策流 ...................................................................................................166
5.8.2　風險策略 ...........................................................................................................167
5.8.3　額度策略 ...........................................................................................................169
第6章　貸中信用風險預測模型（B卡）...................................171
6.1　行為評分卡 ....................................................................................................171
6.1.1　業務理解 ...........................................................................................................171
6.1.2　數據理解 ...........................................................................................................172
6.1.3　特征工程 ...........................................................................................................173
6.1.4　模型構建與評估 ...............................................................................................17
46.2　案例：某信用卡業務行為評分卡構建 ........................................................174
6.2.1　場景介紹 ...........................................................................................................174
6.2.2　數據整理與特征工程 .......................................................................................175
6.2.3　數據清洗與特征初篩 .......................................................................................185
6.2.4　分箱與 WoE 編碼 .............................................................................................187
6.2.5　相關性篩選 .......................................................................................................187
6.2.6　逐步回歸建模 ...................................................................................................187
6.2.7　模型評估 ...........................................................................................................188
6.3　行為評分卡的應用 ........................................................................................190
6.3.1　額度管理 ...........................................................................................................190
6.3.2　續卡或續貸策略 ...............................................................................................191
6.3.3　客戶留存分析和挽留 .......................................................................................191
6.3.4　風險監控 ...........................................................................................................192
第7章　貸後催收模型（C卡）..............................................193
7.1　催收評分卡 ....................................................................................................193
7.1.1　業務理解 ...........................................................................................................193
7.1.2　數據理解 ...........................................................................................................195
7.1.3　特征工程與模型構建 .......................................................................................196
7.2　催收評分卡的應用 ........................................................................................197
7.2.1　預催收階段 .......................................................................................................198
7.2.2　早期催收階段 ...................................................................................................199
第8章　申請反欺詐模型.......................................................200
8.1　業務理解 ........................................................................................................200
8.1.1　申請欺詐產生的背景 .......................................................................................200
8.1.2　申請欺詐的分類 ...............................................................................................201
8.1.3　申請欺詐的應對 ...............................................................................................203
8.2　案例：申請反欺詐模型 ................................................................................205
8.2.1　異常特征構造 ...................................................................................................207
8.2.2　網絡特征提取 ...................................................................................................214
8.2.3　構建識別模型 ...................................................................................................235
第9章　算法工程化.............................................................248
9.1　構建合理的項目結構 ....................................................................................248
9.1.1　為什麼要構建合理的項目結構 .......................................................................248
9.1.2　什麼是一個數據科學項目應有的項目結構 ...................................................250
9.2　如何編寫規範的數據工程代碼 ....................................................................254
9.2.1　代碼可讀性 .......................................................................................................254
9.2.2　數據處理性能 ...................................................................................................259