Information Theory: Three Theorems by Claude Shannon

This book provides an introduction to information theory, focussing on Shannon's three foundational theorems of 1948-1949. Shannon's first two theorems, based on the notion of entropy in probability theory, specify the extent to which a message can be compressed for fast transmission and how to erase errors associated with poor transmission. The third theorem, using Fourier theory, ensures that a signal can be reconstructed from a sufficiently fine sampling of it. These three theorems constitute the roadmap of the book.

The first chapter studies the entropy of a discrete random variable and related notions. The second chapter, on compression and error correcting, introduces the concept of coding, proves the existence of optimal codes and good codes (Shannon's first theorem), and shows how information can be transmitted in the presence of noise (Shannon's second theorem). The third chapter proves the sampling theorem (Shannon's third theorem) and looks at its connections with other results, such as the Poisson summation formula. Finally, there is a discussion of the uncertainty principle in information theory.

Featuring a good supply of exercises (with solutions), and an introductory chapter covering the prerequisites, this text stems out lectures given to mathematics/computer science students at the beginning graduate level.

本書介紹了資訊理論，重點關注於Shannon於1948-1949年提出的三個基礎定理。Shannon的前兩個定理基於概率論中的熵概念，指定了訊息在快速傳輸時可以壓縮的程度，以及如何消除與傳輸品質不佳相關的錯誤。第三個定理則利用傅立葉理論，確保可以從足夠細緻的取樣中重建訊號。這三個定理構成了本書的路線圖。

第一章研究了離散隨機變數的熵及相關概念。第二章介紹了編碼和錯誤修正，引入了編碼的概念，證明了最佳編碼和良好編碼的存在（Shannon的第一個定理），並展示了在噪聲存在的情況下如何傳輸訊息（Shannon的第二個定理）。第三章證明了取樣定理（Shannon的第三個定理），並探討了它與其他結果（如泊松求和公式）的關聯。最後，還討論了資訊理論中的不確定性原理。

本書提供了豐富的練習題（附有解答），並在引言章節中介紹了先備知識。這本書是針對數學/計算機科學研究生入門級別的學生所授課的講義。

Antoine Chambert-Loir is a professor of mathematics at Université Paris Cité. His research addresses questions in algebraic geometry which are motivated by number theoretical problems. He is the author of two books published by Springer-Verlag: A Field Guide To Algebra, an introduction to Galois theory; and (Mostly) Commutative Algebra, an intermediate-level exposition of commutative algebra. With J. Nicaise and J. Sebag, he cowrote the research monograph Motivic Integration (published by Birkhäuser), which was awarded the 2017 Ferran Sunyer i Balaguer prize.

Antoine Chambert-Loir是巴黎城市大學的數學教授。他的研究探討代數幾何中受數論問題啟發的問題。他是Springer-Verlag出版的兩本書的作者: A Field Guide To Algebra，一本介紹 Galois 理論的入門書; 以及(Mostly) Commutative Algebra，一本介紹交換代數的中級教材。他與J. Nicaise和J. Sebag合著的研究專著Motivic Integration（由Birkhäuser出版），獲得了2017年Ferran Sunyer i Balaguer獎。