Web Scraping with Python: Collecting More Data from the Modern Web, 2/e (Paperback)

Ryan Mitchell



If programming is magic then web scraping is surely a form of wizardry. By writing a simple automated program, you can query web servers, request data, and parse it to extract the information you need. The expanded edition of this practical book not only introduces you web scraping, but also serves as a comprehensive guide to scraping almost every type of data from the modern web.

Part I focuses on web scraping mechanics: using Python to request information from a web server, performing basic handling of the server’s response, and interacting with sites in an automated fashion. Part II explores a variety of more specific tools and applications to fit any web scraping scenario you’re likely to encounter.

  • Parse complicated HTML pages
  • Develop crawlers with the Scrapy framework
  • Learn methods to store data you scrape
  • Read and extract data from documents
  • Clean and normalize badly formatted data
  • Read and write natural languages
  • Crawl through forms and logins
  • Scrape JavaScript and crawl through APIs
  • Use and write image-to-text software
  • Avoid scraping traps and bot blockers
  • Use scrapers to test your website




- 解析複雜的HTML頁面
- 使用Scrapy框架開發爬蟲程式
- 學習儲存爬取數據的方法
- 讀取並從文件中提取數據
- 清理和規範格式錯誤的數據
- 讀寫自然語言
- 通過表單和登錄進行爬蟲
- 爬取JavaScript並通過API進行爬蟲
- 使用和編寫圖片轉文字軟體
- 避免爬蟲陷阱和機器人阻擋
- 使用爬蟲測試您的網站