Fast Data Processing with Spark

Holden Karau

  • 出版商: Packt Publishing
  • 出版日期: 2013-09-08
  • 售價: $1,490
  • 貴賓價: 9.5$1,416
  • 語言: 英文
  • 頁數: 120
  • 裝訂: Paperback
  • ISBN: 1782167064
  • ISBN-13: 9781782167068
  • 相關分類: Spark
  • 海外代購書籍(需單獨結帳)

買這商品的人也買了...

商品描述

Spark offers a streamlined way to write distributed programs and this tutorial gives you the know-how as a software developer to make the most of Spark's many great features, providing an extra string to your bow.

Overview

  • Implement Spark's interactive shell to prototype distributed applications
  • Deploy Spark jobs to various clusters such as Mesos, EC2, Chef, YARN, EMR, and so on
  • Use Shark's SQL query-like syntax with Spark

In Detail

Spark is a framework for writing fast, distributed programs. Spark solves similar problems as Hadoop MapReduce does but with a fast in-memory approach and a clean functional style API. With its ability to integrate with Hadoop and inbuilt tools for interactive query analysis (Shark), large-scale graph processing and analysis (Bagel), and real-time analysis (Spark Streaming), it can be interactively used to quickly process and query big data sets.

Fast Data Processing with Spark covers how to write distributed map reduce style programs with Spark. The book will guide you through every step required to write effective distributed programs from setting up your cluster and interactively exploring the API, to deploying your job to the cluster, and tuning it for your purposes.

Fast Data Processing with Spark covers everything from setting up your Spark cluster in a variety of situations (stand-alone, EC2, and so on), to how to use the interactive shell to write distributed code interactively. From there, we move on to cover how to write and deploy distributed jobs in Java, Scala, and Python.

We then examine how to use the interactive shell to quickly prototype distributed programs and explore the Spark API. We also look at how to use Hive with Spark to use a SQL-like query syntax with Shark, as well as manipulating resilient distributed datasets (RDDs).

What you will learn from this book

  • Prototype distributed applications with Spark's interactive shell
  • Learn different ways to interact with Spark's distributed representation of data (RDDs)
  • Load data from the various data sources
  • Query Spark with a SQL-like query syntax
  • Integrate Shark queries with Spark programs
  • Effectively test your distributed software
  • Tune a Spark installation
  • Install and set up Spark on your cluster
  • Work effectively with large data sets

Approach

This book will be a basic, step-by-step tutorial, which will help readers take advantage of all that Spark has to offer.

Who this book is written for

Fast Data Processing with Spark is for software developers who want to learn how to write distributed programs with Spark. It will help developers who have had problems that were too much to be dealt with on a single computer. No previous experience with distributed programming is necessary. This book assumes knowledge of either Java, Scala, or Python.

商品描述(中文翻譯)

Spark 提供了一種簡化的方式來撰寫分散式程式,而本教程將教給軟體開發人員如何充分利用 Spark 的許多優點,為您的技能增添一把利器。

概述:
- 使用 Spark 的互動式 shell 來原型分散式應用程式
- 將 Spark 作業部署到各種集群,如 Mesos、EC2、Chef、YARN、EMR 等
- 使用 Shark 的類似 SQL 查詢的語法與 Spark 一起使用

詳細內容:
Spark 是一個用於撰寫快速分散式程式的框架。Spark 解決了與 Hadoop MapReduce 相似的問題,但採用了快速的內存處理方式和乾淨的函數式 API。它能夠與 Hadoop 整合,並具有用於互動式查詢分析(Shark)、大規模圖形處理和分析(Bagel)以及實時分析(Spark Streaming)的內建工具,可用於快速處理和查詢大型資料集。

《使用 Spark 進行快速資料處理》介紹了如何使用 Spark 撰寫分散式 MapReduce 程式。本書將引導您完成從設置集群和互動式探索 API,到部署作業到集群並針對您的需求進行調優的每一個步驟。

《使用 Spark 進行快速資料處理》涵蓋了從各種情況下(獨立、EC2 等)設置 Spark 集群的所有內容,以及如何使用互動式 shell 進行互動式編寫分散式程式的方法。從那裡,我們將介紹如何使用 Java、Scala 和 Python 編寫和部署分散式作業。

然後,我們將探討如何使用互動式 shell 快速原型分散式程式並探索 Spark API。我們還將研究如何使用 Hive 與 Spark 一起使用類似 SQL 的查詢語法,以及操作可靠分散式資料集(RDD)。

本書將教您以下內容:
- 使用 Spark 的互動式 shell 來原型分散式應用程式
- 學習與 Spark 的分散式資料表示(RDD)互動的不同方式
- 從各種資料來源載入資料
- 使用類似 SQL 的查詢語法查詢 Spark
- 將 Shark 查詢與 Spark 程式整合
- 有效地測試您的分散式軟體
- 調優 Spark 安裝
- 在集群上安裝和設置 Spark
- 有效處理大型資料集

這本書是一本基礎的、逐步教學的指南,將幫助讀者充分利用 Spark 的所有功能。

本書的目標讀者是希望學習如何使用 Spark 撰寫分散式程式的軟體開發人員。它將幫助那些在單台電腦上無法處理的問題。不需要有分散式程式設計的經驗。本書假設讀者具備 Java、Scala 或 Python 的知識。