Data Wrangling on AWS by Navnit Shukla | Sankar M | Sam Palani

Data Wrangling on AWS by Navnit Shukla | Sankar M | Sam Palani

Author:Navnit Shukla | Sankar M | Sam Palani
Language: eng
Format: epub, mobi
Publisher: Packt Publishing Pvt Ltd
Published: 2023-07-30T00:00:00+00:00


AWS Glue building blocks

AWS Glue Crawler allows you to crawl data from various data sources, including S3, and databases such as Redshift, MySQL, MSSQL, Oracle, and MongoDB, among others. Once the crawling process is complete, AWS Glue Crawler automatically creates or updates tables in the AWS Glue Data Catalog. A list of supported data sources can be found at https://docs.aws.amazon.com/glue/latest/dg/crawler-data-stores.html:

AWS Glue Data Catalog: The AWS Glue Data Catalog is a centralized catalog that stores metadata from various data stores. These can be used by other AWS services such as Amazon Athena, Amazon QuickSight, Amazon EMR, and Amazon Redshift.

AWS Glue Triggers: AWS Glue Triggers enable you to create schedules that can manually or automatically start one or more AWS Glue Crawlers or AWS Glue ETL jobs. Triggers can be configured to fire on demand, based on a schedule, or based on a combination of events.

AWS Glue Workflows: AWS Glue Workflows enable you to orchestrate the execution of Glue ETL and Glue Crawler using a Glue Trigger.

AWS Glue ETL: AWS Glue ETL allows users to create Extract, Transform, and Load (ETL) pipelines using Python, PySpark, or Scala. With Glue ETL, you can process data from various data sources, perform transformations, and eventually write to different data sources (such as Amazon Redshift, Oracle, OpenSearch, and Amazon S3). You have the option to choose from different processing engines: for distributed processing, you can choose between PySpark and Scala, while for non-distributed processing, you can select Python:



Download



Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.