Fundamentals of Data Observability (for True Epub) by Andy Petrella

Author:Andy Petrella
Language: eng
Format: epub
Publisher: O'Reilly Media, Inc.
Published: 2023-06-02T00:00:00+00:00

For the transformation activity, some technologies don’t prepare the transformation before applying it (called a lazy evaluation like in Spark). Instead, they use a combo of for-loops and manual manipulations. In these cases, it can be difficult to keep track of all transformations automatically from read to write. But, it’s not impossible. And if it’s challenging in an automated fashion, it’s at least as challenging manually, although working on the automation must be done only once and by a few people.

Continuing with the pandas, if your application filters the data read and combines it with others, you can track these operations to make the whole transformation observable. The following code shows how:

// code transforming without => reuse what we used in Low Level API // code that generates DO

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.