Beginning Apache Spark Using Azure Databricks by Robert Ilijason

Author:Robert Ilijason
Language: eng
Format: epub
ISBN: 9781484257814
Publisher: Apress

As you can see, there is nothing much different from writing to CSV. There are functions ready to handle the data export. When we get to Avro, things change however. Let’s look at how that works:df.write.format("avro").save('/tmp/airlines/alavro/')

%fs ls /tmp/airlines/alavro/

In this case you need to use the format command to define that the output should be Avro. While the command is different, the result looks pretty much the same as the rest of them. Now there’s just one more file type we should test. JSON is pretty much identical to CSV:df.coalesce(1).write.json('/tmp/airlines/aljson')

%fs ls /tmp/airlines/aljson/

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.