Big Data Analysis with Python by Ivan Marin

Author:Ivan Marin
Language: eng
Format: epub
Publisher: Packt Publishing
Published: 2019-04-07T16:00:00+00:00

In this exercise, we will add a new column in the dataset using the withColumn method, and later, using the drop function, will remove it. Now, let's perform the following steps:

Add a new column in a Spark DataFrame using the withColumn method:df = df.withColumn('Half_sepal_width', df['Sepalwidth']/2.0)

Use the following command to show the dataset with the newly added column:df.show(4)

Figure 4.10: Introducing new column, Half_sepal_width

Now, to remove a column in a Spark DataFrame, use the drop method illustrated here:df = df.drop('Half_sepal_width')

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.