Business Intelligence with Databricks SQL by Vihag Gupta

Author:Vihag Gupta
Language: eng
Format: mobi, epub
Publisher: Packt Publishing Ltd.
Published: 2022-08-26T00:00:00+00:00

Figure 6.11 – Setting up cost analysis

Once this has been set up, the cost analysis dashboard should populate the accumulated costs, like so:

Figure 6.12 – Accumulated cost

It should also populate breakdowns of the Databricks cost and the underlying VM cost, like so:

Figure 6.13 – Cost breakdown

If you are an AWS user, you might be expecting something similar in the Billing and Cost Management console. However, that is not the case. Databricks is a first-party service in Azure, so its billing is tightly integrated. We can see the Databricks billing in the cost analysis portal. Databricks is not a first-party service in AWS.

AWS users can see the Databricks spend on their Databricks account console, as documented here: https://docs.databricks.com/administration-guide/account-settings-e2/usage.xhtml.

You will also notice that the console does not show the underlying VM costs. You must get those costs from the Billing and Cost Management console using cluster tags since they are propagated to the underlying VMs. You can read more about cluster tags and tag propagation here: https://docs.databricks.com/administration-guide/account-settings/usage-detail-tags-aws.xhtml.

GCP users can see their spending on their account console, as documented here: https://docs.gcp.databricks.com/administration-guide/account-settings-gcp/usage.xhtml.

In summary, a reliable chargeback is important, and reliability comes from having a sound user assignment strategy and strategy enforcement. Databricks SQL provides all the tools to implement this, and we discussed all the nuances in depth in this section.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.