I am following this tutorial:https://docs.microsoft.com/en-us/learn/modules/intro-to-azure-databricks/4-using-notebooks
In this tutorial we create a database like this:
%sql
CREATE DATABASE IF NOT EXISTS Databricks;
USE Databricks;
CREATE TABLE IF NOT EXISTS AirlineFlight
USING CSV
OPTIONS (
header="true",
delimiter=",",
inferSchema="true",
path="dbfs:/mnt/training/asa/flights/small.csv"
);
CACHE TABLE AirlineFlight;
SELECT * FROM AirlineFlight;
Where is this database created? moreover there is a question asked
Question: Which of the following are good applications for Apache Spark? (Select all that apply.)
Querying, exploring, and analyzing very large files and data sets
Joining data lakes
Machine learning and predictive analytics
Processing streaming data
Graph analytics
Overnight batch processing of very large files
Updating individual records in a database
Answer: All but #7. Apache Spark uses SQL to read and performs analysis on large files, but it is not a Database.
If we can create a database using spark then why can't we change its records too