1
$\begingroup$

My web application stores usage data, for example:

  • tickets opened an closed
  • tasks executed
  • user scores

etc. I need to show dashboards and reports for usage and performance trends, like:

  • How many tickets where opened/closed in a period?
  • what is the average tasks execution time?
  • who is the most active user?
  • which user gets the best score?

etc. the set of questions is open, we can invent more.

Does this scenario fall under a datawarehouse classic approach? Data are coming from a relational database with a specific data model.

Somebody is asking me to create a datalake, just giving him all my raw data. I don't understand how does a datalake fit in this picture. With no model and relationship my data are usueless.

Also, I don't have a stream of events: I have tables in a relational database. For example a row in a "task" table will have an "open time" column and a "close time" column.

I'm confused about the need or advantages of creating a datalake vs a traditional datawarehouse approach

$\endgroup$
2
$\begingroup$

The point of a "Data Lake" is to store unstructured data together, along with the related meta-data, all in one place. The data is usually stored in the format in which it was recorded i.e. the raw format. Data is then only extracted and processed for usage as required.

There is a nice article that compares data warehouses and data lakes in an unbiased and practical manner.


From your description of your data and the problems you want to solve, I don't personally think a data lake would be required.

Perhaps your colleague is planning to create a data lake for larger purposes, e.g. to house the data for many projects such as yours, which might also want to use your data? There are the issues of security, data ownership and data stewardship to think about in this case. Questions to answer would be:

  • who actually owns the data?
  • who is allowed to use what data?
  • who is allowed to view what data?
  • who is responsible for ensuring data is available at all times?
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.