1
$\begingroup$

I have some data in raw csv files which I would like to store in a MySQL database. The problem is there are constant feature engineering done on this dataset so coming up with one schema to fit all the needs is not possible. The approach I thought of was to have one main table where the original data is held, and for each new feature that is created, a new table is created. Then, the user can join the original table with other tables which includes the features they want and use it for their own purposes.

With the above approach, I'm worried about having too many joins when a user needs numerous features. Please advise on alternative approaches to this problem.

Thanks in advance!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Depending on the technical level of your users, the frequency of the update, the complexity of the transformation, the need to share these features among the users, etc. would a custom VIEW for each user be a feasible solution?

Alternately, would you consider some ETL tools where you can create/ modify calculated columns, customizing the data pipeline as needed?

Improve this answer
$\endgroup$
1
  • $\begingroup$ I think that's a good point, will try the custom view approach because it sounds very flexible and the people using this would be from a technical position. Thanks! $\endgroup$
    – masotann
    Commented Jun 4, 2018 at 16:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.