In all courses, the focus is mainly on building models from an already clean dataset that is in a csv file.

If I have a database, Postgres, with a table that has ~10 million rows, what is the best way to export it to do exploratory data analysis (EDA)?

I can export it to a csv (only 1 million rows), but is that the optimal way? How are databases usually used when doing EDA and building models, when dealing with millions of records?

Any resources to read on this topic or suggestions are appreciated.

  • $\begingroup$ How would you do it with a CSV, explore in Excel? $\endgroup$
    – Dave
    Jul 9 at 16:56
  • $\begingroup$ I would use python to run some visualizations and analysis on the csv file. But it is limiting the export to 1 million rows. $\endgroup$
    – EmmaM
    Jul 9 at 17:31
  • $\begingroup$ Python can connect to a Postgres database. $\endgroup$
    – Dave
    Jul 9 at 17:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.