I have a number of large datasets (10GBs) each with data fetched from a NoSQL database that I have remotely downloaded on my desktop. I would like to write a Python program to run some custom data analysis (plots - preferably interactive) and export custom reports in html or pdf.
I was wondering how people do the following:
1) Store the data. For the moment I have plain text files (each file has rows of a fixed number of columns - most of the data are categorical). Would it make sense to save those in some database (SQL) or hdf5? Any hints on which is preferrable?
2) Which plotting library would you propose for the graphs? I have seen about bookeh and matplotlib supports interactive widgets but I don't know what people normally use.
3) Could I export the analysis results in an IPython notebook and then in html programmatically?