Stack Exchange network consists of 183 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.
Store data into memory. So, here memory means local hard disk memory you are telling right? So, if i get those tables available on my PC hard disk. Then do you mean, i should import them into my python notebook from hard disk n run analysis on it? I don't know, what happens behind the scene, when pandas import records or when it processes records? Where it keeps records? What is the role of RAM here n storage(hard drive) here. I am new to this,so please bear with me. Any good link, you can suggest to understand, what i mentioned above, will be greatly appreciated. Thanks a lot for giving time.
Thanks Rajat. If i have good configuration of Server, then can try? Also if i create resulting combined table in Sql, then i just will import it in pandas. But still it will be a table of 6 million rows. So, can i analyse/visualize it in pandas? Or i can aggregate few rows into summary rows like sum, average n reduce rows size to less as .1 million, n then run analysis/visualization on that? Kindly suggest.