2
votes
How to deal with high data volumes? (Tools, techniques, concepts, etc.)
The fastest data structures in Python are sets.
They are low-level structures, and you can manage your data quickly.
To avoid using too much RAM, you can build them progressively with a CSV with a ...
- 4,674
1
vote
How to deal with high data volumes? (Tools, techniques, concepts, etc.)
As it was mentioned, you can use your memory several times more efficiently using other tools (numpy, Redis, C++, R / data.table, etc), but after that you will still hit your laptop RAM limit and then ...
- 1,209
Only top scored, non community-wiki answers of a minimum length are eligible