Timeline for How much data warrants building a pipeline?
Current License: CC BY-SA 3.0
4 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 6, 2017 at 16:39 | vote | accept | Furqan Shakoor | ||
| Aug 3, 2017 at 11:44 | comment | added | CalZ | Daily updates in a regular RDBMS is not bad unless you are always recalculating all your history. | |
| Aug 2, 2017 at 22:56 | comment | added | Furqan Shakoor | I want to use Hadoop because: - I can do cleanup and some normalization of data there (e.g. mapping geo coordinates to cities) - I want the analytics to be updated daily and I want the visualizations to load fast, on unaggreated data they dashboard would become very slow - This pipeline would be future proof, It will allow us to do more complicated stuff in the future - Sounds much cooler than running plain queires on mysql tbh :P | |
| Aug 2, 2017 at 13:23 | history | answered | CalZ | CC BY-SA 3.0 |