I have somewhat philosophical question regarding performing predictive analytics on distributed systems (such as hadoop). I am no expert on this subject so maybe other folks who have more advance knowledge on this subject can enlighten me. Here is where I am getting confused:
How can a statistical model (such as linear regression model) built on Hadoop where data is distributed across many nodes, and where work is done only on subset of original data-set residing in each node have same or better accuracy than linear regression model built on single server where all data resides locally. Would the error-rate (accuracy) of a model build on hadoop be same, better or worse than model build on local system. I was thinking that since all data reside in local system, algorithm can be tuned to optimize error in a linear regression model. However, in case of Hadoop where data is distributed, what if only local optimization lead to drastically bad algorithm overall? Are the algorithms written separately for distributed system to take into account data distribution?