All Questions
Tagged with scalability efficiency
7 questions
1
vote
1
answer
88
views
Ways to speed up Python code for data science purposes
Although it might sound like a pure techie question, I would like to know which ways you usually try out, for very data science-like processes, when you need to speed up your processes (given that the ...
- 2,734
0
votes
1
answer
312
views
When it is time to use Hadoop? [closed]
Hadoop is a buzzword now. A lot of start-ups use it (or just say, that they use it), a lot of widely known companies use it. But when and what is the border? When person can say: "Better to solve it ...
10
votes
3
answers
353
views
How do various statistical techniques (regression, PCA, etc) scale with sample size and dimension?
Is there a known general table of statistical techniques that explain how they scale with sample size and dimension? For example, a friend of mine told me the other day that the computation time of ...
- 229
14
votes
4
answers
2k
views
Looking for example infrastructure stacks/workflows/pipelines
I'm trying to understand how all the "big data" components play together in a real world use case, e.g. hadoop, monogodb/nosql, storm, kafka, ... I know that this is quite a wide range of tools used ...
- 143
8
votes
3
answers
185
views
How to compare experiments run over different infrastructures
I'm developing a distributed algorithm, and to improve efficiency, it relies both on the number of disks (one per machine), and on an efficient load balance strategy. With more disks, we're able to ...
- 4,117
6
votes
2
answers
145
views
How to best accomplish high speed comparison of like data?
I attack this problem frequently with inefficiency because it's always pretty low on the priority list and my clients are resistant to change until things break. I would like some input on how to ...
- 3,128
94
votes
12
answers
20k
views
How big is big data?
Lots of people use the term big data in a rather commercial way, as a means of indicating that large datasets are involved in the computation, and therefore potential solutions must have good ...
- 4,117