4
$\begingroup$

I'm planning to run experiments with large datasets on distributed system in order to evaluate efficiency gains in comparison with previous proposals.

I have limited number of machines nearly ten machines having 200 GB of free space on hard disk on each. On the contrary, I wished to perform experiments on more than available nodes in order to measure scalability, more precisely. Since I don't have any, I thought about using a commodity cluster. However, I'm not sure about the policies of usage, and I need to reliably measure execution times.

Are there commodity services which will grant me that only my application would be running at a given time?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
4
$\begingroup$

I have some thoughts about your question. I hope it may help you to solve your problem.

I'm planning to run some experiments with very large data sets, and I'd like to distribute the computation.

In one of my posts, I have done research on topic of evaluation methods of Data Science. With Learning Curve, you can evaluate your experiments learning ability. To talk a bit more, you will fix commodity configuration, and then will run same experiment on the same number of machines with different size of data set you have (from starting from small chunk in size, incrementally increase the size until you reach the whole data set).

To point on, you should avoid having power distribution for result of performance test being run with different size of data sets. To avoid, you should carefully choose step size (step size = amount of increments).

I have about ten machines available, each with 200 GB of free space on hard disk. However, I would like to perform experiments on a greater number of nodes, to measure scalability more precisely.

For this type question, I have intuitively searched and read materials; afterwards, published as a blog post. At the end of the post, I have briefly talked about how to test your hypothesis on real complex system. If you let me, I want to briefly talk about;

First of all, base requirement should be formed in order to run data set as a whole. The minimum requirement will build your baseline evaluation score which is calculated with one/combination of evaluation metrics you have chosen, or with one/combination of methods being used to calculate Running Time = Computation complexity + Communication cost + Synchronization cost.

After those steps, with an evaluation strategy, add new elements, e.g. new node, to the system you have doing scalability test; meanwhile, for each addition, measure performance w.r.t new system configuration.

Just to note, evaluation strategy must be planned along with considerations of default behavior of parallel and distributed systems. For example, what I mean by behavior is that adding just more cores will, after some point, automatically drop performance of the system not due to your algorithm characteristics. It is because more cores need more RAMs, more hard driver, or etc. In other words, there is a N-Way relationship between hardware components. As a second example, adding more nodes to the distributed system will punished you with more communication and synchronization costs.

As a last step, you will sketch two different graphs with your evaluation results via data analysis program or language (As a recommendation, use GNU Plot or R programming language). Print out and put those results at your desktop, and start to examine them, carefully. According to your investigation, modify/erase + rebuild evaluation strategy and re-do the performance test.

Are there commodity services which would grant me that only my application would be running at a given time? Has anyone used such services yet?

I have no much experiment on commodity services, but I can easily say whether it grants or not depends on your configuration of services. If you configured say Hadoop to your node as an only service, Hadoop will grant your code will be only running at any time.

Improve this answer
$\endgroup$
1
  • $\begingroup$ The links provided in the answer seem to be dead ;( $\endgroup$
    – Janis Peisenieks
    Apr 27, 2015 at 7:27
1
$\begingroup$

If your work is parallelizable enough for a distributed network of cpus to make a difference, why not try to run it on the gpu instead? That will require rather less investment than a large network of cpus with individual software licenses and still provide parallel processing which you can do runtime tracking on yourself.

Improve this answer
$\endgroup$
3
  • $\begingroup$ The computations I perform are both cpu and disk intense, which may occur concurrently. Using gpu's would surely speedup the cpu step, but disk access would still limit the performance. $\endgroup$
    – Rubens
    Aug 2, 2014 at 16:53
  • 1
    $\begingroup$ @Rubens you've to have good reason to use GPU for your computation. It expects fine-grained matrix type data, and well written parallel works, kernel implementations. For your disk problem, you'ld stream your data and feed them into RAM so that both (especially) GPU and CPU will benefits. Then, you can use your data in any, arithmetic, manner e.g. you can use map+reduce approach on your stream. To note, CUDA has built-in support for Map+Reduce+filter operations. However, testing performance on GPU based calculation is tricky (You can use manual calculation putting watcher around your code). $\endgroup$
    – user1361
    Aug 2, 2014 at 20:51
  • $\begingroup$ @Rubens Some patterns in Patterns for Parallel Programming book will probably solve your disk problem. $\endgroup$
    – user1361
    Aug 2, 2014 at 20:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.