1
$\begingroup$

I am on a project dealing with a lot of data in the form of images and videos (Data related to wind engineering). My requirement is to build a predictive algorithm based on the data I have. I have found many tools with which I can analyse the data where each tool has its own advantages and disadvantages. Big data being really new to me, I find it very difficult to choose a platform to start with. There should be other people here who should have dealt with similar situations.

  • What criteria should I mainly take into account before selecting a tool for analysing big data?

Some of the Criteria that I have taken into account : Visualization, Interaction, Security, Data Access and integration, Speed of response, Integrated Data Mining, Pattern Matching, Ease of use etc. As you can see the list that I have made for the criteria comes from the extensive reading of different articles on the topic. But I can't narrow down the list nor find the individual contribution of these criteria in the various tools available for analysis.

Let me also list some of the tools that I found after googling : Knime, Statistica 2, Rapidminer, Orange, WEKA, KEEL, R and RATTLE.

On what basis could I choose a tool from a list of tools that perform similar tasks?

UPDATE based on Comment

Aim : To develop a software that analyses the data coming from the wind mills and generate reports. The software should be able to predict when a wind mill can fail based on the analysis.

The project is still in the phase of gathering User Requirements. Maybe i am so early to come into conclusions about what tool should be used.

Someone else suggested that I should be finalise the requirements and then think about a tool that can help me get things done. So is it possible that I find what and how things should be analysed before finding a tool? And is it also possible that I find an algorithm for predictive analysis without knowing what would be the results of the tool after analysis.

$\endgroup$
  • 1
    $\begingroup$ Is this not the right place for me to ask this question? Why is the question downvoted? $\endgroup$ – Vini Nov 25 '15 at 14:05
  • 2
    $\begingroup$ This is the right place, provided you ask correctly. See How to Ask. Perhaps if you told us what features you liked/disliked about those you googled? What operating system? Free, or do you have a budget? The more (specific) information you can give us, the more that we can help you. I reversed that downvote, sicne you are new, but please help us to help you in future & give more detail in this question if you can (did you ask your professor?) $\endgroup$ – Mawg Nov 25 '15 at 14:22
  • 1
    $\begingroup$ Ok. I will surely update my question with all those details. Thanks for the feedback. $\endgroup$ – Vini Nov 25 '15 at 14:25
  • 1
    $\begingroup$ Welcome to the site! Good luck with the project :) $\endgroup$ – Dawny33 Nov 26 '15 at 10:49
1
$\begingroup$

What criteria should I mainly take into account before selecting a tool for analysing big data?

There are a lot of criterion which are to be taken into account when the tool selection is concerned. The can be:

  • Structure of the data. (The data model Ex: Hierarchical, tabular, etc)
  • Type of data and what is the problem statement. (time series, or classification, etc)
  • Speed
  • Security

Aim : To develop a software that analyses the data coming from the wind mills and generate reports. The software should be able to predict when a wind mill can fail based on the analysis.

Almost all the existing analytics tools like Python, Julia, R, etc can do this.

And is it also possible that I find an algorithm for predictive analysis without knowing what would be the results of the tool after analysis.

Yes. The predictive algorithm or technique can be inferred by looking at the data and the contents of the data. It is not dependant on the tool.

Some points which I would like to include which I believe would be useful to you:

  • Select the database depending on your data, and it's type. According to your data, a NoSQL database would be more relevant and suitable.
  • Select the algorithms and techniques only after you have a clear knowledge about the problem statement and takeaways and also after clearly looking at the data for an exploratory analysis.
  • If you want more flexibility, then use a tool/programming language like Python, R and Julia. Else, you can use a tool like Knime, Orange (it has a Python library too.), RapidMiner, etc.
|improve this answer|||||
$\endgroup$
  • $\begingroup$ Thanks for your answer. i will discuss your points with my team. :) $\endgroup$ – Vini Nov 26 '15 at 10:48
3
$\begingroup$

Syncfusion provides a Big data platform which is an easy to use Hadoop distribution for Windows. It can help you get started quickly. Syncfusion also provides a PMML processing library using which you can execute predictive analytic models. There is a Dashboard platform also available that can help visualize the data.

All of the above are available for free through the community license if you qualify.

Note: I work for Syncfusion.

|improve this answer|||||
$\endgroup$
  • $\begingroup$ Thanks for your answer. I will check about Syncfusion today. $\endgroup$ – Vini Nov 25 '15 at 14:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.