8
$\begingroup$

I'm looking to graph and interactively explore live/continuously measured data. There are quite a few options out there, with plot.ly being the most user-friendly. Plot.ly has a fantastic and easy to use UI (easily scalable, pannable, easily zoomable/fit to screen), but cannot handle the large sets of data I'm collecting. Does anyone know of any alternatives?

I have MATLAB, but don't have enough licenses to simultaneously run this and do development at the same time. I know that LabVIEW would be a great option, but it is currently cost-prohibitive.

Thanks in advance!

$\endgroup$
  • $\begingroup$ You have mentioned neither types of data you collect, nor its volume's order of magnitude. This information would be helpful in providing a more targeted advice. $\endgroup$ – Aleksandr Blekh Dec 18 '14 at 2:35
7
$\begingroup$

For this answer, I have assumed that you prefer open source solutions to big data visualization. This assumption is based on budgetary details from your question. However, there is one exclusion to this - below I will add a reference to one commercial product, which I believe might be beneficial in your case (provided that you could afford that). I also assume that browser-based solutions are acceptable (I would even prefer them, unless you have specific contradictory requirements).

Naturally, the first candidate as a solution to your problem I would consider D3.js JavaScript library: http://d3js.org. However, despite flexibility and other benefits, I think that this solution is too low-level.

Therefore, I would recommend you to take a look at the following open source projects for big data visualization, which are powerful and flexible enough, but operate at a higher level of abstraction (some of them are based on D3.js foundation and sometimes are referred to as D3.js visualization stack).

  • Bokeh - Python-based interactive visualization library, which supports big data and streaming data: http://bokeh.pydata.org
  • Flot - JavaScript-based interactive visualization library, focused on jQuery: http://www.flotcharts.org
  • NodeBox - unique rapid data visualization system (not browser-based, but multi-language and multi-platform), based on generative design and visual functional programming: https://www.nodebox.net
  • Processing - complete software development system with its own programming language, libraries, plug-ins, etc., oriented to visual content: https://www.processing.org (allows executing Processing programs in a browser via http://processingjs.org)
  • Crossfilter - JavaScript-based interactive visualization library for big data by Square (very fast visualization of large multivariate data sets): http://square.github.io/crossfilter
  • bigvis - an R package for big data exploratory analysis (not a visualization library per se, but could be useful to process large data sets /aggregating, smoothing/ prior to visualization, using various R graphics options): https://github.com/hadley/bigvis
  • prefuse - Java-based interactive visualization library: http://prefuse.org
  • Lumify - big data integration, analysis and visualization platform (interesting feature: supports Semantic Web): http://lumify.io

Separately, I'd like to mention two open source big data analysis and visualization projects, focused on graph/network data (with some support for streaming data of that type): Cytoscape and Gephi. If you are interested in some other, more specific (maps support, etc.) or commercial (basic free tiers), projects and products, please see this awesome compilation, which I thoroughly curated to come up with the main list above and analyzed: http://blog.profitbricks.com/39-data-visualization-tools-for-big-data.

Finally, as I promised in the beginning, Zoomdata - a commercial product, which I thought you might want to take a look at: http://www.zoomdata.com. The reason I made an exclusion for it from my open source software compilation is due to its built-in support for big data platforms. In particular, Zoomdata provides data connectors for Cloudera Impala, Amazon Redshift, MongoDB, Spark and Hadoop, plus search engines, major database engines and streaming data.

Disclaimer: I have no affiliation with Zoomdata whatsoever - I was just impressed by their range of connectivity options (which might cost you dearly, but that's another aspect of this topic's analysis).

$\endgroup$
  • 1
    $\begingroup$ Thanks so much Aleksandr Blekh & Nitesh for your insight. I'll definitely look into these options. One thing i should have been more clear about is that i'm capturing live data from a laser-micrometer, sampling at about 60 samples per second. Right after i asked this question, I found a software called KST (kst-plot.kde.org). It works nicely except that it isn't stable and isn't particularly easy to pull older data, zoom in/out, etc. $\endgroup$ – Clayton Pipkin Dec 22 '14 at 23:13
  • $\begingroup$ @ClaytonPipkin: You're very welcome. I'm glad to help. Don't forget to upvote both answers and accept the best, if satisfied. KST looks interesting, but keep in mind that it's an application, hence, it doesn't have a native ability to visualize data for the Web. $\endgroup$ – Aleksandr Blekh Dec 23 '14 at 0:38
  • $\begingroup$ @AleksandrBlekh No one up voted my answer :( But your answer is very satisfying. I am going to up vote it! $\endgroup$ – Nitesh Jun 3 '15 at 3:05
  • $\begingroup$ @Nitesh: Thank you for kind words and upvoting. Don't worry too much about your particular answer - it is not bad, just rather limited in scope. Nevertheless, I'll upvote it for the content and to cheer you up a bit :-). $\endgroup$ – Aleksandr Blekh Jun 3 '15 at 3:45
1
$\begingroup$

Visualizing large datasets is a long standing problem. One of the issues is to understand how we can show over a million points on a screen that has only about ~ million pixels.

Having said that, here are a few tools that can handle big data:

  1. Tableau: you could use their free desktop tool.
  2. Tabplot: built on top of ggplot2 in R to handle larger datasets.
  3. See this review for 5 other products that can help you do your job.
$\endgroup$
0
$\begingroup$

If you are using python, I would suggest using mpld3 which combines D3js javascript visualizations with matplotlib of python.

The installation and usage is really simple and it has some cool plugins and interactive stuffs.

http://mpld3.github.io/

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.