I have graph data which has 250 million nodes and over 500 million edges. My files are roughly 230GB is size, this includes the node and edge files.

I am having major difficulties visualising the data. By visualisation I mean, generating aesthetic plots. The data has attributes such as node number, name, total money going in, total money going out and edges. I'd like to use such attributes to help generate colourful plots that signify groups and communities.

I have already tried NetworkX, Pandas, Neo4J, Gephi and R-Studio. All of which crash and are unable to do anything on my data.

Can anyone please recommend alternatives to create such visualisations?

  • $\begingroup$ Since you haven't told us anything about the nature of your data, the question probably is going to be hard to answer well. $\endgroup$ – D.W. Aug 10 '17 at 23:26
  • $\begingroup$ The data is from a blockchain, there number of columns are limited to 5, the rows are the problem. $\endgroup$ – Oonah Aug 11 '17 at 11:18
  • $\begingroup$ I don't understand what you mean by "columns". A graph doesn't have rows or columns; it has vertices and edges. We still would need to know a lot more. What do you want to know about the graph? What is the purpose of the visualization? What questions are you trying to answer about the data? Then, you should edit the question to include all relevant context in the question. We want questions to stand on their own, so people don't have to read the comments to find important information about your situation. Thank you! $\endgroup$ – D.W. Aug 11 '17 at 15:41

I'll assume you want an image that you could display on a standard 1920x1200 monitor. If every node was PERFECTLY packed and represented by only one pixel, you would have a white box that covered only 1% (2.3 million) of the nodes.

Subset your data to display to something reasonable like 1000 nodes using Neo4j and then you should get more reasonable performance.

| improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.