Hot answers tagged

27

What are graph Embeddings ? "Graph Embeddings" is a hot area today in machine learning. It basically means finding "latent vector representation" of graphs which captures the topology (in very basic sense) of the graph. We can make this "vector representation" rich by also considering the vertex-vertex relationships, edge-information etc. There are roughly ...


21

Graph embedding learns a mapping from a network to a vector space, while preserving relevant network properties. Vector spaces are more amenable to data science than graphs. Graphs contain edges and nodes, those network relationships can only use a specific subset of mathematics, statistics, and machine learning. Vector spaces have a richer toolset from ...


20

I also suggest Gephi software (https://gephi.github.io), which seems to be quite powerful. Some additional information on using Gephi with large networks can be found here and, more generally, here. Cytoscape (http://www.cytoscape.org) is an alternative to Gephi, being an another popular platform for complex network analysis and visualization. If you'd like ...


15

Feeding your column names into the y values argument as a list works for me like so: total_year[-15:].plot(x='year', y=['action', 'comedy'], figsize=(10,5), grid=True) Using something like the answer at this link is better and gives you way more control over the labels and whatnot: adding lines with plt.plot()


8

https://gephi.github.io/ says it can handle a million edges. If your graph has 1000000 vertices and only 50000 edges then most of your vertices won't have any edges anyway. In fact the Gephi spec is the dual of your example: "Networks up to 50,000 nodes and 1,000,000 edges"


8

I think, that Gephi could face with lack-of-memory issues, you will need at least 8Gb of RAM. Though number of edges is not extremely huge. Possibly, more appropriate tool in this case would be GraphViz. It's a command line tool for network visualizations, and presumably would be more tolerant to graph size. Moreover, as I remember, in GraphViz it is ...


7

Neo4j and Spark GraphX are meant for solving problem at different level and they are complimentary to each other. They can be connected by Neo4j's Mazerunner extension: Mazerunner is a Neo4j unmanaged extension and distributed graph processing platform that extends Neo4j to do big data graph processing jobs while persisting the results back to Neo4j. ...


6

Strange as it sounds, graphs and graph databases are typically implemented as linked lists. As alluded to here, even the most popular/performant graph database out there (neo4j), is secretly using something akin to a doubly-linked list. Representing a graph this way has a number of significant benefits, but also a few drawbacks. Firstly, representing a ...


6

I would add a third column called month and then concatenate each list. So if you have a top 100 list for 5 months you will create one big table with 500 entries: User-id | Threat_score | month aaa 45 1 bbb 32 1 ccc 20 1 ... ... ... bbb 64 2 ccc 29 2 .....


6

Good , old and unsolved question! Distributed processing of large graphs as far as I know (speaking as a graph guy) has 2 different approaches, with the knowledge of Big Data frameworks or without it. SNAP library from Jure Leskovec group at Stanford which is originally in C++ but also has a Python API (please check if you need to use C++ API or Python ...


6

TL;DR Use the two functions from below to get the index of the elbow: elbow_index = find_elbow(data, get_data_radiant(data)) Edit: I put all of the code below into a python package called kneebow. Now, you can simply do it like this: from kneebow.rotor import Rotor rotor = Rotor() rotor.fit_rotate(data) elbow_index = rotor.get_elbow_index() Long Answer ...


5

What you are looking for can be found in KONECT (the website is down as I'm writing this but it should be fixed soon!). It's almost the most comprehensive data collection for network analysis. But the question is which one is more standard to use? Well, there is no clear answer except of Zachary's Karate Club! If you do a literature review in Community ...


5

Yes There are! Networkx I think 20k-30k node-edge would be OK on Networkx, IF YOU HAVE A GOOD MACHINE! Networkx is a great library in Python particularly for Graph Analysis so you have access to great analysis tools beside visualizing but visualizing 20k vertices needs much RAM and takes long. Igraph Igraph is another great tool for Graph Analysis with ...


5

Community Detection and Clique Percolation: This is a community detection problem. Here is a very detailed review article surveying the state of the art. The Clique Percolation Method is also useful to explore as it pretty much solves what you may need to know. You can also go through what Matching is, and link the concept with Blossom Algorithm. Though, ...


4

Maybe you can check here - http://snap.stanford.edu/data/ For each data set you will also see references of the works where they have been used


4

Classifiers often return probabilities of belonging to a class. For example in logistic regression the predicted values are the predicted probability of belonging to the non-reference class or $\text{Pr}(Y = 1)$. The discrimination threshold is just the cutoff imposed on the predicted probabilities for assigning observations to each class.


4

Reporting back: I ended up coding graphml and using yEd for visualization (just because I am familiar with this combination. I bet gephi or graphviz would work fine and might even be better). Since I computed the location of all nodes, memory was not such big of an issue. Coding graphml is a little easier comparing to coding svg, since I don't have to ...


4

Just to add a bit. Like it was mentioned before, if you have a classifier (probabilistic) your output is a probability (a number between 0 and 1), ideally you want to say that everything larger than 0.5 is part of one class and anything less than 0.5 is the other class. But if you are classifying cancer rates, you are deeply concerned with false negatives (...


4

Based on what I figured out from your problem: 1 You can easily convert your data to a graph using Networkx, igraph or any other tool/library/software. Then what you need is a Shortest Path Algorithm (Dijkstra is widely used and implemented in all graph/network analysis softwares). Once you created the graph you can simply calculate the average estimated ...


4

I can only speak about Graphs: Advantages: Using graphs, you can easily find products bought/rated by users that bought or liked an item, or users that have similar "taste" to another user. From my experience the traversal process is fast enough. Closely matched products are easy to find, depending on how you model your graph: e.g. (userA)-[]->(basketA)-...


4

The concept is the same but you are getting confused by the type of data. Spectral Clustering as Ng et al. explain is about clustering standard data while the Laplacian matrix is a graph derived matrix used in algebraic graph theory. So the point is that whenever you encode the similarity of your objects into a matrix, this matrix could be used for spectral ...


4

There seem to be a few options, but I found rasterfairy which is very easy to install and use. Has the added bonus of being able to fit to a rectangular grid, but also circular and other arbitrary shapes. A very nice IronPython notebook example: https://github.com/Quasimondo/RasterFairy/blob/master/examples/Raster%20Fairy%20Demo%201.ipynb And some example ...


4

Your question is not clear in a way there are two different Graph Clustering problems. One is having a dataset of different graphs and you would like to cluster similar graphs (in this case each object is a graph), and the other when you have a graph (e.g. a social network) and you would like to group similar nodes inside that graph (here each object is a ...


4

There are many use cases of graph theory in Finance industry and it is a very broad question. As Emre said can be used for Fraud Detection, Risk Modelling, Economic Networks etc. These below links can give you better understanding of different application, please go through for better understanding: Applications of Graph Theory In Finance Graph Theory for ...


3

Using Community detection you can build a recommendation system. The most commonly used algorithm in this field is Blondel Algorithm which u have probably seen in SNAP. Blondel is almost the fastest Community Detection algorithm among widely accepted ones and its result is pretty acceptable (at least according to modularity score). As a side comment, you may ...


3

Speaking as a Graph/Complex Networks guy I'd recommend Networkx package in Python. This is the main library I used for my master thesis and my research during last 2 years. As long as your graph is not gigantic (millions of nodes) you can handle it using Networkx. But what you need is not only the library but a philosophy to convert your data into a graph. ...


3

Even a simple Internet search reveals numerous papers on graph clustering approaches and algorithms. This paper is most likely the best starting point, as it presents a rather comprehensive overview of the topic in terms of the problem as well as approaches, methods and algorithms for solutions. The rest you can find easily via online search. In regard to ...


3

I have found some solution and will post it here, because somebody, who works with graphlab, can have the same question. We can look at the example here: Six degrees of Kevin Bacon At te beginning of the program execution you need to run next command: graphlab.canvas.set_target('ipynb') Exactly this is a key of the whole problem (at least by me:-) At ...


3

Having worked with Facebook data a bit (harvested from Facebook users) we stored it just as a pair of values: USER_ID, FRIEND_USER_ID. But I guess your questions is a bit deeper? You can store it in different ways, depending on your research question. One interesting option is triads for example - http://mypersonality.org/wiki/doku.php?id=...


Only top voted, non community-wiki answers of a minimum length are eligible