Brian Spiering
  • Member for 7 years, 7 months
  • Last seen this week
What are graph embedding?
Accepted answer
22 votes

Graph embedding learns a mapping from a network to a vector space, while preserving relevant network properties. Vector spaces are more amenable to data science than graphs. Graphs contain edges and ...

View answer
One Hot Encoding vs Word Embedding - When to choose one or another?
Accepted answer
18 votes

One-Hot Encoding is a general method that can vectorize any categorical features. It is simple and fast to create and update the vectorization, just add a new entry in the vector with a one for each ...

View answer
Using TF-IDF with other features in scikit-learn
Accepted answer
12 votes

scikit-learn's FeatureUnion concatenates features from different vectorizers. An example of combining heterogeneous data, including text, can be found here.

View answer
What is Hellinger Distance and when to use it?
11 votes

Hellinger distance is a metric to measure the difference between two probability distributions. It is the probabilistic analog of Euclidean distance. Given two probability distributions, $P$ and $Q$, ...

View answer
Sentence similarity prediction
9 votes

Word Mover’s Distance (WMD) is an algorithm for finding the distance between sentences. WMD is based on word embeddings (e.g., word2vec) which encode the semantic meaning of words into dense vectors. ...

View answer
Clustering geo location coordinates (lat,long pairs)
9 votes

GPS coordinates can be directly converted to a geohash. Geohash divides the Earth into "buckets" of different size based on the number of digits (short Geohash codes create big areas and longer codes ...

View answer
SVM on sparse data
Accepted answer
8 votes

Support Vector Machines (SVM) represent data examples as points in space and tries to create a mapping with a wide as possible gap between the separate categories. The data examples closest to the gap ...

View answer
Can machine learning learn a function like finding maximum from a list?
7 votes

Yes - Machine learning can learn to find the maximum in a list of numbers. Here is a simple example of learning to find the index of the maximum: import numpy as np from sklearn.tree import ...

View answer
What is the BLEU score used in Google Brain's "Attention Is All You Need" paper?
Accepted answer
7 votes

BLEU (Bi Lingual Evaluation Understudy) is an algorithm for evaluating the quality of text which has been machine-translated (MT) from one natural language to another. BLEU is typically measured on a ...

View answer
Why does Keras need TensorFlow as backend?
6 votes

Keras is an application programming interface (API). It is a single interface that can support multi-backends, which means a programmer can write Keras code once and it can be executed in a variety of ...

View answer
Text similarity with sentence embeddings
Accepted answer
6 votes

One approach is using Word Mover’s Distance (WMD). WMD is an algorithm for finding the distance between texts of different lengths, where each word is represented as a word embedding vector. The ...

View answer
How to do spelling correction for a language but also correct some words in other language
Accepted answer
6 votes

There are many ways to build a spell corrector. One of the simplest is: Detect an incorrect word Generate candidate suggestions Score and rank the candidate replacements For detecting an incorrect ...

View answer
Splitting a sentence to meaningful parts
6 votes

You are looking to create a parse tree to find multi-token clauses. Here is code to generate a parse tree: import spacy from nltk import Tree nlp = spacy.load('en') def to_nltk_tree(node): ...

View answer
How to create interactive plot of thousands of images as output of t-SNE?
6 votes

Datashader is a Python visualization library designed to handle large datasets. A tutorial to plot t-SNE with datashader can be found here.

View answer
What is the rationale for discretization of continuous features and when should it be done?
6 votes

One reason to discretize continuous features is to improve signal-to-noise ratio. Fitting a model to bins reduces the impact that small fluctuates in the data has on the model, often small fluctuates ...

View answer
When using KNeighborsClassifier, what is the motivation of using weights="distance"?
5 votes

weights = 'distance' is in contrast to the default which is weights = 'uniform'. When weights are uniform, a simple majority vote of the nearest neighbors is used to assign cluster membership. When ...

View answer
Very Fast Training After First Epoch
Accepted answer
5 votes

Keras supports lazy execution. The create_model and model.compile code are not executed until it is absolutely required which is right before the first training epoch. That increased time for the ...

View answer
Determine statistically whether new product cannibalise old product by using data
5 votes

You do not have the data to directly answer the question if a ride for a given service is new customer or old customer. You need to have a customer id to properly attribute if there is new growth or ...

View answer
can I use t-sne or PCA to reduce number of classes?
Accepted answer
5 votes

No. t-Distributed Stochastic Neighbor Embedding (t-SNE) and Principal Component Analysis (PCA) are dimension reduction techniques, aka fewer columns of a tidy dataframe. Clustering will reduce the ...

View answer
How to find out if two datasets are close to each other?
Accepted answer
5 votes

You could take an Information Theory approach by finding the lowest Kullback–Leibler divergence between the distributions. There is a KL divergence option within SciPy's entropy function. >>>...

View answer
Are there any good out-of-the-box language models for python?
5 votes

The spaCy package has many language models, including ones trained on Common Crawl. Language model has a specific meaning in Natural Language Processing (NlP). A language model is a probability ...

View answer
Are dimensionality reduction techniques useful in deep learning
5 votes

Deep learning does not use dimensionality reduction because deep learning itself is a useful dimensionality reduction technique. Deep learning learns a compressed, nonlinear representation of the data ...

View answer
Continuous variable to categorical by quartiles?
5 votes

Panda's Categorical Data Type is designed for that type of analysis, pandas.cut can divide by user-defined bins and pandas.qcut can create quantile-based discretization. Something like this: import ...

View answer
Business exception reporting
5 votes

Try exploring the rich field of "Anomaly Detection in Time Series". Control charts and CUSUMs (or cumulative sum control charts) might help you. Simple Bullet Graphs might be all you need. Based on ...

View answer
What is the difference between TextVectorization and Tokenizer?
4 votes

Tokenization is the process of splitting a stream of language into individual tokens. Vectorization is the process of converting string data into a numerical representation.

View answer
Is it possible to use a Neural Network to interpolate data?
4 votes

It is possible. Creating higher resolution imaging with deep learning has been done in several fields. Medical imaging is one of the most common fields. The general approach is called "learning-...

View answer
When to use bayesian linear regression instead of linear regression?
4 votes

The Bayesian approach should be used in the case of: Strong priors - You have preexisting data and / or domain knowledge that you want to incorporate into the analysis. Distributional estimates - ...

View answer
difference between supervised learning and imitation learning
4 votes

In supervised learning each data example has a label. Imitation learning is mapping from observations to actions and is generally considered part of reinforcement learning. The primary difference ...

View answer
How to classify a new email as spam/not spam?
Accepted answer
4 votes

I modified your code so the code runs as a block and is setup to predict new data: import string from nltk.corpus import stopwords import pandas as pd from sklearn.feature_extraction.text import ...

View answer
How to include categorical fields to enhance a text classification
Accepted answer
4 votes

Scikit-learn has compose.ColumnTransformer which allows different columns or column subsets of the input to be transformed separately and the features generated by each transformer will be ...

View answer
1
2 3 4 5
39