27 votes
Accepted

Word2Vec vs. Sentence2Vec vs. Doc2Vec

Well the names are pretty straight-forward and should give you a clear idea of vector representations. The Word2Vec Algorithm builds distributed semantic representation of words. There are two main ...
user avatar
  • 1,728
26 votes
Accepted

Confused about how to apply KMeans on my a dataset with features extracted

For clustering, your data must be indeed integers. Moreover, since k-means is using euclidean distance, having categorical column is not a good idea. Therefore you should also encode the column ...
user avatar
  • 1,641
14 votes
Accepted

How can autoencoders be used for clustering?

Clustering is difficult to do in high dimensions because the distance between most pairs of points is similar. Using an autoencoder lets you re-represent high dimensional points in a lower-...
user avatar
  • 2,118
13 votes
Accepted

Clustering high dimensional data

I don't think any of the clustering techniques "just" work at such scale. The most scalable supposedly is k-means (just do not use Spark/Mahout, they are really bad) and DBSCAN (there are some good ...
user avatar
12 votes
Accepted

What kinds of learning problems are suitable for Support Vector Machines?

SVM can be used for classification (distinguishing between several groups or classes) and regression (obtaining a mathematical model to predict something). They can be applied to both linear and non ...
user avatar
11 votes

is it possible to do feature selection for unsupervised machine learning problems?

Take a look at these links- https://stats.stackexchange.com/questions/108743/methods-in-r-or-python-to-perform-feature-selection-in-unsupervised-learning http://www.jmlr.org/papers/volume5/dy04a/...
user avatar
  • 1,729
10 votes
Accepted

Does it make sense to train a CNN as an autoencoder?

Yes, it makes sense to use CNNs with autoencoders or other unsupervised methods. Indeed, different ways of combining CNNs with unsupervised training have been tried for EEG data, including using (...
user avatar
10 votes

What is the difference between topic modeling and clustering?

The purpose of topic modeling methods is to discover the latent themes (topics) assumed to have generated the documents of a corpus. Topic modeling methods are built on the distributional hypothesis, ...
user avatar
  • 411
10 votes
Accepted

K-Means vs hierarchical clustering

I would say hierarchical clustering is usually preferable, as it is both more flexible and has fewer hidden assumptions about the distribution of the underlying data. With k-Means clustering, you ...
user avatar
  • 2,118
9 votes

How word2vec can be used to identify unseen words and relate them to already trained data

Every algorithm that deals with text data has a vocabulary. In the case of word2vec, the vocabulary is comprised of all words in the input corpus, or at least those above the minimum-frequency ...
user avatar
  • 3,017
8 votes
Accepted

Gaussian Mixture Models as a classifier?

Some unsupervised models can make predictions, but not ones that necessarily match the original class labels. Once a GaussianMixture model has been fitted, it can ...
user avatar
  • 3,780
7 votes
Accepted

Ideas for prospect scoring model

I faced almost exactly the same scenario a year and a half ago -- basically what you have is a variation of the one-class classification (OCC) problem, specifically PU-learning (learning from Positive ...
user avatar
7 votes
Accepted

Is Overfitting a problem in Unsupervised learning?

Overfitting happens when the model fits the training dataset more than it fits the underlying distribution. In a way, it models the specific sample rather than producing a more general model of the ...
user avatar
  • 2,583
7 votes
Accepted

Is SVD non-linear while PCA (by eigendecompostion) is linear?

To the best of my knowledge no. SVD and PCA are both linear dimensionality reduction algorithms. Some nonlinear dimensionality reduction algorithms are e.g. LLE, Kernel-PCA, Isomap, etc. About t-SNE ...
user avatar
6 votes
Accepted

Combine two sets of clusters

To compare two LDA topics, you're really trying to compute the distance between two probability distributions. One such measure that's commonly used in these circumstances is the Hellinger Distance. ...
user avatar
6 votes
Accepted

Why will the accuracy of a highly unbalanced dataset reduce after oversampling?

Imagine that your data is not easily separable. Your classifier isn't able to do a very good job at distinguishing between positive and negative examples, so it usually predicts the majority class for ...
user avatar
  • 3,780
6 votes

Why will the accuracy of a highly unbalanced dataset reduce after oversampling?

Accuracy is probably not a good metric for your problem. For the original dataset, if the model just makes a dummy prediction that all samples belong to the bigger class, the accuracy will be 83% (...
user avatar
  • 526
6 votes

Confused about the different aspects in Machine Learning

Good question and welcome to Datascience Imagine you have the tree as follows. ...
user avatar
  • 161
6 votes
Accepted

What does it mean by “t-SNE retains the structure of the data”?

You should break this down one step further: retaining local structure and retaining global structure. Other well-understood methods, such as Principal Component Analysis are great at retaining ...
user avatar
  • 13.9k
6 votes

How to do feature selection for clustering and implement it in python?

Often people confuse unsupervised feature selection (UFS) and dimensionality reduction (DR) algorithms as the same. For instance, a famous DR algorithm is Principal Component Analysis (PCA) which is ...
user avatar
  • 391
6 votes
Accepted

Can a novelty detection model overfit?

Answering your question: yes, depending on the hyperparameters you choose, you could overfit the considered normal data, if you fit your separating hyperplane between normal and novel points being too ...
user avatar
  • 2,399
6 votes

How to do clustering assuring more than one class per cluster?

This is a very strange design: The goal is to train an ensemble classification model. In general there is no strong reason to use only subsets of the data to train the individual learners, let alone ...
user avatar
  • 21.8k
5 votes
Accepted

supervised learning and labels

The main difference between supervised and unsupervised learning is the following: In supervised learning you have a set of labelled data, meaning that you have the values of the inputs and the ...
user avatar
5 votes
Accepted

Inferring Relational Hierarchies of Words

Look up taxonomy/ontology construction/induction. Relevant papers: Automatic Taxonomy Construction from Keywords via Scalable Bayesian Rose Trees Topic Models for Taxonomies OntoLearn Reloaded. A ...
user avatar
  • 10.4k
5 votes

What kinds of learning problems are suitable for Support Vector Machines?

Let's assume that we are in a classification setting. For svm feature engineering is cornerstone: the sets have to be linearly separable. Otherwise the data ...
user avatar
5 votes
Accepted

Intuition Behind Restricted Boltzmann Machine (RBM)

RBM's are an interesting beast. To answer your question, and to jog my memory on them, I'll derive RBMs and talk through the derivation. You mentioned that you're confused on the likelihood, so my ...
user avatar
5 votes

Intuition Behind Restricted Boltzmann Machine (RBM)

In addition to the existing answers, I would like to talk about this energy function, and the intuition behind that a bit. Sorry if this is a bit long and physical. The energy function describes a so-...
user avatar
  • 1,094
5 votes

How to use GAN for unsupervised feature extraction from images?

Typically to extract features, you can use the top layer of the network before the output. The intuition is that these features are linearly separable because the top layer is just a logistic ...
user avatar
  • 435
5 votes
Accepted

What is the meaning of spherical dataset?

In this case, a picture is a worth a thousand words. They literally mean data whose distribution on X,Y is roughly a sphere. Different clustering algorithms work better on different distributions. For ...
user avatar
  • 1,633
5 votes
Accepted

Unsupervised feature reduction for anomaly detection with autoencoders

I have used stacked auto-encoders to reduce our 40 features step by step to 5 features and then output back to 40 features (some of my features were all zeros/ non deviating features). Training this ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible