# Tag Info

## Hot answers tagged unsupervised-learning

26

For clustering, your data must be indeed integers. Moreover, since k-means is using euclidean distance, having categorical column is not a good idea. Therefore you should also encode the column timeOfDay into three dummy variables. Lastly, don't forget to standardize your data. This might be not important in your case, but in general, you risk that the ...

25

Well the names are pretty straight-forward and should give you a clear idea of vector representations. The Word2Vec Algorithm builds distributed semantic representation of words. There are two main approaches to training, Distributed Bag of Words and The skip gram model. One involves predicting the context words using a centre word, while the other ...

13

I don't think any of the clustering techniques "just" work at such scale. The most scalable supposedly is k-means (just do not use Spark/Mahout, they are really bad) and DBSCAN (there are some good distributed versions available). But you will be facing many other challenges besides scale because clustering is difficult. It's not as if it's just enough to ...

11

Clustering is difficult to do in high dimensions because the distance between most pairs of points is similar. Using an autoencoder lets you re-represent high dimensional points in a lower-dimensional space. It doesn't do clustering per se - but it is a useful preprocessing step for a secondary clustering step. You would map each input vector $x_i$ to a ...

10

SVM can be used for classification (distinguishing between several groups or classes) and regression (obtaining a mathematical model to predict something). They can be applied to both linear and non linear problems. Until 2006 they were the best general purpose algorithm for machine learning. I was trying to find a paper that compared many implementations ...

10

Yes, it makes sense to use CNNs with autoencoders or other unsupervised methods. Indeed, different ways of combining CNNs with unsupervised training have been tried for EEG data, including using (convolutional and/or stacked) autoencoders. Examples: Deep Feature Learning for EEG Recordings uses convolutional autoencoders with custom constraints to improve ...

9

Every algorithm that deals with text data has a vocabulary. In the case of word2vec, the vocabulary is comprised of all words in the input corpus, or at least those above the minimum-frequency threshold. Algorithms tend to ignore words that are outside their vocabulary. However there are ways to reframe your problem such that there are essentially no Out-Of-...

9

The purpose of topic modeling methods is to discover the latent themes (topics) assumed to have generated the documents of a corpus. Topic modeling methods are built on the distributional hypothesis, suggesting that similar words occur in similar contexts. To this end, they assume a generative process (a sequence of steps), which is a set of assumptions ...

9

I would say hierarchical clustering is usually preferable, as it is both more flexible and has fewer hidden assumptions about the distribution of the underlying data. With k-Means clustering, you need to have a sense ahead-of-time what your desired number of clusters is (this is the 'k' value). Also, k-means will often give unintuitive results if (a) ...

8

Take a look at these links- https://stats.stackexchange.com/questions/108743/methods-in-r-or-python-to-perform-feature-selection-in-unsupervised-learning http://www.jmlr.org/papers/volume5/dy04a/dy04a.pdf https://www.researchgate.net/post/What_is_the_best_unsupervised_method_for_feature_subset_selection

7

I faced almost exactly the same scenario a year and a half ago -- basically what you have is a variation of the one-class classification (OCC) problem, specifically PU-learning (learning from Positive and Unlabelled data). You have your known, labelled positive dataset (clients) and an un-labelled dataset of prospects (some of which are client-like and some ...

7

Overfitting happens when the model fits the training dataset more than it fits the underlying distribution. In a way, it models the specific sample rather than producing a more general model of the phenomena or underlying process. It can be presented using Bayesian methods. If I use Naive Bayes then I have a simple model that might not fit either the ...

7

To the best of my knowledge no. SVD and PCA are both linear dimensionality reduction algorithms. Some nonlinear dimensionality reduction algorithms are e.g. LLE, Kernel-PCA, Isomap, etc. About t-SNE I would like to add a point. It reduces the dimensionality (and does it pretty well!) but it is only for visualization and can not be used in learning process! ...

7

Some unsupervised models can make predictions, but not ones that necessarily match the original class labels. Once a GaussianMixture model has been fitted, it can predict which of the clusters a new example belongs to. This is exactly what the predict and predict_proba functions do in this case, and given that the number of clusters is set to 3, the number ...

6

To compare two LDA topics, you're really trying to compute the distance between two probability distributions. One such measure that's commonly used in these circumstances is the Hellinger Distance. To find the closest match for $x_1$ in the topics for $y$, you would calulate the Hellinger Distance between $x_1$ and each $y$ topic, then take the lowest one....

6

Imagine that your data is not easily separable. Your classifier isn't able to do a very good job at distinguishing between positive and negative examples, so it usually predicts the majority class for any example. In the unbalanced case, it will get 100 examples correct and 20 wrong, resulting in a 100/120 = 83% accuracy. But after balancing the classes, the ...

6

Accuracy is probably not a good metric for your problem. For the original dataset, if the model just makes a dummy prediction that all samples belong to the bigger class, the accuracy will be 83% (100/120). But that's usually not what we want to predict in an imbalanced dataset. Let's take a fraud detection problem. The probability that a transaction is a ...

6

Good question and welcome to Datascience Imagine you have the tree as follows. Machine Learning Models | ---------------------------------------------------- | | Supervised Unsupervised |...

6

Answering your question: yes, depending on the hyperparameters you choose, you could overfit the considered normal data, if you fit your separating hyperplane between normal and novel points being too much "shaped" on your input data. There are, for instance in case of one-class support vector machines, some important hyperparams like nu or gamma: ...

5

Look up taxonomy/ontology construction/induction. Relevant papers: Automatic Taxonomy Construction from Keywords via Scalable Bayesian Rose Trees Topic Models for Taxonomies OntoLearn Reloaded. A Graph-Based Algorithm for Taxonomy Induction Ontology Population and Enrichment: State of the Art Probabilistic Topic Models for Learning Terminological Ontologies

5

The main difference between supervised and unsupervised learning is the following: In supervised learning you have a set of labelled data, meaning that you have the values of the inputs and the outputs. What you try to achieve with machine learning is to find the true relationship between them, what we usually call the model in math. There are many ...

5

RBM's are an interesting beast. To answer your question, and to jog my memory on them, I'll derive RBMs and talk through the derivation. You mentioned that you're confused on the likelihood, so my derivation will be from the perspective of trying to maximize the likelihood. So let's begin. RBMs contain two different sets of neurons, visible and hidden, I'll ...

5

In addition to the existing answers, I would like to talk about this energy function, and the intuition behind that a bit. Sorry if this is a bit long and physical. The energy function describes a so-called Ising model, which is a model of ferromagnetism in terms of statistical mechanics / quantum mechanics. In statistical mechanics, we use a so-called ...

5

Typically to extract features, you can use the top layer of the network before the output. The intuition is that these features are linearly separable because the top layer is just a logistic regression. For GANs, you can use the features from the discriminator. These features are supposed to give a probability if the input came from the training dataset, "...

5

I have used stacked auto-encoders to reduce our 40 features step by step to 5 features and then output back to 40 features (some of my features were all zeros/ non deviating features). Training this on original (assumed to have no outliers) gives you a network which has learnt an abstract representation of the 40 features with 5 features. When outliers show ...

5

You should break this down one step further: retaining local structure and retaining global structure. Other well-understood methods, such as Principal Component Analysis are great at retaining global structure, because it looks at ways in which a dataset's variance is retained, globally, across the entire dataset. t-SNE works differently, by looking at ...

5

You can have a look here, where many open-source algorithms specifically for anomaly detection on time-series data (e.g. metrics) are collected, both for online of offline settings. Almost all of them are unsupervised approaches that require no labels to detect the anomalies. They also automatically handle some of the issues you mentioned, like ...

4

Semi-Supervised Learning The combination of unsupervised learning and supervised learning is referred to as semi-supervised learning, which is the concept that I believe you are searching for. Label propagation is often cited when outlining the heuristics of semi-supervised learning. The essence is to employ clustering, but to use a tiny set of known ...

4

Likelihood-ratio tests are a mainstay of classical hypothesis testing. The idea is to form the likelihoods of the two hypotheses under consideration, and choose the one with the highest likelihood if their ratio is sufficiently large. Hypotheses come in two flavors: simple, and composite. Simple tests are those for which the hypothesis uniquely defines the ...

4

There are couple of ways to deal with missing data. Replace missing values by mean/median. If the missing values is very less, then this method would be apt. Also depends on how skewed your data is. Imputation. Build a linear regression model to predict the missing values based on other parameters. KNN could also be used to predict the missing value Also ...

Only top voted, non community-wiki answers of a minimum length are eligible