24

For clustering, your data must be indeed integers. Moreover, since k-means is using euclidean distance, having categorical column is not a good idea. Therefore you should also encode the column timeOfDay into three dummy variables. Lastly, don't forget to standardize your data. This might be not important in your case, but in general, you risk that the ...


22

Well the names are pretty straight-forward and should give you a clear idea of vector representations. The Word2Vec Algorithm builds distributed semantic representation of words. There are two main approaches to training, Distributed Bag of Words and The skip gram model. One involves predicting the context words using a centre word, while the other ...


12

I don't think any of the clustering techniques "just" work at such scale. The most scalable supposedly is k-means (just do not use Spark/Mahout, they are really bad) and DBSCAN (there are some good distributed versions available). But you will be facing many other challenges besides scale because clustering is difficult. It's not as if it's just enough to ...


11

Clustering is difficult to do in high dimensions because the distance between most pairs of points is similar. Using an autoencoder lets you re-represent high dimensional points in a lower-dimensional space. It doesn't do clustering per se - but it is a useful preprocessing step for a secondary clustering step. You would map each input vector $x_i$ to a ...


9

Every algorithm that deals with text data has a vocabulary. In the case of word2vec, the vocabulary is comprised of all words in the input corpus, or at least those above the minimum-frequency threshold. Algorithms tend to ignore words that are outside their vocabulary. However there are ways to reframe your problem such that there are essentially no Out-Of-...


8

SVM can be used for classification (distinguishing between several groups or classes) and regression (obtaining a mathematical model to predict something). They can be applied to both linear and non linear problems. Until 2006 they were the best general purpose algorithm for machine learning. I was trying to find a paper that compared many implementations ...


8

Yes, it makes sense to use CNNs with autoencoders or other unsupervised methods. Indeed, different ways of combining CNNs with unsupervised training have been tried for EEG data, including using (convolutional and/or stacked) autoencoders. Examples: Deep Feature Learning for EEG Recordings uses convolutional autoencoders with custom constraints to improve ...


8

The purpose of topic modeling methods is to discover the latent themes (topics) assumed to have generated the documents of a corpus. Topic modeling methods are built on the distributional hypothesis, suggesting that similar words occur in similar contexts. To this end, they assume a generative process (a sequence of steps), which is a set of assumptions ...


7

I faced almost exactly the same scenario a year and a half ago -- basically what you have is a variation of the one-class classification (OCC) problem, specifically PU-learning (learning from Positive and Unlabelled data). You have your known, labelled positive dataset (clients) and an un-labelled dataset of prospects (some of which are client-like and some ...


7

I would say hierarchical clustering is usually preferable, as it is both more flexible and has fewer hidden assumptions about the distribution of the underlying data. With k-Means clustering, you need to have a sense ahead-of-time what your desired number of clusters is (this is the 'k' value). Also, k-means will often give unintuitive results if (a) ...


7

To the best of my knowledge no. SVD and PCA are both linear dimensionality reduction algorithms. Some nonlinear dimensionality reduction algorithms are e.g. LLE, Kernel-PCA, Isomap, etc. About t-SNE I would like to add a point. It reduces the dimensionality (and does it pretty well!) but it is only for visualization and can not be used in learning process! ...


6

Overfitting happens when the model fits the training dataset more than it fits the underlying distribution. In a way, it models the specific sample rather than producing a more general model of the phenomena or underlying process. It can be presented using Bayesian methods. If I use Naive Bayes then I have a simple model that might not fit either the ...


6

To compare two LDA topics, you're really trying to compute the distance between two probability distributions. One such measure that's commonly used in these circumstances is the Hellinger Distance. To find the closest match for $x_1$ in the topics for $y$, you would calulate the Hellinger Distance between $x_1$ and each $y$ topic, then take the lowest one....


6

Imagine that your data is not easily separable. Your classifier isn't able to do a very good job at distinguishing between positive and negative examples, so it usually predicts the majority class for any example. In the unbalanced case, it will get 100 examples correct and 20 wrong, resulting in a 100/120 = 83% accuracy. But after balancing the classes, the ...


6

Take a look at these links- https://stats.stackexchange.com/questions/108743/methods-in-r-or-python-to-perform-feature-selection-in-unsupervised-learning http://www.jmlr.org/papers/volume5/dy04a/dy04a.pdf https://www.researchgate.net/post/What_is_the_best_unsupervised_method_for_feature_subset_selection


6

Good question and welcome to Datascience Imagine you have the tree as follows. Machine Learning Models | ---------------------------------------------------- | | Supervised Unsupervised |...


5

Look up taxonomy/ontology construction/induction. Relevant papers: Automatic Taxonomy Construction from Keywords via Scalable Bayesian Rose Trees Topic Models for Taxonomies OntoLearn Reloaded. A Graph-Based Algorithm for Taxonomy Induction Ontology Population and Enrichment: State of the Art Probabilistic Topic Models for Learning Terminological Ontologies


5

RBM's are an interesting beast. To answer your question, and to jog my memory on them, I'll derive RBMs and talk through the derivation. You mentioned that you're confused on the likelihood, so my derivation will be from the perspective of trying to maximize the likelihood. So let's begin. RBMs contain two different sets of neurons, visible and hidden, I'll ...


5

In addition to the existing answers, I would like to talk about this energy function, and the intuition behind that a bit. Sorry if this is a bit long and physical. The energy function describes a so-called Ising model, which is a model of ferromagnetism in terms of statistical mechanics / quantum mechanics. In statistical mechanics, we use a so-called ...


5

Typically to extract features, you can use the top layer of the network before the output. The intuition is that these features are linearly separable because the top layer is just a logistic regression. For GANs, you can use the features from the discriminator. These features are supposed to give a probability if the input came from the training dataset, "...


5

I have used stacked auto-encoders to reduce our 40 features step by step to 5 features and then output back to 40 features (some of my features were all zeros/ non deviating features). Training this on original (assumed to have no outliers) gives you a network which has learnt an abstract representation of the 40 features with 5 features. When outliers show ...


5

You should break this down one step further: retaining local structure and retaining global structure. Other well-understood methods, such as Principal Component Analysis are great at retaining global structure, because it looks at ways in which a dataset's variance is retained, globally, across the entire dataset. t-SNE works differently, by looking at ...


5

Some unsupervised models can make predictions, but not ones that necessarily match the original class labels. Once a GaussianMixture model has been fitted, it can predict which of the clusters a new example belongs to. This is exactly what the predict and predict_proba functions do in this case, and given that the number of clusters is set to 3, the number ...


4

The main difference between supervised and unsupervised learning is the following: In supervised learning you have a set of labelled data, meaning that you have the values of the inputs and the outputs. What you try to achieve with machine learning is to find the true relationship between them, what we usually call the model in math. There are many ...


4

Semi-Supervised Learning The combination of unsupervised learning and supervised learning is referred to as semi-supervised learning, which is the concept that I believe you are searching for. Label propagation is often cited when outlining the heuristics of semi-supervised learning. The essence is to employ clustering, but to use a tiny set of known ...


4

There are couple of ways to deal with missing data. Replace missing values by mean/median. If the missing values is very less, then this method would be apt. Also depends on how skewed your data is. Imputation. Build a linear regression model to predict the missing values based on other parameters. KNN could also be used to predict the missing value Also ...


4

I'm not an expert in this field but you should take a look at the work of Bhargav Srinivasa Desikan, a gensim contributor, who works a lot with topic modelling. He has a couple of notebooks on his github account which could interest you, especially this one (should be pretty much your use case if I understand your problem correctly). The aforementioned ...


4

In this case, a picture is a worth a thousand words. They literally mean data whose distribution on X,Y is roughly a sphere. Different clustering algorithms work better on different distributions. For example, K means does poorly on the arrangement in the first two rows but OK on the last row.


4

In algorithm 1 of the original GAN article (https://arxiv.org/pdf/1406.2661.pdf), the discriminator is said to be updated by "ascending its stochastic gradient". This is referring to equation 1: $$ \min_G \max_D V(D, G)= \mathbb{E}_{x\sim p_{data}(x)}[\log D(x)] + \mathbb{E}_{z\sim p_z(z)}[\log(1 - D(G(z)))] $$ When we want to minimize something, we do ...


4

This question is quite broad. I'll try to set you on the right path, more so than providing a truly complete answer. Theoretical background As others have mentioned, the task you're trying to do is usually known as anomaly detection, also known as novelty detection. There's many possible ways to approach this kind of task, depending on the assumptions you ...


Only top voted, non community-wiki answers of a minimum length are eligible