Questions tagged [unsupervised-learning]
Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.
350
questions
0
votes
1answer
20 views
What algorithm should I use to find which sub-component is associated with a attribute?
I have a n x m one hot encoded matrix of a number of units, where n corresponds to the number of units, and m is the number of total sub-components. This corresponds to a n x 5 matrix which has the 5 ...
3
votes
1answer
102 views
Finding dominating attributes with in the clusters generated
I am having a dataset of customers where each customer is represented as some feature vector and I am applying K-means algorithm to this dataset. On the basis of those features, I can abstract and ...
0
votes
0answers
46 views
What is the principle of Unsupervised Data Augmentation (UDA)? Why does UDA work? [closed]
UDA(https://github.com/google-research/uda) could achieve good accuracy by only 20 training data on text classification.
But I find it is hard to reproduce the result on my own dataset.
So I want to ...
3
votes
1answer
70 views
Sentiment analysis of tweets (Train model on a labelled dataset and use on some other unlabelled data)
I have a huge amount of tweets on a particular topic say 'ABC' and the data is not labelled. I want to perform multi-class sentiment analysis of these tweets. I tried many unsupervised clustering ...
0
votes
0answers
4 views
How to cluster large time series?
I want to perform clustering on a dataset which has a few thousand stock prices time series, each one with daily prices for previous 5 years (a couple thousand days).
What would be the best approach ...
0
votes
1answer
151 views
Kmeans cluster validation when I have labeled test data
I'm trying to implement the unsupervised k-means algorithm for sentiment analysis of imdb movie dataset created by stanford.
The steps that I followed is :
1) Load the comments
2) Apply tokenization ...
0
votes
0answers
14 views
What are some good techniques to decrease the size of Image Embeddings returned by CNN model?
I want to extract features from pre trained ResNet model for over 2M data. Problem? Even with the average pooling applied on the last layer's result, it provides a ...
0
votes
1answer
32 views
Self Organizing Map (SOM)
How do you use SOM as a supervised learning technique? Which approach can be added to SOM to turn it to supervised learning?
1
vote
1answer
29 views
Is it better to have one model with more categories or less with two for multi-label classification?
For classifying text into three classes question, complain and complements
where each sample can have multi-labels (question and complains, question and complements):
is it better to have one model ...
1
vote
0answers
30 views
Help why to apply PCA here
Lets say we have a dataset of 9 dimensional points and I want to apply k-means algorithm.
I was studying an example where they apply PCA before fitting the data into the clustering algorithm. The ...
0
votes
1answer
35 views
Identifying potential customers based on their Rank and Value
I have a dataset which has demographic data available for a list of new customers.
the data does'nt include transaction data of the customers.
I want to identify the top 100 potential customers among ...
0
votes
0answers
8 views
How do we use a Hierarchical clustering model with DTW again?
I've been trying to cluster time series of shape (1, 400), so 1 row and 400 columns which correspond to 400 timesteps. My train set is of size (1000, 400) so 1000 time series. I have calculated a ...
0
votes
0answers
10 views
Identify similar features from a set of features for a given feature in an unsupervised learning way
Suppose I have a set of 1000 features. Now for a given feature, how to find the similar features from this 1000 features in an unsupervised learning way? In other words, no target/label dataset is ...
2
votes
1answer
158 views
Anomaly detection - relation between thresholds and anomalies
I'm developing an anomaly detection program in Python.
Main idea is to create a new LSTM model every day, training it with the previous 7 days and predict the next day.
Then, using thresholds, find ...
4
votes
1answer
161 views
Probability for label correctness in semi-supervised learning
I am aware of the existence of semi-supervised learning approaches, such as the Ladder Network, where only a subset of the data is labeled. Are there any methods or papers which consider correctness ...
2
votes
1answer
22 views
Recommender/Clustering data to support a hypothesis. Is this a valid use-case for unsupervised ML?
I have a dataset where some items have been labelled (categorized into 4 classes [A,B,C,D]). However, there is a vast majority of the dataset which has not been labelled. My hypothesis is that there ...
0
votes
1answer
26 views
how to compare between kmeans and hierarchical clustering results
I am using 2 types of clustering algorithm
I apply hierarchical clustering the K-means clustering using python sklearn library
Now the results are a little bit different so how can I compare the ...
1
vote
1answer
109 views
Clustering (unsupervised learning) for uneven classes
I am looking for an unsupervised method that can see also the points that start to look different from the majority. Which clustering techniques (I use python) can be used for such data sets?
I have ...
2
votes
1answer
36 views
What's the good index to choose number of clusters so that obtained clusters are homogeneous?
I perform a clustering on one-dimensional dataset and I need a way to automatically decide what's the optimal number of clusters from $k \in \{2, 3, 4, 5, 6\}$. The number of observations to cluster ...
2
votes
1answer
44 views
Dendrogram: ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all()
I am trying to plot a Dendrogram to cluster data but this error is stopping me. My datea is here.
I first chose columns to work with:
...
0
votes
2answers
70 views
Clusters with bounded diameter
In my application, I want to have clusters whose diameters are bounded by some fixed number. Also, the number of clusters in the data is unknown and, therefore, the clusters must be discovered without ...
1
vote
1answer
27 views
Working with few instances of specific target feature over large dataset
I have data over a single, a machine includes different components, all the parts are interacting, the data are tracked for those parts, it tracks power consumption and many other relevant feature ...
2
votes
0answers
40 views
Intuition behind One Class SVM (Scholkopf)
I am trying to understand the intuition behind the idea of finding a hyperplane that separates the training data from the origin in the feature space.
Why separation from origin with a hyperplane ...
1
vote
1answer
47 views
Univariate Outlier Detection
Let's say I have a dataset with the following format:
customerid
product
orders_in_last7days
orders_in_last6days
orders_in_last5days
orders_in_last4days
orders_in_last3days
orders_in_last2days
...
2
votes
1answer
674 views
Is it possible to make a label automatically in supervised learning(Machine Learning)?
My background knowledge: Basically, supervised learning is based on labeled data. Using the labeled data, the machine can study and determine results for unlabeled data. To do that, for example, if we ...
1
vote
2answers
36 views
How do I select the “best” unsupervised machine learning algorithm to cluster my specific dataset?
I want to cluster a dataset without prior knowledge on the correct amount of clusters. For different algorithms (i.e. k-means, gmm...) I can iterate through different values and try to find the best ...
1
vote
1answer
74 views
approach for predicting machine failure using maintenance history
I have been struggling with this problem for a while now and I finally decided to post a question here to get some help. The problem i'm trying to solve is about predictive maintenance.
Specifically, ...
0
votes
1answer
32 views
Unsupervised Sentimental Analysis in R
How would you evaluate unsupervised sentimental analysis?
I am reading on evaluating sentimental analysis and learning that much of the classification models that are being used, the data has target/...
1
vote
1answer
38 views
Hierarchical dirichlet process results
I am thinking about using hierarchical dirichlet process to model a patent dataset. I've seen that HDP uses a base distribution and assumes that every topic comes from that base distribution.
The ...
1
vote
1answer
16 views
Document clustering to merge common labels
I am building a recommendation system and I have to clean up some of the labels that I have. For example of the data
df['resolution_modified'].value_counts()
Gives
...
1
vote
1answer
226 views
More weightage to a categorical feature for an Autoencoder model
I am using autoencoder for anomaly detection. I don't have any labels already and so its unsupervised. If I have categorical variables, I usually one hot encode before giving it to the model. I would ...
1
vote
1answer
20 views
Using GANs to generate synthetic tabular data to improve supervised learning
One topic I see some people trying is using GANs to generate synthetic tabular data for supervised learning. Also as a way to oversample the minority class in a binary classification.
For me creating ...
0
votes
3answers
598 views
What kind of learning is needed for anomaly detection? Supervised learning, semi-supervised learning or unsupervised learning?
I am doing anomaly detection recently, one of the methods is using AEs model to learn the pattern of normal samples.
Determine it as an abnormal sample if it doesn’t match the pattern of normal ...
0
votes
2answers
221 views
Percentile as a threshold for Anomaly Detection?
I'm following this article about Unsupervised Anomaly Detection Algorithms. In this article, a threshold value is calculated using the scipy score percentile method to determine whether the point is ...
0
votes
2answers
59 views
Is there an unsupervised learning algorithm that can cluster data based on more than two dimensions?
I am just beginning to get into data science and have never posted here before, apologies if this question is worded incorrectly! I am curious if there is an unsupervised machine learning algorithm ...
1
vote
0answers
31 views
How can realize the evaluation/validation of unsupervised models through unlabeled data?
I'm researching anomaly detection, which is nothing else than outliers detection on a set of time-series web servers access log data or network traffic. Recently I re-faced to following fundamental ...
5
votes
2answers
126 views
Cluster evolution over time
I have a dataset of transactional data with customer ID and I want to segment the dataset into groups using cluster analysis. I'm interested in following the evolution of each cluster over time, but ...
0
votes
2answers
67 views
customer segmentation with unbalanced data
I am trying to do a customer segmentation on my transactional data and I am struggling a little bit on the best approach. Since it is an unsupervised model I can throw it to any algorithm and get some ...
0
votes
1answer
28 views
How would you quantify an experience into a score without labeled data
How would you approach a scenario where you have to quantify an abstract notion like “customer experience” without having any labeled data? So basically what you have are bunch of variables that you ...
1
vote
0answers
25 views
Difference between Q-learning and G-learning in Reinforcement Learning?
What is the difference between Q-learning and G-learning in Reinforcement Learning? Please explain with formulas.
An example source:
Instead of relying on a utility of consumption, we present G-...
1
vote
1answer
45 views
unsupervised anomaly detection on sparse data
Given that I have a very sparse data matrix with continuous features, like this dataframe for example
...
1
vote
1answer
1k views
How to set the Reconstruction error threshold for anomaly detection using autoencoders?
Hi I am doing anomaly detection using auto encoders.I have trained the model using 'Non Anomalous' values.Now when I give anomalous points as test data.
What should be the Reconstruction error ...
1
vote
1answer
58 views
Clustering sequences of sentence embeddings
I have a sequence of events, right now I am not worried about their actual times, just the order. This is a sequence of web page views.
I have modelled my data as the following, where each element ...
0
votes
1answer
34 views
Hot Encode vs Binary Encoding for Binary attribute when clustering
I am planning to use data for a clustering problem that contains a column with a binary value BUY/SELL.
Should I be converting this attribute and assign it binary values (BUY=1, SELL=0), and keep it ...
0
votes
0answers
14 views
LDA-like algorithm but at the character level?
I have a catalog of products and I'd like to find "topics" in their description. The problem is that you might find in the description things like GraphicsCardVendor10.2 ...
0
votes
1answer
23 views
Similarity matching between two distinct datasets (marketing case study)
I am working for a company that sells different products to customers. My objective is to find customers that are likely to purchase product X based on the profiles of customers that already purchased ...
0
votes
0answers
10 views
Using Transcoder Model for language to language conversion
I have a problem statement like Converting deprecated code into a modern version of the same language. I'm currently converting with a custom Rule-based engine. But the modern version of the language ...
0
votes
1answer
36 views
Clustering cartesian coordinates associated with 1 categorical feature
I have a series of 2D coordinates X = {x, y}. Each are associated with one categorical variable W that can take 7 different values.
E.g:
...
1
vote
1answer
63 views
How to get the probability/closeness of a sample belonging to a specific cluster?
I'm new to this so please let me know if my logic of comparing cosine similarity and k-means is incorrect
I got a set of ...
0
votes
1answer
112 views
Clustering with custom criterion (minimum cluster weight)
Edit: following comment from @anony-mousse, I'm changing the question to search for a general clustering approach that matches this criterion (minimum weight per cluster).
I am to use a clustering ...
