Questions tagged [unsupervised-learning]

Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.

Filter by
Sorted by
Tagged with
0
votes
1answer
20 views

What algorithm should I use to find which sub-component is associated with a attribute?

I have a n x m one hot encoded matrix of a number of units, where n corresponds to the number of units, and m is the number of total sub-components. This corresponds to a n x 5 matrix which has the 5 ...
3
votes
1answer
102 views

Finding dominating attributes with in the clusters generated

I am having a dataset of customers where each customer is represented as some feature vector and I am applying K-means algorithm to this dataset. On the basis of those features, I can abstract and ...
0
votes
0answers
46 views

What is the principle of Unsupervised Data Augmentation (UDA)? Why does UDA work? [closed]

UDA(https://github.com/google-research/uda) could achieve good accuracy by only 20 training data on text classification. But I find it is hard to reproduce the result on my own dataset. So I want to ...
3
votes
1answer
70 views

Sentiment analysis of tweets (Train model on a labelled dataset and use on some other unlabelled data)

I have a huge amount of tweets on a particular topic say 'ABC' and the data is not labelled. I want to perform multi-class sentiment analysis of these tweets. I tried many unsupervised clustering ...
0
votes
0answers
4 views

How to cluster large time series?

I want to perform clustering on a dataset which has a few thousand stock prices time series, each one with daily prices for previous 5 years (a couple thousand days). What would be the best approach ...
0
votes
1answer
151 views

Kmeans cluster validation when I have labeled test data

I'm trying to implement the unsupervised k-means algorithm for sentiment analysis of imdb movie dataset created by stanford. The steps that I followed is : 1) Load the comments 2) Apply tokenization ...
0
votes
0answers
14 views

What are some good techniques to decrease the size of Image Embeddings returned by CNN model?

I want to extract features from pre trained ResNet model for over 2M data. Problem? Even with the average pooling applied on the last layer's result, it provides a ...
0
votes
1answer
32 views

Self Organizing Map (SOM)

How do you use SOM as a supervised learning technique? Which approach can be added to SOM to turn it to supervised learning?
1
vote
1answer
29 views

Is it better to have one model with more categories or less with two for multi-label classification?

For classifying text into three classes question, complain and complements where each sample can have multi-labels (question and complains, question and complements): is it better to have one model ...
1
vote
0answers
30 views

Help why to apply PCA here

Lets say we have a dataset of 9 dimensional points and I want to apply k-means algorithm. I was studying an example where they apply PCA before fitting the data into the clustering algorithm. The ...
0
votes
1answer
35 views

Identifying potential customers based on their Rank and Value

I have a dataset which has demographic data available for a list of new customers. the data does'nt include transaction data of the customers. I want to identify the top 100 potential customers among ...
0
votes
0answers
8 views

How do we use a Hierarchical clustering model with DTW again?

I've been trying to cluster time series of shape (1, 400), so 1 row and 400 columns which correspond to 400 timesteps. My train set is of size (1000, 400) so 1000 time series. I have calculated a ...
0
votes
0answers
10 views

Identify similar features from a set of features for a given feature in an unsupervised learning way

Suppose I have a set of 1000 features. Now for a given feature, how to find the similar features from this 1000 features in an unsupervised learning way? In other words, no target/label dataset is ...
2
votes
1answer
158 views

Anomaly detection - relation between thresholds and anomalies

I'm developing an anomaly detection program in Python. Main idea is to create a new LSTM model every day, training it with the previous 7 days and predict the next day. Then, using thresholds, find ...
4
votes
1answer
161 views

Probability for label correctness in semi-supervised learning

I am aware of the existence of semi-supervised learning approaches, such as the Ladder Network, where only a subset of the data is labeled. Are there any methods or papers which consider correctness ...
2
votes
1answer
22 views

Recommender/Clustering data to support a hypothesis. Is this a valid use-case for unsupervised ML?

I have a dataset where some items have been labelled (categorized into 4 classes [A,B,C,D]). However, there is a vast majority of the dataset which has not been labelled. My hypothesis is that there ...
0
votes
1answer
26 views

how to compare between kmeans and hierarchical clustering results

I am using 2 types of clustering algorithm I apply hierarchical clustering the K-means clustering using python sklearn library Now the results are a little bit different so how can I compare the ...
1
vote
1answer
109 views

Clustering (unsupervised learning) for uneven classes

I am looking for an unsupervised method that can see also the points that start to look different from the majority. Which clustering techniques (I use python) can be used for such data sets? I have ...
2
votes
1answer
36 views

What's the good index to choose number of clusters so that obtained clusters are homogeneous?

I perform a clustering on one-dimensional dataset and I need a way to automatically decide what's the optimal number of clusters from $k \in \{2, 3, 4, 5, 6\}$. The number of observations to cluster ...
2
votes
1answer
44 views

Dendrogram: ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all()

I am trying to plot a Dendrogram to cluster data but this error is stopping me. My datea is here. I first chose columns to work with: ...
0
votes
2answers
70 views

Clusters with bounded diameter

In my application, I want to have clusters whose diameters are bounded by some fixed number. Also, the number of clusters in the data is unknown and, therefore, the clusters must be discovered without ...
1
vote
1answer
27 views

Working with few instances of specific target feature over large dataset

I have data over a single, a machine includes different components, all the parts are interacting, the data are tracked for those parts, it tracks power consumption and many other relevant feature ...
2
votes
0answers
40 views

Intuition behind One Class SVM (Scholkopf)

I am trying to understand the intuition behind the idea of finding a hyperplane that separates the training data from the origin in the feature space. Why separation from origin with a hyperplane ...
1
vote
1answer
47 views

Univariate Outlier Detection

Let's say I have a dataset with the following format: customerid product orders_in_last7days orders_in_last6days orders_in_last5days orders_in_last4days orders_in_last3days orders_in_last2days ...
2
votes
1answer
674 views

Is it possible to make a label automatically in supervised learning(Machine Learning)?

My background knowledge: Basically, supervised learning is based on labeled data. Using the labeled data, the machine can study and determine results for unlabeled data. To do that, for example, if we ...
1
vote
2answers
36 views

How do I select the “best” unsupervised machine learning algorithm to cluster my specific dataset?

I want to cluster a dataset without prior knowledge on the correct amount of clusters. For different algorithms (i.e. k-means, gmm...) I can iterate through different values and try to find the best ...
1
vote
1answer
74 views

approach for predicting machine failure using maintenance history

I have been struggling with this problem for a while now and I finally decided to post a question here to get some help. The problem i'm trying to solve is about predictive maintenance. Specifically, ...
0
votes
1answer
32 views

Unsupervised Sentimental Analysis in R

How would you evaluate unsupervised sentimental analysis? I am reading on evaluating sentimental analysis and learning that much of the classification models that are being used, the data has target/...
1
vote
1answer
38 views

Hierarchical dirichlet process results

I am thinking about using hierarchical dirichlet process to model a patent dataset. I've seen that HDP uses a base distribution and assumes that every topic comes from that base distribution. The ...
1
vote
1answer
16 views

Document clustering to merge common labels

I am building a recommendation system and I have to clean up some of the labels that I have. For example of the data df['resolution_modified'].value_counts() Gives ...
1
vote
1answer
226 views

More weightage to a categorical feature for an Autoencoder model

I am using autoencoder for anomaly detection. I don't have any labels already and so its unsupervised. If I have categorical variables, I usually one hot encode before giving it to the model. I would ...
1
vote
1answer
20 views

Using GANs to generate synthetic tabular data to improve supervised learning

One topic I see some people trying is using GANs to generate synthetic tabular data for supervised learning. Also as a way to oversample the minority class in a binary classification. For me creating ...
0
votes
3answers
598 views

What kind of learning is needed for anomaly detection? Supervised learning, semi-supervised learning or unsupervised learning?

I am doing anomaly detection recently, one of the methods is using AEs model to learn the pattern of normal samples. Determine it as an abnormal sample if it doesn’t match the pattern of normal ...
0
votes
2answers
221 views

Percentile as a threshold for Anomaly Detection?

I'm following this article about Unsupervised Anomaly Detection Algorithms. In this article, a threshold value is calculated using the scipy score percentile method to determine whether the point is ...
0
votes
2answers
59 views

Is there an unsupervised learning algorithm that can cluster data based on more than two dimensions?

I am just beginning to get into data science and have never posted here before, apologies if this question is worded incorrectly! I am curious if there is an unsupervised machine learning algorithm ...
1
vote
0answers
31 views

How can realize the evaluation/validation of unsupervised models through unlabeled data?

I'm researching anomaly detection, which is nothing else than outliers detection on a set of time-series web servers access log data or network traffic. Recently I re-faced to following fundamental ...
5
votes
2answers
126 views

Cluster evolution over time

I have a dataset of transactional data with customer ID and I want to segment the dataset into groups using cluster analysis. I'm interested in following the evolution of each cluster over time, but ...
0
votes
2answers
67 views

customer segmentation with unbalanced data

I am trying to do a customer segmentation on my transactional data and I am struggling a little bit on the best approach. Since it is an unsupervised model I can throw it to any algorithm and get some ...
0
votes
1answer
28 views

How would you quantify an experience into a score without labeled data

How would you approach a scenario where you have to quantify an abstract notion like “customer experience” without having any labeled data? So basically what you have are bunch of variables that you ...
1
vote
0answers
25 views

Difference between Q-learning and G-learning in Reinforcement Learning?

What is the difference between Q-learning and G-learning in Reinforcement Learning? Please explain with formulas. An example source: Instead of relying on a utility of consumption, we present G-...
1
vote
1answer
45 views

unsupervised anomaly detection on sparse data

Given that I have a very sparse data matrix with continuous features, like this dataframe for example ...
1
vote
1answer
1k views

How to set the Reconstruction error threshold for anomaly detection using autoencoders?

Hi I am doing anomaly detection using auto encoders.I have trained the model using 'Non Anomalous' values.Now when I give anomalous points as test data. What should be the Reconstruction error ...
1
vote
1answer
58 views

Clustering sequences of sentence embeddings

I have a sequence of events, right now I am not worried about their actual times, just the order. This is a sequence of web page views. I have modelled my data as the following, where each element ...
0
votes
1answer
34 views

Hot Encode vs Binary Encoding for Binary attribute when clustering

I am planning to use data for a clustering problem that contains a column with a binary value BUY/SELL. Should I be converting this attribute and assign it binary values (BUY=1, SELL=0), and keep it ...
0
votes
0answers
14 views

LDA-like algorithm but at the character level?

I have a catalog of products and I'd like to find "topics" in their description. The problem is that you might find in the description things like GraphicsCardVendor10.2 ...
0
votes
1answer
23 views

Similarity matching between two distinct datasets (marketing case study)

I am working for a company that sells different products to customers. My objective is to find customers that are likely to purchase product X based on the profiles of customers that already purchased ...
0
votes
0answers
10 views

Using Transcoder Model for language to language conversion

I have a problem statement like Converting deprecated code into a modern version of the same language. I'm currently converting with a custom Rule-based engine. But the modern version of the language ...
0
votes
1answer
36 views

Clustering cartesian coordinates associated with 1 categorical feature

I have a series of 2D coordinates X = {x, y}. Each are associated with one categorical variable W that can take 7 different values. E.g: ...
1
vote
1answer
63 views

How to get the probability/closeness of a sample belonging to a specific cluster?

I'm new to this so please let me know if my logic of comparing cosine similarity and k-means is incorrect I got a set of ...
0
votes
1answer
112 views

Clustering with custom criterion (minimum cluster weight)

Edit: following comment from @anony-mousse, I'm changing the question to search for a general clustering approach that matches this criterion (minimum weight per cluster). I am to use a clustering ...

1
2 3 4 5
7