Questions tagged [unsupervised-learning]

Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

Two problems with the Association Rules widget

I am using the heart disease prediction dataset from UCI but analyzing it with association rules. The target class is set as "diameter narrowing" which means the presence or absence of ...
1
vote
1answer
29 views

Working with few instances of specific target feature over large dataset

I have data over a single, a machine includes different components, all the parts are interacting, the data are tracked for those parts, it tracks power consumption and many other relevant feature ...
0
votes
1answer
21 views

Is it possible to implement a Recommender System without having a ratings/previous purchases similar data?

I'm trying to implement a recommender system for a website that hosts a wide variety of softwares and you can search the website to find what you need. The need is to implement a recommender system to ...
0
votes
0answers
4 views

Inverse Autoregressive Spline Flow Implementation

I have to implement an algorithm for a university project, however I can not seem to wrap my head around it. The algorithm should be an inverse autoregressive normalizing flow using splines. It should ...
1
vote
0answers
21 views

How can I apply K means algorithm to detect a pattern? [closed]

I have a dataset: How can I apply kmeans algorithm to find clusters based on "date" column . So that I can retrieve tweeting/retweeting activity every hour and generate a pattern. It would ...
0
votes
0answers
22 views

Which is the best Machine Learning appoach/model when all features are not available for prediction?

I have 50 Features in a Dataset to predict 1 Variable "Units Sold". I am currently using XGBoost model (Supervised Learning) to train all these 50 Features and the accuracy of the model on ...
2
votes
1answer
208 views

Anomaly detection - relation between thresholds and anomalies

I'm developing an anomaly detection program in Python. Main idea is to create a new LSTM model every day, training it with the previous 7 days and predict the next day. Then, using thresholds, find ...
2
votes
1answer
36 views

What's the good index to choose number of clusters so that obtained clusters are homogeneous?

I perform a clustering on one-dimensional dataset and I need a way to automatically decide what's the optimal number of clusters from $k \in \{2, 3, 4, 5, 6\}$. The number of observations to cluster ...
5
votes
3answers
194 views

Cluster evolution over time

I have a dataset of transactional data with customer ID and I want to segment the dataset into groups using cluster analysis. I'm interested in following the evolution of each cluster over time, but ...
0
votes
0answers
13 views

target encoding and weighting

I am working on a project in which I use data of movies and I represent each movie as a vector of length of 15. So there are 15 features ranging from genre to director. Most of the features are ...
3
votes
1answer
165 views

Probability for label correctness in semi-supervised learning

I am aware of the existence of semi-supervised learning approaches, such as the Ladder Network, where only a subset of the data is labeled. Are there any methods or papers which consider correctness ...
1
vote
1answer
134 views

Clustering (unsupervised learning) for uneven classes

I am looking for an unsupervised method that can see also the points that start to look different from the majority. Which clustering techniques (I use python) can be used for such data sets? I have ...
0
votes
1answer
10 views

Clustering 2D curves

I have a set of curves in 2D space each expressed as a set of (sampled) data points. Each set has more or less the same number of items - eventually I guess I’ll use binning to make sure the number of ...
0
votes
1answer
14 views

Avoiding overfitting in unsupervised ML

I am using a unsupervised pattern matching approach to create a trade strategy. I use the output of the pattern matched results to decide whether to enter a trade or not. For deciding the best pattern ...
1
vote
2answers
43 views

How do I select the “best” unsupervised machine learning algorithm to cluster my specific dataset?

I want to cluster a dataset without prior knowledge on the correct amount of clusters. For different algorithms (i.e. k-means, gmm...) I can iterate through different values and try to find the best ...
0
votes
0answers
11 views

Anomaly Detection in Highly Variable Time-Series Data

I am trying to detect anomalies through a column called count. The data is a time-series data and it is present for every 5 minutes for each day. The dataframe looks like this: ...
2
votes
1answer
686 views

Is it possible to make a label automatically in supervised learning(Machine Learning)?

My background knowledge: Basically, supervised learning is based on labeled data. Using the labeled data, the machine can study and determine results for unlabeled data. To do that, for example, if we ...
0
votes
2answers
28 views

How can i decide on which features to use for clustering?

I am clustering on a dataset where each row is a customer and each column is a feature. I have 200 features, this seems like alot for clustering. I plan to experiment with a variety of clustering ...
1
vote
1answer
56 views

Univariate Outlier Detection

Let's say I have a dataset with the following format: customerid product orders_in_last7days orders_in_last6days orders_in_last5days orders_in_last4days orders_in_last3days orders_in_last2days ...
1
vote
1answer
16 views

What would cause a Hierarchical Cluster to look like skewed (if this is the right term to use!)?

I am surprised about this hierarchical cluster! For me, it looks somehow abnormal. Or, maybe normal but I am not able to identify why it looks like this. Any idea why data would be clustered in such a ...
1
vote
1answer
38 views

Can we fine-tune a model on the same dataset which it is pretrained on?

So I was reading this paper (about a use case of pretraining then self-training) which got me thinking - suppose I pre-train a model on a particular dataset, then fine-tune it again on the same ...
0
votes
2answers
55 views

How to JUST represent words as embeddings by pretrained BERT?

I don't have enough data (i.e. I don't have enough texts) --- have only around 4k words in my dictionary. I need to compare given words, then I need to representate it as embedding. After the ...
1
vote
2answers
83 views

approach for predicting machine failure using maintenance history

I have been struggling with this problem for a while now and I finally decided to post a question here to get some help. The problem i'm trying to solve is about predictive maintenance. Specifically, ...
0
votes
1answer
28 views

How to cluster words automatically?

I have a problem where I have a list of n words with truly k different ones (k is unknown) because some may be malformed or contracted. I would like to automatically cluster them. I thought about ...
1
vote
1answer
41 views

Hierarchical dirichlet process results

I am thinking about using hierarchical dirichlet process to model a patent dataset. I've seen that HDP uses a base distribution and assumes that every topic comes from that base distribution. The ...
0
votes
3answers
748 views

What kind of learning is needed for anomaly detection? Supervised learning, semi-supervised learning or unsupervised learning?

I am doing anomaly detection recently, one of the methods is using AEs model to learn the pattern of normal samples. Determine it as an abnormal sample if it doesn’t match the pattern of normal ...
1
vote
1answer
244 views

More weightage to a categorical feature for an Autoencoder model

I am using autoencoder for anomaly detection. I don't have any labels already and so its unsupervised. If I have categorical variables, I usually one hot encode before giving it to the model. I would ...
0
votes
2answers
361 views

Percentile as a threshold for Anomaly Detection?

I'm following this article about Unsupervised Anomaly Detection Algorithms. In this article, a threshold value is calculated using the scipy score percentile method to determine whether the point is ...
2
votes
0answers
55 views

Unsupervised document similarity state of the art

I have a set of N documents with lengths ranging from 0 to more than 20000 characters. I want to calculate a similarity score between 0 and 1 between all pairs of documents where a higher number ...
0
votes
2answers
65 views

Is there an unsupervised learning algorithm that can cluster data based on more than two dimensions?

I am just beginning to get into data science and have never posted here before, apologies if this question is worded incorrectly! I am curious if there is an unsupervised machine learning algorithm ...
0
votes
2answers
89 views

customer segmentation with unbalanced data

I am trying to do a customer segmentation on my transactional data and I am struggling a little bit on the best approach. Since it is an unsupervised model I can throw it to any algorithm and get some ...
1
vote
2answers
60 views

When should we choose agglomerative clustering over K-means clustering?

I was working on a clustering based model and I read about hierarchical clustering and K-Means clustering. Under what conditions should I choose agglomerative over K-means clustering?
0
votes
2answers
36 views

How to group every data point with HDBSCAN to some group to have no noise?

TASK I am clustering products with about 70 dimensions ex.: price, rating 5/5, product tag(cleaning, toy, food, fruits) I use HDBSCAN to do it GOAL The goal is when users come on our site and I can ...
3
votes
0answers
60 views

How to remove noise using morphological filtering

I have two groups of dots that both contain noise between them: The line that separates the two groups in the picture is diagonal in shape. I tried to use morphological filtering on this image to ...
0
votes
1answer
30 views

How would you quantify an experience into a score without labeled data

How would you approach a scenario where you have to quantify an abstract notion like “customer experience” without having any labeled data? So basically what you have are bunch of variables that you ...
0
votes
1answer
41 views

Hot Encode vs Binary Encoding for Binary attribute when clustering

I am planning to use data for a clustering problem that contains a column with a binary value BUY/SELL. Should I be converting this attribute and assign it binary values (BUY=1, SELL=0), and keep it ...
1
vote
1answer
1k views

How to set the Reconstruction error threshold for anomaly detection using autoencoders?

Hi I am doing anomaly detection using auto encoders.I have trained the model using 'Non Anomalous' values.Now when I give anomalous points as test data. What should be the Reconstruction error ...
1
vote
1answer
99 views

Clustering sequences of sentence embeddings

I have a sequence of events, right now I am not worried about their actual times, just the order. This is a sequence of web page views. I have modelled my data as the following, where each element ...
3
votes
1answer
110 views

PCA and k-means for categorical variables?

I have a clustering task at hand. The data that I have contains only categorical variables. So, k-modes seemed like the best option. But I am not sure what are the data pre processing steps required ...
0
votes
1answer
46 views

Clustering cartesian coordinates associated with 1 categorical feature

I have a series of 2D coordinates X = {x, y}. Each are associated with one categorical variable W that can take 7 different values. E.g: ...
0
votes
0answers
17 views

Compute similiarty between labels

I have a labeled dataset and I created a duplicate of this dataset and removed the labels and applied K-means clustering with k= the number of labels in the original data set I want to compute ...
0
votes
1answer
31 views

Similarity matching between two distinct datasets (marketing case study)

I am working for a company that sells different products to customers. My objective is to find customers that are likely to purchase product X based on the profiles of customers that already purchased ...
1
vote
1answer
53 views

Anomaly detection on sparse categorical data

I have a big dataset with a column "clientid" and a categorical column "choice". I want to find out what are the clients that have strange combinations of choices (less frequent ...
0
votes
0answers
19 views

Is there a way to implement nested features in unsupervised models?

Our project has built an unsupervised model that uses data about a number of companies. Some of these companies are public and some are private. The ones that are public have much higher financial ...
0
votes
0answers
7 views

Accuracy of a Cluster in Tableau

I have certain data , in that I have labels and formed a groups , Also with the same data without using any labels I used clustering option that is available on tableau . So, Now I want to check the ...
-2
votes
2answers
391 views

Clustering data set with multiple dimensions

I have a data set which is similar to the following: It is recipe data along with the composition of the recipe (in %) I have 91 recipes and 40 ingredients in total. I want to be able to cluster ...
0
votes
0answers
23 views

Appropriate Approaches for Unsupervised Textual Classification

Let's say I take 1000 dissertations from a variety of different fields, and thereafter extract the top 100 words from each dissertation. What would be the best approach to categorize them into STEM/...
1
vote
1answer
158 views

How to get the probability/closeness of a sample belonging to a specific cluster?

I'm new to this so please let me know if my logic of comparing cosine similarity and k-means is incorrect I got a set of ...
0
votes
1answer
132 views

Clustering with custom criterion (minimum cluster weight)

Edit: following comment from @anony-mousse, I'm changing the question to search for a general clustering approach that matches this criterion (minimum weight per cluster). I am to use a clustering ...
0
votes
1answer
53 views

What methods are available to evaluate similarity between different clustering algorithms?

I am performing extensive customer segmentation analysis and so far implemented Gaussian Mixture Models, K-Means, and Hierarchical Clustering. For the most part, the algorithms agree on the structure ...

1
2 3 4 5
8