Questions tagged [unsupervised-learning]

Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.

Filter by
Sorted by
Tagged with
0 votes
0 answers
18 views

How can I cluster my weighted directed graph?

I've been trying to cluster my graph of jobs. The edges weights are the count of the transitions between 2 nodes(jobs). I've been reading about and I've based my code in this paper: https://hal....
user avatar
0 votes
0 answers
13 views

Is EDA mandatory for unsupervised problems

I have seen many problems solving videos where in the users not performing Exploratory Data Analysis for unsupervised problems. Is this true? But I feel EDA is necessary as we are dealing with ...
user avatar
2 votes
1 answer
56 views

Clustering ordered categorical data

Suppose I have a list of, say, 100 countries, as well as their respective historical sovereign credit ratings as such ...
user avatar
  • 123
1 vote
1 answer
15 views

Best Clustering Method for Dataset with Few Distinct Values

I have a dataset that has the opinions of 30 different TV shows for 2000 high school students. A student could have said they liked the show, did not have an opinion, or disliked the show. These ...
user avatar
1 vote
0 answers
25 views

Which clustering/partitioning algorithms can operate on arbitrary pairwise similarity or distance matrices?

I'm relatively new to cluster analysis, and I'm exploring options for general-purpose, non-hierarchical, strict partitioning of data based on a pre-computed $N\times N$ pairwise similarity matrix. ...
user avatar
1 vote
1 answer
30 views

Approach for unsupervised time series clustering/segmentation

I have a big sample of data on a human's everyday life. The snapshots of the life are taken every 5 minutes. The data include time, the location of the human, accelerometer data, gyroscope data, and ...
user avatar
  • 133
2 votes
1 answer
70 views

Using k-means to create labels for supervised learning

I want to know if the following is a valid approach to create labels, if I have measurements under some conditions, and the conditions are similar but never exactly the same. This doesn't correspond ...
user avatar
  • 23
0 votes
0 answers
8 views

density vs shape

Can you explain this passage please: "A key feature of HDBScan is that it clusters data of varying density, this is in comparison to DBScan, which tend to cluster data of varying shapes only.&...
user avatar
1 vote
1 answer
21 views

reuse of LDA model for new data

I am working with the LDA (Latent Dirichlet Allocation) model from sklearn and I have a question about reusing the model I have. After training my model with data how do I use it to make a prediction ...
user avatar
0 votes
0 answers
20 views

Is it impossible to predict defects with data that are not labeled?

There is manufacturing data with 10 process variables. Normal and bad labeling are not done. It's tabular fdata. Do you have a paper that only uses data that are not labeled to predict defects or to ...
user avatar
  • 11
1 vote
0 answers
24 views

massively imbalanced data

I am dealing with time series data with +200K (every minute for 6 months)record of gas turbine I am trying to early detect the fault (0 or 1-fault). The issues with the data are: 1.the fault occurred ...
user avatar
  • 11
1 vote
1 answer
27 views

Can clustering results based on probability be used for supervised learning?

I'm a beginner and I have a question. Can clustering results based on probability be used for supervised learning? Manufacturing data with 80000 rows. It is not labeled, but there is information that ...
user avatar
  • 11
0 votes
0 answers
7 views

Is Self-Supervised Learning a task of Representation Learning?

Maybe a weird question but: Currently, I'm writing a seminar paper about Self Supervised Learning for time series data. For this paper, I have to find methods to prepare unlabelled time series data ...
user avatar
0 votes
0 answers
33 views

Unsupervised learning for anomaly detection based on memory and cpu usage

Recently got into Data Science, I've been working on a data science project, I have built a system that collects real-time logs on virtual machines in the cloud, the logs include memory consumption ...
user avatar
1 vote
1 answer
73 views

how to select threshold for unsupervised anomaly detection

I am working on an anomaly detection use case. I studied one technique of selecting the threshold that marks 5% of validation data as anomalies. how it works in anomaly detection cases. and there is ...
user avatar
  • 175
1 vote
0 answers
20 views

How do I evaluate if my data represent the target variable before training a machine learning algorithm?

I have a dataset of points cloud where each point in the point cloud has a variable. I am trying to relate the local geometry features to that point variable by using FPFH, This means I am generating ...
user avatar
1 vote
1 answer
29 views

Confusion about the value of within-cluster SSE

I have a dataset of shape (29088, 11). When I apply the Kmeans where K=2 I get the following plot: I am surprised that the value of Sum Squared Error (SSE) for C0 (...
user avatar
  • 238
0 votes
0 answers
21 views

finding better threshold for the unsupervised problem

I am working on a unsupervised image reconstruction task. so i need to verify whether the reconstructed image by the model is good are not. I cant do that visually. so i was calculating the sum of ...
user avatar
  • 101
0 votes
0 answers
14 views

How to find syntactic dependencies in text using unsupervised method and context information?

I know there are ready libraries to find syntactic dependencies and besides supervised methods, I have studied some of the unsupervised dependency parsing which uses POS tags and other mathematical ...
user avatar
  • 111
2 votes
0 answers
23 views

Multi-Class Document Classification with both known and un-known classes

Currently, I am building a multi-class document classifier which has to classify either 3 known classes, namely "Financial Report", "Insurance_Sheet", "Endorsement", and ...
user avatar
0 votes
0 answers
12 views

Can I run isolation forest on existing data to find anomalies, save it for the future and use it on incoming data?

One of the major arguments I had recently is if we can save an unsupervised learning model to disk and use it later on incoming data. Isolation forest is one of the models that I use a lot for ...
user avatar
0 votes
1 answer
21 views

forcasting anomaly in products

I have a question about the forecasting of anomalies. I would be very grateful if you could refer me to some papers that deal with this kind of problem or give me some hints to start with this problem....
user avatar
  • 101
0 votes
0 answers
24 views

Self-supervised learning. Why is it useful?

Self-supervised learning implies that algorithms are trained to predict missing pieces. Say, I take a sentence "I like cats", remove the word "cats" and train an algorithm to ...
user avatar
  • 1
0 votes
1 answer
14 views

is using feature selection(supervised) methods after running kmeans and taking the 'cluster' variable(0,1,2 for eg.) as the labeled data correct?

Feature selection in a gist from what i understand is reducing the variables but retaining the labels as much as possible, from that pov this seems correct but i haven't found anything on this. Any ...
user avatar
  • 1
1 vote
1 answer
14 views

Partially labelled open-class classification problem with heavy overlap

Let's say we have a corpus of text, including discussions about movies and about sports. Unsupervised clustering would typically cluster into the two topics of discussion. However, we are interested ...
user avatar
0 votes
0 answers
6 views

Recommended unsupervised learning model for categorizing stamps

I'm trying to solve the following problem: a program is given pictures of collections of stamps (e.g. those large unsorted boxes you find on eBay and other sites), and it then uses ML to scan the ...
user avatar
1 vote
0 answers
37 views

Unsupervized latent truth discovery on text data

I have text infomration from several different sources. I need to identify the most reliable source in an unsupervized manner (no labels about true/false or ground truth to train on available). I ...
user avatar
  • 6,912
0 votes
1 answer
49 views

Word2vec to encode medical procedures when using isolation forests

I am planning to use Isolation Forests in R (solitude package) to identify outlier medical claims in my data. Each row of my data represents the group of drugs that each provider has administered in ...
user avatar
  • 241
0 votes
0 answers
14 views

Representaion Learning - Self-supervision methods that do well with a limited amount of classes when

I understand that a contrastive learning approach such as SimCLR has an inherent problem when dealing with a low number of classes (let's say 2,3,5,6, maybe even 10). Problem is that the chances of ...
user avatar
  • 101
0 votes
0 answers
26 views

Can mean-shift clustering algorithm be used for text clustering?

Mean Shift clustering is an Unsupervised learning that assigns the data points to the clusters iteratively by shifting points towards the mode (mode is the highest density of data points in the region,...
user avatar
1 vote
1 answer
30 views

Supervised vs Unsupervised - Flag fake accounts on social medias

I have this project I'm working on where I scraped users' data from social media to predict if they are bots, fake accounts or legit users based on their comments, likes, posts, public data only. I'm ...
user avatar
  • 93
0 votes
0 answers
125 views

Isolation Forest in R using Solitude - From the results how can I identify the anomalous records

I am trying to use the Isolation Forest algorithm in the Solitude package to identify anomalous rows in my data. I'm using the examples in the documentation to learn about the algorithm, this example ...
user avatar
  • 241
4 votes
3 answers
136 views

Which ML approach is the best for huge state spaces?

My issue derives from the challenge of solving a seemingly easy-looking game. To spare you the full catalogue of rules, here is a short summary of the game: Single player card game You go through a ...
user avatar
0 votes
0 answers
179 views

SHAP Kernel explainer for ensemble model

I am currently working on a project involving an unsupervised outlier detection ensemble model. However I am getting stuck by an error passed by the shap.KernelExplainer: "The passed model is not ...
user avatar
0 votes
0 answers
20 views

Why are the Order Of Initial Centroids effecting Kmeans Clustering?

For Iris Dataset I am doing the experiment. iris_k_mean_model_vor = KMeans(n_clusters=3, init=arr_4d) this is my model. Here I am feeding an Initial array of ...
user avatar
  • 133
0 votes
0 answers
18 views

Association rule mining for continuous variables

I'm trying to study the relationships between several numerical variables, eg. electricity generation between different stations at 30min intervals over several months. My data has the format I want ...
user avatar
0 votes
0 answers
21 views

Clustering Ensemble or Consensus to combine the different cluster output

I tried to use ClusterEnsemble, getting error while using ClusterEnsemble package in Python: ...
user avatar
0 votes
0 answers
175 views

Restricted Boltzmann Machine (RBM) implementation in Tensorflow (TF) 2.x

I‘m looking for a Python implementation of a Restricted Boltzmann Machine (RBM), e.g. applied to MNIST data as mentioned in „Elements of Statistical Learning“ Ch. 17, in Tensorflow 2.x. I‘m aware of ...
user avatar
  • 6,912
2 votes
1 answer
21 views

Trim left tail of music in audio file

I have audio files, most of them start with the same music, and then a conversation begins. I want to trim the part of the music (which can be varied in length). I have no labels, I can transcribe the ...
user avatar
1 vote
0 answers
16 views

Two sets of topics/words in Topic Modeling

In short, the question is: I have two sets of words per document. I would like to extract two sets of topics per document corresponding to sets of words. To be more precise: Document(d) can be ...
user avatar
  • 11
1 vote
1 answer
96 views

Clustering text data based on sentiment?

I am scraping reviews off Amazon with the intent to perform sentiment analysis to classify them into positve, negative and neutral. Now the data I would get would be text and unlabeled. My approach to ...
user avatar
  • 1,263
0 votes
0 answers
11 views

How to score different clusters of features for predictiveness?

I have a set of true/false data that represents whether or not a given feature was or was not active when the data snapshot was recorded. Data snapshots are recorded when the user takes an action. The ...
user avatar
  • 101
0 votes
0 answers
21 views

How to treat demographic variables in clustering?

I'm working on a project to cluster franchises of a certain company. In this case, the dataset is grouped by city, so I'm basically clustering cities. I'm ending up with variables such as: Population:...
user avatar
  • 127
0 votes
0 answers
18 views

Dynamic Clustering - Two States

I wondered if anyone was aware of research and corresponding R packages based upon unsupervised clustering in two different states. For example, suppose I have a panel data sample with 12 ordinal ...
user avatar
  • 203
1 vote
0 answers
27 views

Model doesn't know German well enough

I have a model that generates questions and answers based on input text. The texts are in German and based on observations it seems like the model doesn't know German well enough. I need to pretrain ...
user avatar
2 votes
1 answer
31 views

Tiering after clustering with Kmeans

I would like to have some suggestions on possible avenues that would make sense in the following context. 3 Optimal clusters have been identified in a 5000 list of customers using Kmeans Data model ...
user avatar
3 votes
1 answer
88 views

Machine learning roadmap not for beginners [closed]

To introduce myself: I know what is RL, know some RL algorithms such as PPO, A2C. Know about offline RL, online RL. I have read many papers about RL. Such as MuZero, AplhaZero, Decision Transformer ...
user avatar
  • 41
1 vote
1 answer
49 views

Clustering with hierarchical data dependencies

I am currently looking into how to cluster data with hierarchical dependencies. An example of a problem that I want to cluster: we would like to cluster cities to identify similar characteristics with ...
user avatar
0 votes
0 answers
71 views

Silhouette Score for different Clustering algorithms

I am trying to compare different clustering algorithms on a dataset and compare the model performance. Since the dataset is quite big (56 features), I applied PCA to reduce the number of features to ...
user avatar
  • 55
1 vote
1 answer
88 views

Correct approach to scale (min-max scaler) both input and output signal data for unsupervised learning?

I am working on a denoising autoencoder problem with noisy and clean signals. Before I pass the signals to my model I want to apply min-max normalization and am unsure of the correct way to apply this....
user avatar
  • 13

1
2 3 4 5
9