Skip to main content

Questions tagged [unsupervised-learning]

Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.

Filter by
Sorted by
Tagged with
4 votes
3 answers
293 views

Which ML approach is the best for huge state spaces?

My issue derives from the challenge of solving a seemingly easy-looking game. To spare you the full catalogue of rules, here is a short summary of the game: Single player card game You go through a ...
1 vote
1 answer
47 views

Supervised vs Unsupervised - Flag fake accounts on social medias

I have this project I'm working on where I scraped users' data from social media to predict if they are bots, fake accounts or legit users based on their comments, likes, posts, public data only. I'm ...
3 votes
1 answer
32 views

Trim left tail of music in audio file

I have audio files, most of them start with the same music, and then a conversation begins. I want to trim the part of the music (which can be varied in length). I have no labels, I can transcribe the ...
1 vote
0 answers
21 views

Two sets of topics/words in Topic Modeling

In short, the question is: I have two sets of words per document. I would like to extract two sets of topics per document corresponding to sets of words. To be more precise: Document(d) can be ...
1 vote
1 answer
956 views

Clustering text data based on sentiment?

I am scraping reviews off Amazon with the intent to perform sentiment analysis to classify them into positve, negative and neutral. Now the data I would get would be text and unlabeled. My approach to ...
10 votes
3 answers
13k views

Isolation forest sklearn contamination param

I am working on an unsupervised anomaly detection task on time series data using an isolation forest algorithm. I am developing it in Python, more in detail using ...
1 vote
0 answers
37 views

Model doesn't know German well enough

I have a model that generates questions and answers based on input text. The texts are in German and based on observations it seems like the model doesn't know German well enough. I need to pretrain ...
25 votes
2 answers
20k views

What kinds of learning problems are suitable for Support Vector Machines?

What are the hallmarks or properties that indicate that a certain learning problem can be tackled using support vector machines? In other words, what is it that, when you see a learning problem, makes ...
1 vote
1 answer
92 views

How to properly train your Self-Organized Map?

I recently stumbled upon the Self-Organized Map - an ANN architecture used to cluster high dimensional data - while simultaneously imposing a neighborhood structure on it. It is trained through a ...
3 votes
1 answer
246 views

Machine learning roadmap not for beginners [closed]

To introduce myself: I know what is RL, know some RL algorithms such as PPO, A2C. Know about offline RL, online RL. I have read many papers about RL. Such as MuZero, AplhaZero, Decision Transformer ...
1 vote
1 answer
599 views

Correct approach to scale (min-max scaler) both input and output signal data for unsupervised learning?

I am working on a denoising autoencoder problem with noisy and clean signals. Before I pass the signals to my model I want to apply min-max normalization and am unsure of the correct way to apply this....
1 vote
1 answer
39 views

Working with few instances of specific target feature over large dataset

I have data over a single, a machine includes different components, all the parts are interacting, the data are tracked for those parts, it tracks power consumption and many other relevant feature ...
3 votes
1 answer
2k views

Does BERT need supervised data only when fine-tuning?

I've read many articles and papers mentioning how unsupervised training is conducted while pre-training a BERT model. I would like to know if it is possible to fine-tune a BERT model in an ...
4 votes
4 answers
1k views

What machine learning algorithms to use for unsupervised POS tagging?

I am interested in an unsupervised approach to training a POS-tagger. Labeling is very difficult and I would like to test a tagger for my specific domain (chats) where users typically write in lower ...
3 votes
1 answer
187 views

Probability for label correctness in semi-supervised learning

I am aware of the existence of semi-supervised learning approaches, such as the Ladder Network, where only a subset of the data is labeled. Are there any methods or papers which consider correctness ...
1 vote
1 answer
53 views

How do I evaluate a K-Means unsupervised anomaly detection approach?

how do I evaluate K-means clustering anomaly detection method as there is no labelled data of anomaly class. To find the cluster (K), I have used the silhouette score from Scikit learn library. Scikit ...
2 votes
0 answers
23 views

What technique's can be used to identify and count individual animals in a dataset?

Problem: I have an image dataset that contains a lot of different chitals (a species of deer). The images are taken by cameratraps in a National Park. I would like to count the individual animals. For ...
0 votes
1 answer
18 views

Clustering of events

I've a sequence of time ordered set of points: for each $t=1...T$ I have a set of points $(x_{t,i},y_{t,i})$. I need to cluster them together in space-time. I don't know however a priori the number of ...
0 votes
2 answers
93 views

Is there an unsupervised learning algorithm that can cluster data based on more than two dimensions?

I am just beginning to get into data science and have never posted here before, apologies if this question is worded incorrectly! I am curious if there is an unsupervised machine learning algorithm ...
1 vote
2 answers
188 views

Hot Encode vs Binary Encoding for Binary attribute when clustering

I am planning to use data for a clustering problem that contains a column with a binary value BUY/SELL. Should I be converting this attribute and assign it binary values (BUY=1, SELL=0), and keep it ...
0 votes
0 answers
217 views

Difference between self supervised learning and unsupervised learning

Self supervised learning is considered a subset of unsupervised learning. Is there any major difference between the two owing to the similarity of self supervised methods towards supervised learning.
29 votes
1 answer
37k views

Word2Vec vs. Sentence2Vec vs. Doc2Vec

I recently came across the terms Word2Vec, Sentence2Vec and Doc2Vec and kind of confused as I am new to vector semantics. Can someone please elaborate the differences in these methods in simple words. ...
1 vote
0 answers
17 views

Clustering dataset with and without estimating means (no EM algorithm)

Given a dataset $D$ of the form $$ D = \{ (x_0,y_0), (x_1,y_1),\ldots,(x_{n},y_n) $$ sampled from a Gaussian mixture model with identity covariance matrices, I want to understand what are my options ...
-1 votes
1 answer
79 views

Is it possible to implement a Recommender System without having a ratings/previous purchases similar data?

I'm trying to implement a recommender system for a website that hosts a wide variety of software and you can search the website to find what you need. The need is to implement a recommender system to ...
0 votes
2 answers
265 views

Can I use labelled data in unsupervised learning algorithms like neural network?

I am working on a transaction dataset that consists of some labeled features like gender, product categories, membership types, and so on. There are also some numeric data like the amount of ...
1 vote
0 answers
364 views

Using STL(Seasonal-Trend decomposition using LOESS) for Anomaly detection

I am using STL to decompose my time series data in Season, trend and residual and then by applying this(see below) on residual. I am detecting the anomaly ...
1 vote
3 answers
119 views

How to detect outliers to lable my unsupervised data?

I have unsupervised sensor data and i want to lable each row of data as anomaly or normal. Here K-mean clustering will work?
3 votes
1 answer
419 views

How to do clustering assuring more than one class per cluster?

I have a dataset with 4 classes and i'm trying to use an ensemble model where each base classifier trains with a portion of data. To distribute data along the classifiers, i am using KMeans algorithm. ...
1 vote
1 answer
40 views

What is the best method to *classify* series of data?

I'm working on a project for CCG (such as HearthStone, Yu-Gi-Oh!, etc.) which does classify (or give labels) deck type when user ...
0 votes
1 answer
192 views

Distance between any two points after DBSCAN

DBSCAN is a clustering model which is robust to detect the outliers also. A parameter $\epsilon$ i.e. radius is an input of the algorithm, a point is said to be outlier if it's circle with radius $\...
3 votes
2 answers
4k views

Does it make sense to do train test split when trainning GANS?

For normal supervised learning the dataset is split in train and test (let's keep it simple). Generative Adversarial Networks are unsupervised learning but there is a supervised loss function in the ...
2 votes
1 answer
70 views

Clustering Weekday Weekend Data and Multicollinearity

Hi I have data of weekday and weekend step counts in which I extracted metrics from them such as the wd steps, we steps, standard deviation of wd steps, standard deviation of we steps and so on... <...
1 vote
0 answers
60 views

Unsupervised Learning with audio recordings

I had a (probably crazy) idea for a project and I was wondering if you all think it would be in any way possible. I'm interested in analyzing sounds made by different types of animals (for example ...
0 votes
1 answer
33 views

Avoiding overfitting in unsupervised ML

I am using a unsupervised pattern matching approach to create a trade strategy. I use the output of the pattern matched results to decide whether to enter a trade or not. For deciding the best pattern ...
0 votes
2 answers
880 views

How can i decide on which features to use for clustering?

I am clustering on a dataset where each row is a customer and each column is a feature. I have 200 features, this seems like alot for clustering. I plan to experiment with a variety of clustering ...
1 vote
1 answer
224 views

What would cause a Hierarchical Cluster to look like skewed (if this is the right term to use!)?

I am surprised about this hierarchical cluster! For me, it looks somehow abnormal. Or, maybe normal but I am not able to identify why it looks like this. Any idea why data would be clustered in such a ...
2 votes
1 answer
3k views

Can we fine-tune a model on the same dataset which it is pretrained on?

So I was reading this paper (about a use case of pretraining then self-training) which got me thinking - suppose I pre-train a model on a particular dataset, then fine-tune it again on the same ...
0 votes
2 answers
5k views

How to JUST represent words as embeddings by pretrained BERT?

I don't have enough data (i.e. I don't have enough texts) --- have only around 4k words in my dictionary. I need to compare given words, then I need to representate it as embedding. After the ...
0 votes
1 answer
41 views

How to cluster words automatically?

I have a problem where I have a list of n words with truly k different ones (k is unknown) because some may be malformed or contracted. I would like to automatically cluster them. I thought about ...
2 votes
0 answers
430 views

Unsupervised document similarity state of the art

I have a set of N documents with lengths ranging from 0 to more than 20000 characters. I want to calculate a similarity score between 0 and 1 between all pairs of documents where a higher number ...
2 votes
2 answers
6k views

When should we choose agglomerative clustering over K-means clustering?

I was working on a clustering based model and I read about hierarchical clustering and K-Means clustering. Under what conditions should I choose agglomerative over K-means clustering?
3 votes
0 answers
196 views

How to remove noise using morphological filtering

I have two groups of dots that both contain noise between them: The line that separates the two groups in the picture is diagonal in shape. I tried to use morphological filtering on this image to ...
1 vote
1 answer
312 views

Clustering sequences of sentence embeddings

I have a sequence of events, right now I am not worried about their actual times, just the order. This is a sequence of web page views. I have modelled my data as the following, where each element ...
1 vote
0 answers
20 views

Compute similiarty between labels

I have a labeled dataset and I created a duplicate of this dataset and removed the labels and applied K-means clustering with k= the number of labels in the original data set I want to compute ...
1 vote
1 answer
110 views

Anomaly detection on sparse categorical data

I have a big dataset with a column "clientid" and a categorical column "choice". I want to find out what are the clients that have strange combinations of choices (less frequent ...
-2 votes
2 answers
3k views

Clustering data set with multiple dimensions

I have a data set which is similar to the following: It is recipe data along with the composition of the recipe (in %) I have 91 recipes and 40 ingredients in total. I want to be able to cluster ...
0 votes
0 answers
30 views

Appropriate Approaches for Unsupervised Textual Classification

Let's say I take 1000 dissertations from a variety of different fields, and thereafter extract the top 100 words from each dissertation. What would be the best approach to categorize them into STEM/...
2 votes
0 answers
60 views

unsupervised learning time series datasets

I experiment on building electricity power consumption datasets and try to see relationships of the power consumption with weather data and dummy variables that represent time-of-week. The only thing ...
0 votes
0 answers
159 views

Making Keras RNN to proceed input sequence step by step

I'm currently trying to create a neural network for playing Tetris. I'm using evolutionary algorythms for it's learning, so the behavior that I need to get from the neural network is the following: ...
3 votes
1 answer
240 views

Geolocation Based Anomaly Detection in IPs Using Isolation Forest

I'm trying to detect anomalies based on geolocation from IP addresses on a server access log file. I have created two features country and geo_velocity, using the IP address and the timestamp of each ...

1 2 3
4
5
10