Questions tagged [unsupervised-learning]

Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.

Filter by
Sorted by
Tagged with
1
vote
1answer
15 views

Clustering using both text and numerical features

I have a dataset that contains 2 types of features, one is generated from doc2vec and one is numerical feature. I would like to perform clustering analysis on them. However, due to the size of doc2vec ...
0
votes
0answers
4 views

How to cluster large time series?

I want to perform clustering on a dataset which has a few thousand stock prices time series, each one with daily prices for previous 5 years (a couple thousand days). What would be the best approach ...
0
votes
0answers
15 views

What are some good techniques to decrease the size of Image Embeddings returned by CNN model?

I want to extract features from pre trained ResNet model for over 2M data. Problem? Even with the average pooling applied on the last layer's result, it provides a ...
-1
votes
1answer
35 views

Self Organizing Map (SOM)

How do you use SOM as a supervised learning technique? Which approach can be added to SOM to turn it to supervised learning?
1
vote
0answers
32 views

Help why to apply PCA here

Lets say we have a dataset of 9 dimensional points and I want to apply k-means algorithm. I was studying an example where they apply PCA before fitting the data into the clustering algorithm. The ...
0
votes
0answers
8 views

How do we use a Hierarchical clustering model with DTW again?

I've been trying to cluster time series of shape (1, 400), so 1 row and 400 columns which correspond to 400 timesteps. My train set is of size (1000, 400) so 1000 time series. I have calculated a ...
0
votes
0answers
10 views

Identify similar features from a set of features for a given feature in an unsupervised learning way

Suppose I have a set of 1000 features. Now for a given feature, how to find the similar features from this 1000 features in an unsupervised learning way? In other words, no target/label dataset is ...
2
votes
1answer
23 views

Recommender/Clustering data to support a hypothesis. Is this a valid use-case for unsupervised ML?

I have a dataset where some items have been labelled (categorized into 4 classes [A,B,C,D]). However, there is a vast majority of the dataset which has not been labelled. My hypothesis is that there ...
2
votes
1answer
58 views

Dendrogram: ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all()

I am trying to plot a Dendrogram to cluster data but this error is stopping me. My datea is here. I first chose columns to work with: ...
2
votes
0answers
41 views

Intuition behind One Class SVM (Scholkopf)

I am trying to understand the intuition behind the idea of finding a hyperplane that separates the training data from the origin in the feature space. Why separation from origin with a hyperplane ...
0
votes
1answer
30 views

how to compare between kmeans and hierarchical clustering results

I am using 2 types of clustering algorithm I apply hierarchical clustering the K-means clustering using python sklearn library Now the results are a little bit different so how can I compare the ...
1
vote
1answer
16 views

Document clustering to merge common labels

I am building a recommendation system and I have to clean up some of the labels that I have. For example of the data df['resolution_modified'].value_counts() Gives ...
1
vote
1answer
20 views

Using GANs to generate synthetic tabular data to improve supervised learning

One topic I see some people trying is using GANs to generate synthetic tabular data for supervised learning. Also as a way to oversample the minority class in a binary classification. For me creating ...
0
votes
0answers
36 views

How can realize the evaluation/validation of unsupervised models through unlabeled data?

I'm researching anomaly detection, which is nothing else than outliers detection on a set of time-series web servers access log data or network traffic. Recently I re-faced to following fundamental ...
0
votes
0answers
14 views

LDA-like algorithm but at the character level?

I have a catalog of products and I'd like to find "topics" in their description. The problem is that you might find in the description things like GraphicsCardVendor10.2 ...
0
votes
0answers
10 views

Using Transcoder Model for language to language conversion

I have a problem statement like Converting deprecated code into a modern version of the same language. I'm currently converting with a custom Rule-based engine. But the modern version of the language ...
1
vote
0answers
28 views

Difference between Q-learning and G-learning in Reinforcement Learning?

What is the difference between Q-learning and G-learning in Reinforcement Learning? Please explain with formulas. An example source: Instead of relying on a utility of consumption, we present G-...
1
vote
1answer
39 views

Hierarchical dirichlet process results

I am thinking about using hierarchical dirichlet process to model a patent dataset. I've seen that HDP uses a base distribution and assumes that every topic comes from that base distribution. The ...
1
vote
0answers
24 views

Building an unsupervised learning model to detect suspicious transactions using DBSCAN [closed]

I am working on building a unsupervised learning model to detect suspicious transactions using DBSCAN. Do I train the model on all data columns (columns like account number, transaction date, ...
1
vote
2answers
36 views

Unsupervised Learning - Using the Outcome of Learning

My understanding in Unsupervised Learning is that -- when you want a computer to learn on its own by examining a large dataset. The goal is to establish some form of cluster or association-based ...
1
vote
1answer
51 views

unsupervised anomaly detection on sparse data

Given that I have a very sparse data matrix with continuous features, like this dataframe for example ...
1
vote
0answers
32 views

K-Medoid Clustering with Point Weights

I asked the same question at Cross Validated, here I implemented a K-Medoid clustering algorithm recently; I have a number of points $x_1, ..., x_n$ which have various properties and a distance ...
4
votes
1answer
392 views

Can a novelty detection model overfit?

Can a novelty detection model overfit? In novelty detection, the model is trained on normal data instances (not polluted by outliers) where no labels are used in the training process, while validated ...
1
vote
0answers
18 views

Explanation of Excess Mass(EM)

I was researching on evaluation metrics to understand the performance of unsupervised anomaly detection algorithms and I came across this paper The author suggests that EM and MV based numerical ...
0
votes
0answers
24 views

need to create 4 group using the data

i have to create four group using this data :- ...
0
votes
1answer
23 views

How to categorize unlabelled promotional email data

I have unlabelled data of promotional emails. I want to categorize those emails based on the topics like fashion, health & wellness, sports, media, Entertainment, etc. Can anyone let me know any ...
0
votes
1answer
33 views

what is meant by minimizing and maximizing in GANs?

It is a subtle change that involves the generator maximizing the log of the discriminator probabilities for generated images instead of minimizing the log of the inverted discriminator probabilities ...
0
votes
0answers
21 views

Perceptual Loss for 3D VAE

I'm working on 3D VAE implementation for biomedical images. The results are too blurring, so I'm searching to improve the performance of the network. Many people recommend the use of 'perceptual loss',...
4
votes
1answer
69 views

What is major difference between different dimensionality reduction algorithms? [closed]

I find many algorithms are used for dimensionality reduction. The more commonly used ones (e.g. on this page ) are: ...
0
votes
1answer
37 views

Identifying potential customers based on their Rank and Value

I have a dataset which has demographic data available for a list of new customers. the data does'nt include transaction data of the customers. I want to identify the top 100 potential customers among ...
1
vote
1answer
112 views

Can Shapley/Lime values be used for unsupervised learning?

One thing that is really useful when trying to understand what a machine learning model does, is seeing why some instances got predicted. For that Shapley Values and Lime are really usefull. But can ...
1
vote
1answer
30 views

Is it better to have one model with more categories or less with two for multi-label classification?

For classifying text into three classes question, complain and complements where each sample can have multi-labels (question and complains, question and complements): is it better to have one model ...
1
vote
1answer
33 views

How to properly train your Self-Organized Map?

I stumbled recently upon the Self-Organized Map, an ANN architecture used to cluster high dimensional data, while simultaneously imposing a neighbourhood structure on it. It's trained through a ...
0
votes
0answers
74 views

How to go about fine-tuning BERT using a next-sentence task

I've got a large corpus of documents, and I want to use bert to generate embeddings for a variety of predictive tasks. The documents are multi-sentence, in a non-standard domain, and have labels at ...
0
votes
1answer
24 views

Similarity matching between two distinct datasets (marketing case study)

I am working for a company that sells different products to customers. My objective is to find customers that are likely to purchase product X based on the profiles of customers that already purchased ...
1
vote
1answer
50 views

How can we identify potential customers for a new list of customers?

I have two data sets: Customer demographic data; Transaction data of the customers. Now, if I have to identify potential customers to develop a marketing strategy, I would make use of clustering to ...
1
vote
1answer
23 views

How to cluster government census data in order to group Metropolitan statistical areas

I have collected a bunch of census data from 2012 - 2018. I wanted to apply some clustering algorithms in order to compare Metropolitan statistical area (MSA's). Ideally once I run the clustering ...
0
votes
0answers
13 views

Using AutoEncoder for Greedy Layer-Wise Pretraining [Convolutional Neural Networks]

I am trying to implement greedy layer-wise pretraining for Convolutional Neural Network binary classifier using AutoEncoders. However, I am a little bit confused regarding the logic of implementation. ...
1
vote
1answer
16 views

Advise resources on un-supervised learning [closed]

I have seen that people coming into data science will rush into scikit-learn or other libraries without trying to learn the knowledge behind. Its good to follow a top-down approach but most of times ...
0
votes
1answer
35 views

Unsupervised Sentimental Analysis in R

How would you evaluate unsupervised sentimental analysis? I am reading on evaluating sentimental analysis and learning that much of the classification models that are being used, the data has target/...
1
vote
1answer
27 views

Identifying templates from SMS text

I am building an app where I identify information from the SMS, something similar to expense management apps. I have a parser which reads all the SMS of user, identifies SMS of interest and parses ...
0
votes
0answers
22 views

Bottleneck Distance

Is there a range of values for the bottleneck distance in persim package (python) to conclude that the two datasets are similar? Also, does it make sense to compute the bottleneck distance using $H_0$ ...
2
votes
1answer
85 views

Choosing a distance metric and measuring similarity

I am trying to decide which particular algorithm would be most appropriate for my use-case. I have dataset of about 1000 physical buildings in a city with feature space such as location, distance, ...
3
votes
1answer
49 views

Expectation Maximization Algorithm (EM) for Gaussian Mixture Models (GMMs)

I'm trying to apply the Expectation Maximization algorithm (EM) to a Gaussian Mixture Model (GMM) using Python and NumPy. The PDF document I am basing my implementation on can be found here. Below are ...
3
votes
1answer
70 views

Sentiment analysis of tweets (Train model on a labelled dataset and use on some other unlabelled data)

I have a huge amount of tweets on a particular topic say 'ABC' and the data is not labelled. I want to perform multi-class sentiment analysis of these tweets. I tried many unsupervised clustering ...
1
vote
1answer
47 views

What type of consideration can be made using clustering?

I am clustering my data to see how information look like and which group may be identified. Since clustering is an unsupervised algorithm, I cannot test the accuracy of the classification. So I was ...
-1
votes
1answer
30 views

Does Anomaly Detection Algorithm works when the features are not correlated?

I am working on an Anomaly Detection Problem and the algorithm I used is an Autoencoder Multivariate Gaussian. The problem with my data is that it is unlabeled and not correlated. For example, let's ...
0
votes
0answers
22 views

Best practices for avoiding spurious artifacts in image cluster detection / color quantization

I want to know whether there are some common best practices for unsupervised detection of clusters / colors in images, in order to avoid spurious artifacts. To understand what I mean by 'spurious', ...
1
vote
2answers
36 views

How do I select the “best” unsupervised machine learning algorithm to cluster my specific dataset?

I want to cluster a dataset without prior knowledge on the correct amount of clusters. For different algorithms (i.e. k-means, gmm...) I can iterate through different values and try to find the best ...

1
2 3 4 5
7