Questions tagged [unsupervised-learning]

Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.

Filter by
Sorted by
Tagged with
0
votes
0answers
17 views

Hierarchical clustering and Dendrogram interpretation [closed]

I'm quite new to cluster analysis and I was trying to perform a hierarchical clustering algorithm (in R) on my data to spot some groups in my dataset. Initially, I tried with the k-means, with the ...
0
votes
0answers
14 views

LDA-like algorithm but at the character level?

I have a catalog of products and I'd like to find "topics" in their description. The problem is that you might find in the description things like GraphicsCardVendor10.2 ...
0
votes
0answers
9 views

Using Transcoder Model for language to language conversion

I have a problem statement like Converting deprecated code into a modern version of the same language. I'm currently converting with a custom Rule-based engine. But the modern version of the language ...
1
vote
0answers
18 views

Difference between Q-learning and G-learning in Reinforcement Learning?

What is the difference between Q-learning and G-learning in Reinforcement Learning? Please explain with formulas. An example source: Instead of relying on a utility of consumption, we present G-...
1
vote
1answer
32 views

Hierarchical dirichlet process results

I am thinking about using hierarchical dirichlet process to model a patent dataset. I've seen that HDP uses a base distribution and assumes that every topic comes from that base distribution. The ...
1
vote
0answers
21 views

Building an unsupervised learning model to detect suspicious transactions using DBSCAN [closed]

I am working on building a unsupervised learning model to detect suspicious transactions using DBSCAN. Do I train the model on all data columns (columns like account number, transaction date, ...
1
vote
2answers
34 views

Unsupervised Learning - Using the Outcome of Learning

My understanding in Unsupervised Learning is that -- when you want a computer to learn on its own by examining a large dataset. The goal is to establish some form of cluster or association-based ...
0
votes
1answer
21 views

unsupervised anomaly detection on sparse data

Given that I have a very sparse data matrix with continuous features, like this dataframe for example ...
1
vote
0answers
31 views

K-Medoid Clustering with Point Weights

I asked the same question at Cross Validated, here I implemented a K-Medoid clustering algorithm recently; I have a number of points $x_1, ..., x_n$ which have various properties and a distance ...
4
votes
1answer
380 views

Can a novelty detection model overfit?

Can a novelty detection model overfit? In novelty detection, the model is trained on normal data instances (not polluted by outliers) where no labels are used in the training process, while validated ...
1
vote
0answers
17 views

Explanation of Excess Mass(EM)

I was researching on evaluation metrics to understand the performance of unsupervised anomaly detection algorithms and I came across this paper The author suggests that EM and MV based numerical ...
0
votes
0answers
24 views

need to create 4 group using the data

i have to create four group using this data :- ...
0
votes
1answer
21 views

How to categorize unlabelled promotional email data

I have unlabelled data of promotional emails. I want to categorize those emails based on the topics like fashion, health & wellness, sports, media, Entertainment, etc. Can anyone let me know any ...
0
votes
1answer
26 views

what is meant by minimizing and maximizing in GANs?

It is a subtle change that involves the generator maximizing the log of the discriminator probabilities for generated images instead of minimizing the log of the inverted discriminator probabilities ...
1
vote
0answers
28 views

What are the commonly used approaches to perform row-wise dimensionaltiy reduction (not PCA) for large datasets? [closed]

I have this problem where for each row (datapoint) in my dataset, I have minute level data. To make it clear, say I have 10k rows as the main dataset (dimensions: 10k X 15) and for each of those 10k ...
0
votes
0answers
15 views

Perceptual Loss for 3D VAE

I'm working on 3D VAE implementation for biomedical images. The results are too blurring, so I'm searching to improve the performance of the network. Many people recommend the use of 'perceptual loss',...
4
votes
1answer
63 views

What is major difference between different dimensionality reduction algorithms? [closed]

I find many algorithms are used for dimensionality reduction. The more commonly used ones (e.g. on this page ) are: ...
0
votes
1answer
32 views

Identifying potential customers based on their Rank and Value

I have a dataset which has demographic data available for a list of new customers. the data does'nt include transaction data of the customers. I want to identify the top 100 potential customers among ...
1
vote
1answer
66 views

Can Shapley/Lime values be used for unsupervised learning?

One thing that is really useful when trying to understand what a machine learning model does, is seeing why some instances got predicted. For that Shapley Values and Lime are really usefull. But can ...
1
vote
1answer
27 views

Is it better to have one model with more categories or less with two for multi-label classification?

For classifying text into three classes question, complain and complements where each sample can have multi-labels (question and complains, question and complements): is it better to have one model ...
1
vote
1answer
28 views

How to properly train your Self-Organized Map?

I stumbled recently upon the Self-Organized Map, an ANN architecture used to cluster high dimensional data, while simultaneously imposing a neighbourhood structure on it. It's trained through a ...
0
votes
0answers
21 views

How to go about fine-tuning BERT using a next-sentence task

I've got a large corpus of documents, and I want to use bert to generate embeddings for a variety of predictive tasks. The documents are multi-sentence, in a non-standard domain, and have labels at ...
0
votes
1answer
22 views

Similarity matching between two distinct datasets (marketing case study)

I am working for a company that sells different products to customers. My objective is to find customers that are likely to purchase product X based on the profiles of customers that already purchased ...
1
vote
1answer
49 views

How can we identify potential customers for a new list of customers?

I have two data sets: Customer demographic data; Transaction data of the customers. Now, if I have to identify potential customers to develop a marketing strategy, I would make use of clustering to ...
1
vote
1answer
21 views

How to cluster government census data in order to group Metropolitan statistical areas

I have collected a bunch of census data from 2012 - 2018. I wanted to apply some clustering algorithms in order to compare Metropolitan statistical area (MSA's). Ideally once I run the clustering ...
0
votes
0answers
8 views

Using AutoEncoder for Greedy Layer-Wise Pretraining [Convolutional Neural Networks]

I am trying to implement greedy layer-wise pretraining for Convolutional Neural Network binary classifier using AutoEncoders. However, I am a little bit confused regarding the logic of implementation. ...
1
vote
1answer
16 views

Advise resources on un-supervised learning [closed]

I have seen that people coming into data science will rush into scikit-learn or other libraries without trying to learn the knowledge behind. Its good to follow a top-down approach but most of times ...
0
votes
1answer
23 views

Unsupervised Sentimental Analysis in R

How would you evaluate unsupervised sentimental analysis? I am reading on evaluating sentimental analysis and learning that much of the classification models that are being used, the data has target/...
1
vote
1answer
27 views

Identifying templates from SMS text

I am building an app where I identify information from the SMS, something similar to expense management apps. I have a parser which reads all the SMS of user, identifies SMS of interest and parses ...
0
votes
0answers
20 views

Bottleneck Distance

Is there a range of values for the bottleneck distance in persim package (python) to conclude that the two datasets are similar? Also, does it make sense to compute the bottleneck distance using $H_0$ ...
2
votes
1answer
69 views

Choosing a distance metric and measuring similarity

I am trying to decide which particular algorithm would be most appropriate for my use-case. I have dataset of about 1000 physical buildings in a city with feature space such as location, distance, ...
3
votes
1answer
41 views

Expectation Maximization Algorithm (EM) for Gaussian Mixture Models (GMMs)

I'm trying to apply the Expectation Maximization algorithm (EM) to a Gaussian Mixture Model (GMM) using Python and NumPy. The PDF document I am basing my implementation on can be found here. Below are ...
3
votes
1answer
68 views

Sentiment analysis of tweets (Train model on a labelled dataset and use on some other unlabelled data)

I have a huge amount of tweets on a particular topic say 'ABC' and the data is not labelled. I want to perform multi-class sentiment analysis of these tweets. I tried many unsupervised clustering ...
1
vote
1answer
47 views

What type of consideration can be made using clustering?

I am clustering my data to see how information look like and which group may be identified. Since clustering is an unsupervised algorithm, I cannot test the accuracy of the classification. So I was ...
-1
votes
1answer
28 views

Does Anomaly Detection Algorithm works when the features are not correlated?

I am working on an Anomaly Detection Problem and the algorithm I used is an Autoencoder Multivariate Gaussian. The problem with my data is that it is unlabeled and not correlated. For example, let's ...
0
votes
0answers
22 views

Best practices for avoiding spurious artifacts in image cluster detection / color quantization

I want to know whether there are some common best practices for unsupervised detection of clusters / colors in images, in order to avoid spurious artifacts. To understand what I mean by 'spurious', ...
1
vote
2answers
31 views

How do I select the “best” unsupervised machine learning algorithm to cluster my specific dataset?

I want to cluster a dataset without prior knowledge on the correct amount of clusters. For different algorithms (i.e. k-means, gmm...) I can iterate through different values and try to find the best ...
1
vote
1answer
138 views

Choosing attributes for k-means clustering

The k-means clustering tries to minimize the within-cluster scatter and maximizing the distances between clusters. It does so on all attributes. I am learning about this method on several datasets. To ...
-1
votes
1answer
23 views

How to explain the results from this kmeans?

I got the following results by using k-means algorithm. There are $10$ elements in Cluster $0$ and $3$ elements in Cluster $1$. Do you think it makes sense and it might be an acceptable result? How ...
1
vote
1answer
37 views

How to get the probability/closeness of a sample belonging to a specific cluster?

I'm new to this so please let me know if my logic of comparing cosine similarity and k-means is incorrect I got a set of ...
0
votes
1answer
21 views

What methods are available to evaluate similarity between different clustering algorithms?

I am performing extensive customer segmentation analysis and so far implemented Gaussian Mixture Models, K-Means, and Hierarchical Clustering. For the most part, the algorithms agree on the structure ...
0
votes
1answer
32 views

What is the principle of Unsupervised Data Augmentation (UDA)? Why does UDA work?

UDA(https://github.com/google-research/uda) could achieve good accuracy by only 20 training data on text classification. But I find it is hard to reproduce the result on my own dataset. So I want to ...
3
votes
3answers
81 views

Better approach to assign values to determine potential fake sentences

I am trying to assign different values for each sentences based on information about the presence of hashtags, upper case letters/words (e.g. HATE) and some others. I created a data frame which ...
0
votes
0answers
7 views

Data points are highly overlapped and do not follow smoothness rule assumption

I am working on a very high dimensional categorical features based data set. There are two output classes and 2-dimensional PCA plot suggests that the data points belonging to both +ve and -ve classes ...
0
votes
0answers
24 views

clustering more than optimal k and Overfitting in k-means

In my data by using elbow method. i got optimal k to be 3. but , i clustered them into 5 clusters.and the patterns in the cluster are as i wanted them . But, does using k more than optimal k decreases ...
0
votes
2answers
50 views

Finding point of interest in time series data

I am working on a project where I need to figure out the point of interest in time series data. From the picture you can probably understand a bit more what I mean. Basically, imagine this is the ...
0
votes
0answers
22 views

What ML model should I use for matching 3 datasets on a selection of common columns?

I am looking for suggestions on which model might be appropriate for my case. I have 3 tables that I need to match. Column names don't match but I can modify them. Fields ... do match to some ...
0
votes
0answers
19 views

Marrying classic features with text/chat data

This is an approach to a problem / advice seeking question: I just wanted to get more experienced opinions on how best to combine (in a categorization context) user text/chat entry data and the ...
0
votes
0answers
82 views

Unsupervised learning-Feature selection in python

I need to select the most important features from my data frame before starting with nearest neighbours problem. Which methods are the best to do this? My data frame has around 8 categorical features ...

1
2 3 4 5
7