A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

Questions tagged [unsupervised-learning]

Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.

Filter by
Sorted by
Tagged with
1
vote
1answer
20 views

Does K - Means clustering on data reduced using PCA and the original data make any difference?

I am working on clustering and I have 90 features with 13500 data points and after removing the correlated variables which had pearson correlation more than 90% my feature space reduced to 70. Also, ...
0
votes
0answers
21 views

Learning Process in Machine Learning

I want to use Unsupervised Leaning to detect Anomalies within a huge Csv file (consisting of headers that are named and thousands of rows belonging to them) Here on this link I have read about the ...
1
vote
2answers
27 views

Does it make sense to do train test split when trainning GANS?

For normal supervised learning the dataset is split in train and test (let's keep it simple). Generative Adversarial Networks are unsupervised learning but there is a supervised loss function in the ...
2
votes
0answers
28 views

From unsupervised to supervised in fraud detection

I have a question. I am working on the fraud detection domain. And I have data from imports to the country. As you can get from the title, I have unsupervised data. I do not know that the record is ...
2
votes
1answer
19 views

When unsupervised learning is more beneficial in comparison with supervised learning even the labelings are existed?

When unsupervised learning is more beneficial in comparison with supervised learning even the labeling are existed? If there is no labeling the unsupervised learning is better than supervised learning ...
2
votes
2answers
42 views

What is a manifold for Unsupervised Learning?

I've been watching Dr. G. Hinton lectures on Neural Networks in Machine Learning, and in one of the lectures he explains what the goals of Unsupervised Learning are. I am having trouble ...
0
votes
0answers
11 views

Interaction with unseen data (Generalization and evaluation the performance on unseen data in supervised and unsupevised learning methods)

How to generalizes model and performs on unseen data for a highly imbalanced binary classification problem (99.827%,0.173%)? 1-When using supervised learning methods such as logistic reg, RFs, ...
1
vote
0answers
19 views

How we compare two paragraphs of unlabelled dataset semantically (STS)?

Column representation: Unique_id | Text1 | Text2 Unique_id 0 Text1 public show for reynolds suspension of his coaching licence. portrait sir joshua reynolds portrait of omai will get a public airing ...
1
vote
2answers
32 views

How can we perform STS(Semantic Textual Similarity) on UnSupervised dataset using Deep Learning?

How to implement STS(Semantic Textual Similarity) on unlabelled dataset. Dataset column contains Unique_id, text1(contains paragraph), text2(contains paragraph). Ex: Column representation: Unique_id ...
2
votes
1answer
16 views

Clustering Weekday Weekend Data and Multicollinearity

Hi I have data of weekday and weekend step counts in which I extracted metrics from them such as the wd steps, we steps, standard deviation of wd steps, standard deviation of we steps and so on... <...
0
votes
1answer
19 views

Is there a paper accomplishing finding physical law from observation without premade perception, using machine learning?

For example: Isaac Newton finds law of universal gravitation just by looking a falling apple, without any premade perception of that phenomenon. Is it possible to accomplish that kind of discovery ...
2
votes
0answers
29 views

What's the good index to choose number of clusters so that obtained clusters are homogeneous?

I perform a clustering on one-dimensional dataset and I need a way to automatically decide what's the optimal number of clusters from $k \in \{2, 3, 4, 5, 6\}$. The number of observations to cluster ...
6
votes
3answers
96 views

Why spectral clustering results in disjointed cluster?

I'm working on a project where I have to dynamically cluster the position of objects with respect to one coordinate. So I'm essentially dealing with subsequent frames and each frame represents a one-...
3
votes
1answer
23 views

Is pattern recognition the same as unsupervised learning? Is machine learning the same as supervised learning?

Firstly, here is the definition of a well-posed learning problem: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its ...
1
vote
1answer
19 views

Is there any good practice to cluster 3D data array?

So I'm not sure what word fits best to describe this data, probably "dimension" would be wrong since it may be used for flat samples with 3 features; but by 3D data I mean some structure in a form of ...
0
votes
0answers
15 views

clustering changing Label

I have dataset with clustering label (this label is the group of each point) and I want to create such recommendation system or any other model to help the point for changing his group (for example ...
0
votes
1answer
22 views

What is the best way to encode features when clustering data? [duplicate]

I have a dataset with numerical and categorical features. I am trying to run a k-means algorithm to find clusters of data. What is the best way to encode categorical features? I have been doing one ...
0
votes
0answers
16 views

Univariate Outlier Detection

Let's say I have a dataset with the following format: customerid product orders_in_last7days orders_in_last6days orders_in_last5days orders_in_last4days orders_in_last3days orders_in_last2days ...
1
vote
2answers
38 views

Clustering initialization

I'm running into a problem while working on clustering. I work on data with white Gaussian noise. All of the methods I have come across use some sort of random initialization to set up the mean and ...
1
vote
0answers
19 views

Hierarchical Clustering on transaction data

Problem Statement: Let's say I have buyer transactional data for every product, features are categorical and numeric. I want to cluster purchases that have similar attributes in terms of who's ...
0
votes
0answers
14 views

Which features are selected in isolation forest h2o?

As a method for anomaly detection, I'm looking at isolation forest in the H2O implementation in Python. I'd like to identify the features corresponding to a certain data point in a tree. I can see ...
0
votes
1answer
11 views

Topic models for non-textual data?

I am looking to employ an unsupervised clustering on a dataset where each observation has a mix of textual and non-textual features. For each observation, I combine the features into a single vector ...
2
votes
1answer
24 views

What is the difference between all the different types of learning within machine learning?

This is a question that is really hard to google, and the differences are confusing. Does anyone have good examples of the differences between them all? Supervised Learning Semi-Supervised Learning ...
1
vote
1answer
53 views

Anomaly detection - relation between thresholds and anomalies

I'm developing an anomaly detection program in Python. Main idea is to create a new LSTM model every day, training it with the previous 7 days and predict the next day. Then, using thresholds, find ...
2
votes
1answer
39 views

K-Means initialization

K-Means initializes the centroids randomly, but there are other methods to initialize. In this paper, http://ilpubs.stanford.edu:8090/778/1/2006-13.pdf, they propose randomly choosing a data point ...
1
vote
0answers
22 views

Predict with some probability the day of the month paycheck received through daily transactions

I am using R to do machine learning. I have daily shopping expenditure data of individuals over a couple of months and my goal is to be able to identify through their shopping pattern the day of the ...
1
vote
0answers
25 views

Unsupervised Function Optimization using Input and Output for Loss Function?

I have some vectors {$\mathbf{X_1 ... X_n}$} and they are all of dimension 1 x N. Vectors {$\mathbf{X_1' ... X_n'}$} are also 1 x N and are related to {$\mathbf{X_1 ... X_n}$}, but the relation cannot ...
0
votes
0answers
100 views

Autoencoder clustering with a small dataset

I have a dataset which is relatively small (less than a 1000 samples). I run an autoencoder on a training set, and then check the reconstruction error on a validation set and stop training before it ...
1
vote
2answers
38 views

Clustering Small Text Descriptions

Im presented with a unique text classification problem. Im given a list of descriptions each containing 3-8 words. I know that there are some descriptions that are nearly the same, but the majority ...
0
votes
0answers
17 views

Error on prediction keras multi_gpu_model

I've an issue running a keras model on a Google Cloud Platform instance. The model is the following one: ...
0
votes
0answers
6 views

Is there any library available for Manifold learning using 'Diffusion map' in python?

I would like to use an unsupervised learning with technique called 'Diffusion map based manifold learning' in python. The original paper on diffusion map is here. I have already checkout pydiffmap ...
0
votes
0answers
15 views

Mixing unsupervised and supervised algorithms in image classification model

I am trying to replicate the general image classification model used in a paper that I cite later below. The following image is an extract from a paper that proposes a novel method of performing image ...
0
votes
2answers
27 views

customer segmentation with unbalanced data

I am trying to do a customer segmentation on my transactional data and I am struggling a little bit on the best approach. Since it is an unsupervised model I can throw it to any algorithm and get some ...
0
votes
1answer
19 views

How would you quantify an experience into a score without labeled data

How would you approach a scenario where you have to quantify an abstract notion like “customer experience” without having any labeled data? So basically what you have are bunch of variables that you ...
0
votes
1answer
18 views

Recommendation needed for unsupervised clustering on mixed data task

I have a task to perform unsupervised cluster analysis on mixed datatypes: images, physical and business measures – continuous and categorical. Businesswise: there are images of products and ...
0
votes
1answer
37 views

Doing predictive modeling on predicted value

It's a project that I'm working on. Here are the steps I took: I want to make a recommendation service based on the customer data. I first used a collaborative filtering method to get the recommended ...
1
vote
1answer
45 views

Anomaly detection thresholds issue

I'm working on an anomaly detection development in Python. More in details, I need to analysed timeseries in order to check if anomalies are present. An anomalous value is typically a peak, so a ...
0
votes
1answer
30 views

hierarchical clustering doesn't work as expected

I have a precomputed distance matrix. I'm trying to do an hierarchical clustering using scipy: ...
2
votes
0answers
15 views

Temporal outlier Analysis on sensor data

I am working to find anomaly/outliers in sensor data using unsupervised machine learning (without training dataset). I have around 20000 samples taken per minute of various sensors. I just need to ...
0
votes
0answers
7 views

Perform unsupervised anomaly identification with causation in Python?

I have a time series data, which contains information from various sensors measured at every 20 minutes interval. I would like to use information from all these sensors as features to a Deep Learning/...
3
votes
2answers
148 views

Suggestion for stacked modelling in machine learning

I have built several models on the training dataset and i am not happy with the results and I wish to club them all together and generate a new model, so here is my idea as i already have the results ...
1
vote
0answers
14 views

Choosing a distance metric and a clustering algorithm for time series

For every entity I have a corresponding time series which is built by a sliding window (win_size=7d, win_shift=3d, so we have overlapped windows) With every win-shift, we count how many users are ...
0
votes
0answers
10 views

Is it wise to include the target labels when a supervised learning problem is tackled as an unsupervised learning problem?

I have a problem which requires both a supervised and unsupervised learning approach. This is because I am trying to find some hidden clusters (if they even exist) beyond the labels of the dataset. ...
0
votes
1answer
26 views

Neural Networks for Unsupervised Learning

Why cannot we use neural networks for unsupervised learning problem. I do know that it can be used using the Kohenon’s Self Organizing Map (KSOM) but is this the only method that we can use or are ...
0
votes
1answer
13 views

Cluster evolution over time

I have a dataset of transactional data with customer ID and I want to segment the dataset into groups using cluster analysis. I'm interested in following the evolution of each cluster over time, but ...
1
vote
1answer
53 views

Risk score from Neural Network classifier (more than 2 categories)

I am trying to use a Neural Network to perform multiclass classification. The classes represent Insurance Risk Level. The most risky level is Level 1, the least risk corresponds to Level 10. The ...
1
vote
1answer
70 views

NER with Unsupervised Learning?

If we treated NER as a classification/prediction problem, how would we handle name entities that weren't in training corpus? For example, "James was born in England." James was labeled as a PERSON ...
0
votes
0answers
48 views

How to evaluate unsupervised KNN?

I'm creating a recommender system using an unsupervised nearest neighbors model to suggest similar publishers for a given publisher, advertiser combination. I'm wondering how to evaluate the model I ...
1
vote
0answers
32 views

How to tune / choose the preference parameter of AffinityPropagation?

I have large dictionary of "pairwise similarity matrixes" that would look like the following: similarity['group1']: ...
0
votes
0answers
28 views

Dataset Image creation suggestions

I am trying to create a dataset where the Text is mixed, e.g. " I love football" can be written as " I l()v3 F00tba11". The idea is that by using Tesseract I can find the pattern to match these two ...