Questions tagged [unsupervised-learning]

Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.

Filter by
Sorted by
Tagged with
0
votes
0answers
10 views

Can someone provide me the code of the MiLoF(Memory Efficient Local Outlier Factor) algorithm?

I have to code the MiLoF algorithm for detecting outliers in an unsupervised manner using non-stationary data. I am attaching the paper which explains the algorithm here. However, there are many ...
1
vote
1answer
19 views

What is the most effective unsupervised ML algorithm to use when outliers are present in data set?

I am analyzing a portfolio of about 225 stocks and have gotten data for each of them based on their "Price/Earnings ratio", "Return on Assets", and "Earnings per share growth". I would like to cluster ...
0
votes
0answers
12 views

How do I evaluate a K-Means unsupervised anomaly detection approach?

how do I evaluate K-means clustering anomaly detection method as there is no labelled data of anomaly class. To find the cluster (K), I have used the silhouette score from Scikit learn library. Scikit ...
1
vote
2answers
21 views

Pre-processing mixed data prior to clustering

I am new to hierarchical clustering, and wish to perform clustering on mixed data. I am slightly confused on the necessary pre-processing steps. I understand how to pre-process purely continuous data, ...
0
votes
2answers
30 views

Percentile as a threshold for Anomaly Detection?

I'm following this article about Unsupervised Anomaly Detection Algorithms. In this article, a threshold value is calculated using the scipy score percentile method to determine whether the point is ...
0
votes
2answers
35 views

Is there an unsupervised learning algorithm that can cluster data based on more than two dimensions?

I am just beginning to get into data science and have never posted here before, apologies if this question is worded incorrectly! I am curious if there is an unsupervised machine learning algorithm ...
0
votes
0answers
19 views

Summarize events per ID

Data: Each corresponds to an event (a person's visit to the hospital, as an example). I have a series of data associated with this event (duration of visit, motive, etc...). Objective: Summarize the ...
-2
votes
0answers
20 views

Clustering for Mixed Categorical and Numeric Data

My dataset contains three categorical variables and three numerical variables. brand model trim mileage price age honda civic vti oriel 91000 1750000 8 I need to find ...
1
vote
0answers
23 views

Benchmarking unsupervised learning by second stage classifiers?

Suppose we have a dataset $\{x^{(i)}, y^{(i)}\}_{i = 1}^N$ where $x^{(i)} \in \mathbb{R}^n$ and $y^{(i)} \in \{0, 1\}$ for simplicity. Our main goal is to apply some unsupervised learning algorithm ...
3
votes
0answers
34 views

What are the data preprocessing steps required before running K-Modes?

I have a clustering task at hand. The data that I have contains only categorical variables. So, k-modes seemed like the best option. But I am not sure what are the data pre processing steps required ...
-1
votes
1answer
23 views

Unsupervised learning algorithms for object detection

Are there any unsupervised learning techniques or algorithms which can be used for object detection just like CNN? If there are any, what are your thoughts on it? Thanks for your time.
0
votes
1answer
19 views

Hot Encode vs Binary Encoding for Binary attribute when clustering

I am planning to use data for a clustering problem that contains a column with a binary value BUY/SELL. Should I be converting this attribute and assign it binary values (BUY=1, SELL=0), and keep it ...
-1
votes
1answer
25 views

How to speedup K-Means used in 'for loop'

I'm trying to solve an interesting problem. One solution which seems to work well, involves using K-Means in a 'for loop'. The dataset per loop is independent and fairly small (Minibatch not required)....
1
vote
1answer
18 views

Clustering sequences of sentence embeddings

I have a sequence of events, right now I am not worried about their actual times, just the order. This is a sequence of web page views. I have modelled my data as the following, where each element ...
1
vote
0answers
16 views

Anomaly Detection Methods for Clean Training Data

The goal is to find anomalies in my dataset, univariant anomaly detection which basically means looking for anomalies only in one column. Up until this point I have tried DBSCAN; Isolation Forest and ...
2
votes
1answer
34 views

Adding anomalies to the Dataset

Recently I have been trying different Scikit-Learn anomaly detection clustering methods, like DBSCAN Isolation Forest. Based on how many training data I use, how I tweak on the algorithms ...
0
votes
1answer
18 views

Clustering cartesian coordinates associated with 1 categorical feature

I have a series of 2D coordinates X = {x, y}. Each are associated with one categorical variable W that can take 7 different values. E.g: ...
0
votes
0answers
23 views

Online courses for Anomaly Detection

As the title say I have been looking for some online courses that would teach me about anomaly detection using Unsupervised Machine Learning. I want to focus only on Scikit-Learn and not go deeper ...
0
votes
0answers
10 views

Unsupervised classification of satellite images sequences derived from time series with SOFM in python?

I have the following data: Up to 2 images per day (time series from 2015 - 2019 with gaps) for a specific region (AOI - Germany - Hesse) with 2 variables (soil moisture, precipitation). Out of this ...
0
votes
0answers
42 views

Semantic Search Help

There is a problem we are trying to solve where we want to do semantic search on our set of data, i.e we have a domain specific data (example: sentences talking about automobiles) Our data is just a ...
1
vote
0answers
9 views

Clustering and producing final results to find next best customer to target(Ranked)

I have a problem where I need to cluster customer data that has all possible attributes to identify the next potential customer who can succeed the last customer in terms of buying a certain product. ...
0
votes
0answers
13 views

PyOD algorithms that support datasets with empty cells

The idea is to find outliers in large datasets. What am I going to use to detect these outliers? I am going to use a library called PyOD for the detection of anomalies which was developed by Yue ...
1
vote
1answer
23 views

Effects that Empty cells have in Unsupervised Machine Learning

I have a dataframe in a csv file that I would like to perform different unsupervised machine learning algorithms on. The file itself has some cells that are not ...
0
votes
0answers
21 views

Are there any methods to detect whole multivariate time-series as anomalous from a set of multivariate time-series?

Consider a scenario with Dataset D as {T1, T2, ..., Tn} and Ti is a multivariate time-series of length mi as {X1, X2, ..., Xmi}. Here each record of the time-series Xi is a vector of attribute values {...
-1
votes
2answers
56 views

Clustering data set with multiple dimensions

I have a data set which is similar to the following: It is recipe data along with the composition of the recipe (in %) I have 91 recipes and 40 ingredients in total. I want to be able to cluster ...
2
votes
1answer
45 views

Are there any examples other than anomaly detection where unsupervised deep learning could be useful?

I am new to deep learning and its concepts. After reading a while I understood that unsupervised deep learning techniques usually try to reconstruct the input data(probably with less number of ...
0
votes
0answers
10 views

How to find top N neighbors of a datapoint in a cluster sorted in increasing order of distance from that point?

I am doing a clustering exercise and I am doing it using K-Means. After doing the clustering part, I have a dataframe that looks something like this : ...
2
votes
2answers
45 views

Importance of Variance in Machine Learning

Selecting a column from a Dataframe, plotting its Histogram using matplotlib and then finding the variance is the steps that I have to take for this part of a project. The final goal of the project ...
4
votes
1answer
23 views

semi supervised learning doubt only classify points with confidence above threshold

I currently have a dataset with approximately 5% labelled points and 95% unlabelled. I would like to label some of the unlabelled points only if I am very confident and leave the rest NaN. Personally ...
3
votes
0answers
154 views

How to do feature selection for clustering and implement it in python?

I am trying to implement k-means clustering on 60-70 features and I came across a post for feature selection technique on quora by Julian Ramos, but I fail to understand few steps mentioned. I am ...
5
votes
1answer
51 views

How do I interpret my result of clustering?

I am working on a clustering problem. I have 11 features. My complete data frame has 70-80% zeros. The data had outliers that I capped at 0.5 and 0.95 percentile. However, I tried k-means (python) on ...
2
votes
2answers
140 views

Document similarity

I have close to 50000 documents in plain text format. Is there a way in which I can group similar documents together? Similarity mostly here is the content similarity. Will transforming the text ...
2
votes
1answer
32 views

Unsupervised Learning upon a column of dataset (graph shown) [closed]

I’m new into Machine Learning so here I am asking for a sanity check, if the question I am asking is even reasonable. I have a Dataset of columns, so I want to call one of the columns from the csv ...
3
votes
1answer
60 views

What value can I gain by doing exploratory data analysis on features (and thus data) before doing clustering?

This might not be a very good question, but I would still ask if it's beneficial to do EDA before running a clustering algorithm? I understand that EDA helps us generate good and helpful insights ...
2
votes
2answers
98 views

How to deal with with rows with zero in every feature while clustering?

I am working on a clustering problem which has 13000 observations and 15 features. Around 3000 observations in the dataset has zero in every features ( i.e all values zero in 3000 rows). I am trying ...
1
vote
0answers
15 views

Built strong base for Unsupervised Learning [closed]

I’m am new into machine learning, recently I have put a task upon my shoulders to Detect Outliers in Dataset. The anomaly detection should be done using Unsupervised learning and preferably use ...
2
votes
1answer
56 views

Does K - Means clustering on data reduced using PCA and the original data make any difference?

I am working on clustering and I have 90 features with 13500 data points and after removing the correlated variables which had pearson correlation more than 90% my feature space reduced to 70. Also, ...
0
votes
0answers
34 views

Learning Process in Machine Learning

I want to use Unsupervised Leaning to detect Anomalies within a huge Csv file (consisting of headers that are named and thousands of rows belonging to them) Here on this link I have read about the ...
1
vote
2answers
77 views

Does it make sense to do train test split when trainning GANS?

For normal supervised learning the dataset is split in train and test (let's keep it simple). Generative Adversarial Networks are unsupervised learning but there is a supervised loss function in the ...
2
votes
0answers
45 views

From unsupervised to supervised in fraud detection

I have a question. I am working on the fraud detection domain. And I have data from imports to the country. As you can get from the title, I have unsupervised data. I do not know that the record is ...
2
votes
1answer
20 views

When unsupervised learning is more beneficial in comparison with supervised learning even the labelings are existed?

When unsupervised learning is more beneficial in comparison with supervised learning even the labeling are existed? If there is no labeling the unsupervised learning is better than supervised learning ...
2
votes
2answers
53 views

What is a manifold for Unsupervised Learning?

I've been watching Dr. G. Hinton lectures on Neural Networks in Machine Learning, and in one of the lectures he explains what the goals of Unsupervised Learning are. I am having trouble ...
0
votes
0answers
13 views

Interaction with unseen data (Generalization and evaluation the performance on unseen data in supervised and unsupevised learning methods)

How to generalizes model and performs on unseen data for a highly imbalanced binary classification problem (99.827%,0.173%)? 1-When using supervised learning methods such as logistic reg, RFs, ...
1
vote
0answers
21 views

How we compare two paragraphs of unlabelled dataset semantically (STS)?

Column representation: Unique_id | Text1 | Text2 Unique_id 0 Text1 public show for reynolds suspension of his coaching licence. portrait sir joshua reynolds portrait of omai will get a public airing ...
1
vote
2answers
78 views

How can we perform STS(Semantic Textual Similarity) on UnSupervised dataset using Deep Learning?

How to implement STS(Semantic Textual Similarity) on unlabelled dataset. Dataset column contains Unique_id, text1(contains paragraph), text2(contains paragraph). Ex: Column representation: Unique_id ...
2
votes
1answer
20 views

Clustering Weekday Weekend Data and Multicollinearity

Hi I have data of weekday and weekend step counts in which I extracted metrics from them such as the wd steps, we steps, standard deviation of wd steps, standard deviation of we steps and so on... <...
2
votes
1answer
26 views

Is there a paper accomplishing finding physical law from observation without premade perception, using machine learning?

For example: Isaac Newton finds law of universal gravitation just by looking a falling apple, without any premade perception of that phenomenon. Is it possible to accomplish that kind of discovery ...
2
votes
0answers
31 views

What's the good index to choose number of clusters so that obtained clusters are homogeneous?

I perform a clustering on one-dimensional dataset and I need a way to automatically decide what's the optimal number of clusters from $k \in \{2, 3, 4, 5, 6\}$. The number of observations to cluster ...
6
votes
3answers
112 views

Why spectral clustering results in disjointed cluster?

I'm working on a project where I have to dynamically cluster the position of objects with respect to one coordinate. So I'm essentially dealing with subsequent frames and each frame represents a one-...
3
votes
1answer
28 views

Is pattern recognition the same as unsupervised learning? Is machine learning the same as supervised learning?

Firstly, here is the definition of a well-posed learning problem: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its ...

1
2 3 4 5
7