Questions tagged [unsupervised-learning]

Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.

Filter by
Sorted by
Tagged with
0 votes
0 answers
30 views

Machine learning can be broken down into supervised, unsupervised, and reinforcement learning. Is there anything else?

Is it even logically possible to have a type of machine learning that doesn't fall into those three paradigms? Supervised: a dataset of inputs and outputs are fed to an algorithm which learns a ...
Austin Capobianco's user avatar
0 votes
0 answers
12 views

Clustering of two datasets in different years

I want to analyze two datasets by running a clustering algorithm on both and comparing the results. The two datasets have the same variables. The only difference is that one dataset is from 2010 and ...
Ahmad Bhatti's user avatar
0 votes
0 answers
19 views

How to process dictionary of 2d-arrays with River?

I want to perform non-supervised anomaly detection of water quality spectra (2d-array: intensity vs wavelength) using incremental River python library. As far as I have seen, the input data of an ...
Marianne's user avatar
0 votes
0 answers
14 views

Ranking risky routes

i'm looking to get the top 10 rank of the most dangerous routes. I have a routes table where each row is a route and it has features such as avg daily traffic for past three years No. of times where ...
Joe's user avatar
  • 1
1 vote
1 answer
31 views

Removing specific phrases from textual data (R)

I have the following reddit posts and I would like to clean the posts and remove from the data the specific phrase "Click to expand", while keeping all other words within a post the same. <...
maldini1990's user avatar
0 votes
0 answers
15 views

Unsupervised rule extraction of categorical data

I have a dataset of network traffic with three features that I would like to extract rules from in order to apply firewall/flow control rules i.e. the permitted flows. I am able to classify a ...
user153170's user avatar
0 votes
1 answer
39 views

9-dimensional data classification

I have a dataset with 9 columns and 20 million rows. I have now split the dataset into two parts using IQR. One part is my inliers, the other part is my outliers. I now want to build a categorizer ...
Sponge_Data_45's user avatar
1 vote
1 answer
72 views

How to Justify Anomalies Detected by Unsupervised Anomaly Detection Models? [closed]

I'm working on an unsupervised anomaly detection project involving a large sensor dataset, where I aim to identify anomalies without the aid of labeled data. While I've implemented several ...
Jais Varghese Joseph's user avatar
0 votes
1 answer
28 views

Overfitting in the trained model

For my project on the classification problem of predicting churn customers, I trained various base models using k-fold validation on the training dataset and out of which random forest gave the best ...
Yuvika Yadav's user avatar
0 votes
0 answers
15 views

How to update the weights of a ranking algorithms through unsupervised learning?

I have, let's say, 100 samples with n features, and I want to rank them by learning the weights for each feature. Weights must be learned from implicit ratings from the user (eg. user click). However, ...
Naman Lazarus's user avatar
0 votes
1 answer
32 views

Different scaling methods of different features results in a faux dependency between them

My dataset contains the following two features: "movie duration" (minutes) and "tv shows duration" (seasons). If a certain sample is of type "movie", it's duration will ...
Shir's user avatar
  • 1
0 votes
1 answer
81 views

Clustering task: drop or not drop a categorical attribute/feature for which each row in the dataset contains a different value

I am dealing with a clustering task. In the dataset I am using there is a categorical feature and for each row in the dataset I have a different value for that feature (my dataset consists of 1000 ...
vito97's user avatar
  • 1
0 votes
0 answers
11 views

What kind of learning do I need ? (use-case specific)

Consider a scenario where I have a model trained on gesture videos (say a 3D ResNet). I am looking for a technique (or a combination) that allows me to further train the model every time I have a new ...
batman's user avatar
  • 149
0 votes
0 answers
22 views

Using Autoencoder in Python for Data Selection and Risk Level Calculation

I have a dataset that includes 100 data points from each sensor, representing various measurements. These measurements can be used to calculate the level of risk associated with each sensor. However, ...
JACK1's user avatar
  • 1
0 votes
0 answers
46 views

Equivalence between K-Means objective functions proof

I am working on a project involving the K-Means clustering algorithm, and I am trying to prove the equivalence between different formulations of the objective function. Specifically, I want to show ...
Oren Ben Eliyahu's user avatar
0 votes
0 answers
18 views

Unable to reproduce result for CIFAR10 with ResNet50 backbone and SIMCLR self-supervised algorithm

I am recently working on self-supervised learning, particularly simclr paradigm. I found it hard to reproduce results on cifar10 with linear evaluation. I now get round 60%, while the paper reports 80%...
noah0822's user avatar
2 votes
1 answer
74 views

How to improve the preservation of the global data structure in UMAP?

I have a dataset, where the features are comprised of points arranged in a regular grid on a simplex. Each of these points are defined as follows: A point $\mathbf{x}$ on the simplex can be ...
pmu2022's user avatar
  • 121
0 votes
0 answers
21 views

Can TSNE and other visualisation methods separate multivariate normal blobs?

Consider we have two classes of points. Both of them come from a multivariate normal distribution with an unrestricted covariance matrix. Let's assume, that the densities of those distributions do not ...
Karol Szustakowski's user avatar
0 votes
1 answer
351 views

Rigth way to find Lorentzian distance between 2 point

Following this paper and this paper, I'm trying to implement the formula for the Lorentzian distance between 2 points (aka the distance between 2 points in Lorentzian space). I'll use this a the ...
Ricardo Chavarria's user avatar
1 vote
0 answers
15 views

What algorithm should I use when trying to find closest record matches when records contain both categorical and discrete attributes?

I have 100000 records that have discrete features like topic (analytics etc) and categorical features like ticket details (eg: I need help with analytics for my business). When creating a new record, ...
Jack Smith's user avatar
0 votes
0 answers
64 views

How to use IsolationForest for anomaly detection of a 1D-array, given given another feature?

[Beginner at ML] Hi, I would like to use unsupervised anomaly detection of spectrograms. Currently, I am trying IsolationForest on a bunch of 1D-arrays (light intensity vs wavelength) in order to ...
Marianne's user avatar
0 votes
0 answers
24 views

Varying feature vector lengths for learning

[I am a total beginner in machine learning algorithms] I have 10 spectrograms (lines) for phytoplankton (each composed of 288 points). Each spectrogram is associated with a phytoplankton dendity data ...
Marianne's user avatar
0 votes
0 answers
24 views

Can sigmoid function applied on distance calculation of Self Organizing Map improve the accuracy?

I just thought how sigmoid function helps logistic regression making more accurate classification and wonder if it can be applied on distance calculation of Self Organizing Map and making more better/...
darieem's user avatar
0 votes
0 answers
8 views

How do you know which examples of data are the most accurate classified using Self Organizing Map?

Here is an example of data where I apply the SOM general rule: If distance1 > distance2 than class C1 else class C2 ; So in this example I use a self organizing map which clusters only two classes ...
darieem's user avatar
0 votes
2 answers
1k views

Why is UMAP used in combination with other Clustering Algorithm?

I've noticed that UMAP is often used in combination with other clustering algorithms, such as K-means, DBSCAN, HDBSCAN. However, from what I've understood, UMAP can be used for clustering tasks. So ...
coelidonum's user avatar
0 votes
1 answer
15 views

Unsupervised ranking of samples

Say I have a dataset of n samples. I want to maximize every feature’s value. I’m not sure if feature 1 is more important than feature 2, etc. Are there any methods of ranking my samples out there? If ...
mjw467's user avatar
  • 1
0 votes
1 answer
34 views

Statistical test for comparing number of clusters in data

I am performing $K$-means clustering on a dataset consisting of $n$ observations and $d$ variables, and I'm trying to determine the optimal number of clusters. Is there a test that can determine the ...
RyRy the Fly Guy's user avatar
2 votes
3 answers
761 views

What's the fastest clustering package in Python?

I'd like to perform clustering analysis on a dataset with 1,300 columns and 500,000 rows. I've seen that clustering algorithms are available in SciKit-Learn. But I'm worried that the algorithms will ...
Connor's user avatar
  • 617
2 votes
1 answer
20 views

Im looking for good neurons silmilarity metric

Recently I managed to create simple neural network visualization, to help to understand how neural network works on the signal level. I also wanted to arrange neurons by similarity cause I was ...
dismedia's user avatar
0 votes
0 answers
18 views

Scaling nominal vars K means

I had a discussion recently with a coworker. We are running a K means clustering algorithm. He said that when dummy variables are made into 1s and 0s,these columns must be scaled in a specific way. If ...
NKI's user avatar
  • 1
1 vote
1 answer
312 views

Topic classification on text data with no/few labels

I would like to achieve a classification of a text input into predefined categories. From what I have understand unsupervised approach are unfeasible if my target label is something very rare in ...
Andrea's user avatar
  • 45
0 votes
1 answer
527 views

How to compare labels from clustering analysis and original ones?

I was asked to run a clustering analysis to assess the validity of labels for a manually labelled dataset. I can simply save the actual labels (4 classes: 0, 1, 2, 3) and run clustering analysis (let'...
AngelMarcos's user avatar
0 votes
0 answers
28 views

General approach for "regime detection" within timeseries data

Assume a stock market type of dataset with a handful of timeseries, representing volume of trades, volatility of trades, and similar meta metrics. If necessary, assume that is on an hourly basis. What ...
Nik's user avatar
  • 101
0 votes
1 answer
31 views

ML Predicted Model for 2 values

I have a data set with 96 rows. It contains date, source, spend and number of customers. I have 4 different sources that generate customers and you can see in the dataset how much I spend and how many ...
Leonardo Urbiola's user avatar
0 votes
2 answers
106 views

NLP topic clustering

In my dataset, I have 500 abstracts. The goal is to cluster them in 2 topics. One topic must have those abstracts which contain some list of words or similar words and the rest of the abstracts in ...
Nency Bansal's user avatar
1 vote
1 answer
20 views

Underfitting and perfomance metrics in unsupervised methods

My question is simple and yet quite hard to find an answer to. In an unsupervised method, for example, when you have to reconstruct an input, how can you tell if your loss is good enough? Generally, ...
BilboBuggins's user avatar
0 votes
0 answers
10 views

Unsupervised vine trunk foreground segmentation

I'm currently working on a computer vision project for a vineyard robot. I trained a robust object detection for the vine trunks but now I need to apply semantic segmentation on the trunks so I can ...
hellvetica's user avatar
1 vote
1 answer
30 views

Predicting many classes, is it a known solution to build n-group classifiers?

Imagine you want to predict 2048 classes. Instead of asking one model to predict all of them at once, is it a known type of solution to have a model predict which cluster or group of classes an input ...
Jonathan Allen Grant's user avatar
1 vote
1 answer
46 views

Looking for an approach to automatically recognize faulted data

I'm currently looking into different approaches to handle the following problem: Let's say there is an X amount of Class A which embodies some kind of absence of work on a daily basis. E.g. Vacation, ...
bitfish31's user avatar
0 votes
0 answers
36 views

Video anomaly detection/ Evaluation AUC

I have trained an unsupervised anomaly detector for surveillance videos. After inference, I rescale the scores between max/min from the resulting scores array. scores = (scores - min(scores))/max(...
TecK97's user avatar
  • 1
0 votes
0 answers
19 views

How to find a unique feature vector for retail items?

I'm working on this problem where we get images of SKUs (Stock Keeping Unit) or in other words retail products and my job is to classify which product is it. I want to find a unique feature vector ...
Hissaan Ali's user avatar
0 votes
1 answer
35 views

detecting abnormality in a specific feature with respect to others (unsupervised?)

I have a large dataset with a feature y which is dependent in part on features x1 and x2. All features are noisy, and y is also dependent on other parameters not captured in the dataset. I would like ...
user18236139's user avatar
1 vote
1 answer
179 views

Encode categorical data for unsupervised learning

What is the best encoder for categorical data in unsupervised learning? I am using unsupervised learning on mixed data (such as K-means). Before running my unsupervised algorithm, I am using dimension ...
Julien PETOT's user avatar
0 votes
0 answers
15 views

Can/should a logical multi-class classification anomaly detection system be described as "unsupervised machine learning"?

I would like to ensure that my use of terminology is accurate. My question is: what terminology should I be using in this case? The system I am building assigns classes (-1, 0, +1) to observations ...
HackJob99's user avatar
0 votes
2 answers
323 views

Time Series - Anomaly Detection

I have time-series data with alerts (every minute) that I need to find anomalies in. I am looking for a library which can do unsupervised learning of this data and detect anomalies in the data. Which ...
Jilaba Hindga's user avatar
1 vote
2 answers
61 views

How to cluster components of a graph containing text data?

Suppose that I have a graph that has components like the image below. Graph nodes contain text data (titles) and the edges data is the similarity (percentage). I know that each component represents a ...
Reza Shabrang's user avatar
2 votes
1 answer
434 views

In Orange Data Mining, how do I use results from clustering a training-set to test and score a test-set?

I am performing analysis on the well-known 'Adult' data-set, available on UCI using Orange Data Mining. In a PhD thesis, Pelleg (2004; pg 79) uses unsupervised clustering of the prescribed training ...
Paul Higgins's user avatar
1 vote
1 answer
997 views

Clustering for Sparse Data Matrix of high dimension

I currently have a dataset of 1000 entries with 512 features that are sparse. I want to cluster them. I have attempted using kmeans, but found that the clustering wasn't very good, and have been ...
Is land's user avatar
  • 11
1 vote
2 answers
56 views

Validate Unsupervised Binary Classification

I’m working on a fully unsupervised anomaly detection problem. Since it’s completely unsupervised, I’m having hard times in defining some metrics to kind of validate the results (I run several ...
fpialcoi_o's user avatar
0 votes
0 answers
14 views

Find the most impactfuls parameters multivariate output unsupervised ML

I am currently on a proect where my df has more than 600 parameters of analog sensors (A parameters) and about 50 other parameters (F parameters). I want to find for each of these 50 parameters (F ...
Art's user avatar
  • 1

1
2 3 4 5
9