Skip to main content
OverflowAI is here! AI power for your Stack Overflow for Teams knowledge community. Learn more

Questions tagged [unsupervised-learning]

Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.

Filter by
Sorted by
Tagged with
0 votes
0 answers
12 views

What to put for X_train, y_train when using it for unsupervised LSTM for anomaly detection?

I have a dataset with 5 features (excluding the date) [Result, Ward, Age, Facility, Resource] . The train dataset has non-anomalous data, and the test dataset will have some anomalous data. This ...
Chiken's user avatar
  • 1
0 votes
0 answers
6 views

How to Identify Equipment Churn from Laboratory Service Records Without Direct Churn Labels?

I'm analyzing a dataset encompassing 20 years of laboratory equipment service records, which includes the equipment ID, service dates, types of equipment (HOOD_TYPE), and descriptions of performed ...
tlengman's user avatar
0 votes
0 answers
5 views

How do I determine syndicate and collusion indication with clustering and network analysis on a large unlabeled user transaction data?

How do I determine syndicate and collusion indication with clustering and network analysis on a large unlabeled user transaction data? So far, I've only been training with labeled data on fraud-...
user161454's user avatar
0 votes
0 answers
10 views

unsupervised clustering followed by modeling each cluster to create a mixed model

I am curious if this is an advisable approach. I am not applying this approach and am only interested in the theory of it. let's say you have some set of features X and target Y. X can account for ...
Phillip Maire's user avatar
0 votes
0 answers
44 views

Unsupervised Log Anomaly Detection

I am thinking about using the variational autoencoder model for anomaly detection . I have an Android Logs dataset. As the logs generated are a representative of time series type of data I thought ...
MLenthusiast's user avatar
1 vote
0 answers
38 views

Can unsupervised pretraining (autoencoders) be used for u-nets?

TLDR: Will a u-net pretrained as an autoencoder be able to learn a latent representation of the data if the encoder weights are frozen (can't game the system and pass forward the unmodified image)? ...
user avatar
1 vote
1 answer
23 views

Topic modeling evaluation

I'm working on topic modeling and I have generated clusters with two different methods. How can I evaluate which method performs better than the other?
user5520049's user avatar
0 votes
1 answer
19 views

Univariate anomaly / outlier detection

I'm facing a problem that seems 'easy,' but I've been struggling with it for a while now in the field of anomaly/outlier detection. I have a dataset of around 60K data points. Each data point is part ...
EyalG's user avatar
  • 1
0 votes
0 answers
20 views

Alternatives to Model-Based Feature Selection for Unsupervised Clustering

I am running a clustering model on a group of patients who are hypertensive with hopes of identifying different variations in clinical characteristics among hypertensive individuals. One of the issues ...
Zory Dory's user avatar
0 votes
0 answers
48 views

Unsupervised Learning with Time Series data?

I'm working with data which has daily aggregated user actvity over the course of several months (a user doesn't have to make an activity every day). I'm looking for a way to cluster similar users ...
user6132211's user avatar
0 votes
0 answers
9 views

I'm trying to build image search like Google Photo-Image with face is given to model & it'll get all the images in database in which he/she is present

When a user upload a selfie, the model search same person in dataset of images of multiple persons and get back all the images in which that person is present. Step 1: From dataset of images I detect ...
BKP's user avatar
  • 1
0 votes
0 answers
11 views

How can I detect patterns in a csv of numeric values ​with a success failure indicator?

Hello everyone I have a need and I would like help to identify the technological approach, I tentatively consider that I need unsupervised ml for the following: I have a data dataset about investment ...
Luis Madueño's user avatar
0 votes
0 answers
38 views

Machine learning can be broken down into supervised, unsupervised, and reinforcement learning. Is there anything else?

Is it even logically possible to have a type of machine learning that doesn't fall into those three paradigms? Supervised: a dataset of inputs and outputs are fed to an algorithm which learns a ...
Austin Capobianco's user avatar
0 votes
0 answers
18 views

Clustering of two datasets in different years

I want to analyze two datasets by running a clustering algorithm on both and comparing the results. The two datasets have the same variables. The only difference is that one dataset is from 2010 and ...
Ahmad Bhatti's user avatar
0 votes
0 answers
20 views

How to process dictionary of 2d-arrays with River?

I want to perform non-supervised anomaly detection of water quality spectra (2d-array: intensity vs wavelength) using incremental River python library. As far as I have seen, the input data of an ...
Marianne's user avatar
0 votes
0 answers
14 views

Ranking risky routes

i'm looking to get the top 10 rank of the most dangerous routes. I have a routes table where each row is a route and it has features such as avg daily traffic for past three years No. of times where ...
Joe's user avatar
  • 1
1 vote
1 answer
36 views

Removing specific phrases from textual data (R)

I have the following reddit posts and I would like to clean the posts and remove from the data the specific phrase "Click to expand", while keeping all other words within a post the same. <...
maldini1990's user avatar
0 votes
0 answers
18 views

Unsupervised rule extraction of categorical data

I have a dataset of network traffic with three features that I would like to extract rules from in order to apply firewall/flow control rules i.e. the permitted flows. I am able to classify a ...
user153170's user avatar
0 votes
1 answer
42 views

9-dimensional data classification

I have a dataset with 9 columns and 20 million rows. I have now split the dataset into two parts using IQR. One part is my inliers, the other part is my outliers. I now want to build a categorizer ...
Sponge_Data_45's user avatar
1 vote
1 answer
88 views

How to Justify Anomalies Detected by Unsupervised Anomaly Detection Models? [closed]

I'm working on an unsupervised anomaly detection project involving a large sensor dataset, where I aim to identify anomalies without the aid of labeled data. While I've implemented several ...
Jais Varghese Joseph's user avatar
0 votes
1 answer
30 views

Overfitting in the trained model

For my project on the classification problem of predicting churn customers, I trained various base models using k-fold validation on the training dataset and out of which random forest gave the best ...
Yuvika Yadav's user avatar
0 votes
0 answers
20 views

How to update the weights of a ranking algorithms through unsupervised learning?

I have, let's say, 100 samples with n features, and I want to rank them by learning the weights for each feature. Weights must be learned from implicit ratings from the user (eg. user click). However, ...
Naman Lazarus's user avatar
0 votes
1 answer
36 views

Different scaling methods of different features results in a faux dependency between them

My dataset contains the following two features: "movie duration" (minutes) and "tv shows duration" (seasons). If a certain sample is of type "movie", it's duration will ...
Shir's user avatar
  • 1
0 votes
1 answer
280 views

Clustering task: drop or not drop a categorical attribute/feature for which each row in the dataset contains a different value

I am dealing with a clustering task. In the dataset I am using there is a categorical feature and for each row in the dataset I have a different value for that feature (my dataset consists of 1000 ...
vito97's user avatar
  • 1
0 votes
0 answers
12 views

What kind of learning do I need ? (use-case specific)

Consider a scenario where I have a model trained on gesture videos (say a 3D ResNet). I am looking for a technique (or a combination) that allows me to further train the model every time I have a new ...
batman's user avatar
  • 149
0 votes
0 answers
28 views

Using Autoencoder in Python for Data Selection and Risk Level Calculation

I have a dataset that includes 100 data points from each sensor, representing various measurements. These measurements can be used to calculate the level of risk associated with each sensor. However, ...
JACK1's user avatar
  • 1
0 votes
0 answers
64 views

Equivalence between K-Means objective functions proof

I am working on a project involving the K-Means clustering algorithm, and I am trying to prove the equivalence between different formulations of the objective function. Specifically, I want to show ...
Oren Ben Eliyahu's user avatar
2 votes
1 answer
96 views

How to improve the preservation of the global data structure in UMAP?

I have a dataset, where the features are comprised of points arranged in a regular grid on a simplex. Each of these points are defined as follows: A point $\mathbf{x}$ on the simplex can be ...
pmu2022's user avatar
  • 121
0 votes
0 answers
28 views

Can TSNE and other visualisation methods separate multivariate normal blobs?

Consider we have two classes of points. Both of them come from a multivariate normal distribution with an unrestricted covariance matrix. Let's assume, that the densities of those distributions do not ...
Karol Szustakowski's user avatar
0 votes
1 answer
724 views

Rigth way to find Lorentzian distance between 2 point

Following this paper and this paper, I'm trying to implement the formula for the Lorentzian distance between 2 points (aka the distance between 2 points in Lorentzian space). I'll use this a the ...
Ricardo Chavarria's user avatar
1 vote
0 answers
25 views

What algorithm should I use when trying to find closest record matches when records contain both categorical and discrete attributes?

I have 100000 records that have discrete features like topic (analytics etc) and categorical features like ticket details (eg: I need help with analytics for my business). When creating a new record, ...
Jack Smith's user avatar
0 votes
0 answers
24 views

Varying feature vector lengths for learning

[I am a total beginner in machine learning algorithms] I have 10 spectrograms (lines) for phytoplankton (each composed of 288 points). Each spectrogram is associated with a phytoplankton dendity data ...
Marianne's user avatar
0 votes
2 answers
3k views

Why is UMAP used in combination with other Clustering Algorithm?

I've noticed that UMAP is often used in combination with other clustering algorithms, such as K-means, DBSCAN, HDBSCAN. However, from what I've understood, UMAP can be used for clustering tasks. So ...
coelidonum's user avatar
0 votes
1 answer
17 views

Unsupervised ranking of samples

Say I have a dataset of n samples. I want to maximize every feature’s value. I’m not sure if feature 1 is more important than feature 2, etc. Are there any methods of ranking my samples out there? If ...
mjw467's user avatar
  • 1
0 votes
1 answer
62 views

Statistical test for comparing number of clusters in data

I am performing $K$-means clustering on a dataset consisting of $n$ observations and $d$ variables, and I'm trying to determine the optimal number of clusters. Is there a test that can determine the ...
RyRy the Fly Guy's user avatar
2 votes
3 answers
1k views

What's the fastest clustering package in Python?

I'd like to perform clustering analysis on a dataset with 1,300 columns and 500,000 rows. I've seen that clustering algorithms are available in SciKit-Learn. But I'm worried that the algorithms will ...
Connor's user avatar
  • 651
2 votes
1 answer
22 views

Im looking for good neurons silmilarity metric

Recently I managed to create simple neural network visualization, to help to understand how neural network works on the signal level. I also wanted to arrange neurons by similarity cause I was ...
dismedia's user avatar
1 vote
1 answer
621 views

Topic classification on text data with no/few labels

I would like to achieve a classification of a text input into predefined categories. From what I have understand unsupervised approach are unfeasible if my target label is something very rare in ...
Andrea's user avatar
  • 45
0 votes
1 answer
913 views

How to compare labels from clustering analysis and original ones?

I was asked to run a clustering analysis to assess the validity of labels for a manually labelled dataset. I can simply save the actual labels (4 classes: 0, 1, 2, 3) and run clustering analysis (let'...
AngelMarcos's user avatar
0 votes
1 answer
37 views

ML Predicted Model for 2 values

I have a data set with 96 rows. It contains date, source, spend and number of customers. I have 4 different sources that generate customers and you can see in the dataset how much I spend and how many ...
Leonardo Urbiola's user avatar
0 votes
2 answers
147 views

NLP topic clustering

In my dataset, I have 500 abstracts. The goal is to cluster them in 2 topics. One topic must have those abstracts which contain some list of words or similar words and the rest of the abstracts in ...
Nency Bansal's user avatar
1 vote
1 answer
22 views

Underfitting and perfomance metrics in unsupervised methods

My question is simple and yet quite hard to find an answer to. In an unsupervised method, for example, when you have to reconstruct an input, how can you tell if your loss is good enough? Generally, ...
BilboBuggins's user avatar
1 vote
1 answer
32 views

Predicting many classes, is it a known solution to build n-group classifiers?

Imagine you want to predict 2048 classes. Instead of asking one model to predict all of them at once, is it a known type of solution to have a model predict which cluster or group of classes an input ...
Jonathan Allen Grant's user avatar
1 vote
1 answer
47 views

Looking for an approach to automatically recognize faulted data

I'm currently looking into different approaches to handle the following problem: Let's say there is an X amount of Class A which embodies some kind of absence of work on a daily basis. E.g. Vacation, ...
bitfish31's user avatar
0 votes
0 answers
48 views

Video anomaly detection/ Evaluation AUC

I have trained an unsupervised anomaly detector for surveillance videos. After inference, I rescale the scores between max/min from the resulting scores array. scores = (scores - min(scores))/max(...
TecK97's user avatar
  • 1
0 votes
1 answer
42 views

detecting abnormality in a specific feature with respect to others (unsupervised?)

I have a large dataset with a feature y which is dependent in part on features x1 and x2. All features are noisy, and y is also dependent on other parameters not captured in the dataset. I would like ...
user18236139's user avatar
1 vote
1 answer
275 views

Encode categorical data for unsupervised learning

What is the best encoder for categorical data in unsupervised learning? I am using unsupervised learning on mixed data (such as K-means). Before running my unsupervised algorithm, I am using dimension ...
Julien PETOT's user avatar
0 votes
0 answers
18 views

Can/should a logical multi-class classification anomaly detection system be described as "unsupervised machine learning"?

I would like to ensure that my use of terminology is accurate. My question is: what terminology should I be using in this case? The system I am building assigns classes (-1, 0, +1) to observations ...
HackJob99's user avatar
0 votes
2 answers
420 views

Time Series - Anomaly Detection

I have time-series data with alerts (every minute) that I need to find anomalies in. I am looking for a library which can do unsupervised learning of this data and detect anomalies in the data. Which ...
Jilaba Hindga's user avatar
1 vote
2 answers
77 views

How to cluster components of a graph containing text data?

Suppose that I have a graph that has components like the image below. Graph nodes contain text data (titles) and the edges data is the similarity (percentage). I know that each component represents a ...
Reza Shabrang's user avatar

1
2 3 4 5
10