Skip to main content

Questions tagged [unsupervised-learning]

Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.

Filter by
Sorted by
Tagged with
2 votes
2 answers
902 views

Unsupervised Classification for documents

I'm trying to create a classifier in which there is less "manual" work for the user. For less manual work I mean that there won't be an initial phase of manual labeling of a training set, like in ...
Nikaido's user avatar
  • 135
2 votes
1 answer
2k views

Online Variational Autoencoder

when training a VAE, typically one samples from the latent distribution using the reparametrization trick using a fairly large minibatch size (>100) in the decoder/generator half of the VAE. I'm ...
NMR's user avatar
  • 33
2 votes
1 answer
600 views

In Orange Data Mining, how do I use results from clustering a training-set to test and score a test-set?

I am performing analysis on the well-known 'Adult' data-set, available on UCI using Orange Data Mining. In a PhD thesis, Pelleg (2004; pg 79) uses unsupervised clustering of the prescribed training ...
Paul Higgins's user avatar
2 votes
0 answers
74 views

Multi-Class Document Classification with both known and un-known classes

Currently, I am building a multi-class document classifier which has to classify either 3 known classes, namely "Financial Report", "Insurance_Sheet", "Endorsement", and ...
Quan Nguyen Ha's user avatar
2 votes
1 answer
63 views

Tiering after clustering with Kmeans

I would like to have some suggestions on possible avenues that would make sense in the following context. 3 Optimal clusters have been identified in a 5000 list of customers using Kmeans Data model ...
Roger Steinberg's user avatar
2 votes
0 answers
23 views

What technique's can be used to identify and count individual animals in a dataset?

Problem: I have an image dataset that contains a lot of different chitals (a species of deer). The images are taken by cameratraps in a National Park. I would like to count the individual animals. For ...
hyilmaz's user avatar
  • 21
2 votes
0 answers
430 views

Unsupervised document similarity state of the art

I have a set of N documents with lengths ranging from 0 to more than 20000 characters. I want to calculate a similarity score between 0 and 1 between all pairs of documents where a higher number ...
user7017793's user avatar
2 votes
0 answers
60 views

unsupervised learning time series datasets

I experiment on building electricity power consumption datasets and try to see relationships of the power consumption with weather data and dummy variables that represent time-of-week. The only thing ...
bbartling's user avatar
  • 403
2 votes
1 answer
974 views

Dendrogram: ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all()

I am trying to plot a Dendrogram to cluster data but this error is stopping me. My datea is here. I first chose columns to work with: ...
Sam.H's user avatar
  • 121
2 votes
0 answers
71 views

Intuition behind One Class SVM (Scholkopf)

I am trying to understand the intuition behind the idea of finding a hyperplane that separates the training data from the origin in the feature space. Why separation from origin with a hyperplane ...
batman's user avatar
  • 149
2 votes
0 answers
502 views

K-Means Clustering Profile Plot & Data Normalization

I am new to k-means clustering and I am working on a project on cryptoanalysis. I have a few questions and I hope to get some help here. I have four variables and my variables data values can range ...
Fabian Tan's user avatar
2 votes
2 answers
431 views

Pre-processing mixed data prior to clustering

I am new to hierarchical clustering, and wish to perform clustering on mixed data. I am slightly confused on the necessary pre-processing steps. I understand how to pre-process purely continuous data, ...
Tom Evans's user avatar
2 votes
2 answers
219 views

Semantic Search

There is a problem we are trying to solve where we want to do semantic search on our set of data, i.e we have a domain specific data (example: sentences talking about automobiles) Our data is just a ...
Farhaan Bukhsh's user avatar
2 votes
1 answer
57 views

Unsupervised Learning upon a column of dataset (graph shown) [closed]

I’m new into Machine Learning so here I am asking for a sanity check, if the question I am asking is even reasonable. I have a Dataset of columns, so I want to call one of the columns from the csv ...
E199504's user avatar
  • 605
2 votes
0 answers
101 views

From unsupervised to supervised in fraud detection

I have a question. I am working on the fraud detection domain. And I have data from imports to the country. As you can get from the title, I have unsupervised data. I do not know that the record is ...
gammazplaude's user avatar
2 votes
1 answer
60 views

What's the good index to choose number of clusters so that obtained clusters are homogeneous?

I perform a clustering on one-dimensional dataset and I need a way to automatically decide what's the optimal number of clusters from $k \in \{2, 3, 4, 5, 6\}$. The number of observations to cluster ...
jakes's user avatar
  • 95
2 votes
1 answer
514 views

Anomaly detection - relation between thresholds and anomalies

I'm developing an anomaly detection program in Python. Main idea is to create a new LSTM model every day, training it with the previous 7 days and predict the next day. Then, using thresholds, find ...
Giordano's user avatar
  • 345
2 votes
0 answers
25 views

Temporal outlier Analysis on sensor data

I am working to find anomaly/outliers in sensor data using unsupervised machine learning (without training dataset). I have around 20000 samples taken per minute of various sensors. I just need to ...
sdave1's user avatar
  • 21
2 votes
1 answer
674 views

How to tune / choose the preference parameter of AffinityPropagation?

I have large dictionary of "pairwise similarity matrixes" that would look like the following: similarity['group1']: ...
Mehdi's user avatar
  • 121
2 votes
0 answers
170 views

PCA for unsupervised feature selection [closed]

If I understood correctly, "using results of PCA to select features" (as recommended in this answer) implies visually analysing bi-plots of first two principal components - i.e. the angle ...
anomaly_detection's user avatar
2 votes
0 answers
551 views

Training detector without bounding box data

From what I can see most object detection NNs (Fast(er) R-CNN, YOLO etc) are trained on data including bounding boxes indicating where in the picture the objects are localized. Is there any model ...
Rahul's user avatar
  • 21
2 votes
0 answers
191 views

Exploratory analysis and feature engineering for time till failure prediction using sensor data of engines

I am trying to do some data exploration and analysis on a dataset of engine sensor readings. I would like to determine if the data I have is good enough to predict a time till failure and possibly ...
broffesor_matt's user avatar
2 votes
4 answers
133 views

Unsupervised clustering without of Data which is supposed to be on a linear function

When I have a dataset where each datum has x and y, and the (x,y) has a relation of one of <...
kensuke1984's user avatar
2 votes
1 answer
91 views

Conceptual clustering with sklearn?

How can I perform conceptual clustering in sklearn? My use case is that I have English Wikipedia articles that I'm doing unsupervised learning on (tfidf -> truncated svd -> l2 normalize), and I'd like ...
michaelsnowden's user avatar
2 votes
0 answers
2k views

SOM initial values for learning rate and neighborhood sigma

I am using SOM (Self-Organizing Maps) of Kohonen, or more specifically, the MiniSom, found here to cluster and visualize my data. As you can see in the above site, the example given is: ...
passion's user avatar
  • 121
2 votes
0 answers
4k views

Grid Search on Unsupervised Sklearn Clustering?

I am trying to use clustering algorithms in sklearn and am using Silhouette score with cosine similarity as a metric to compare different algorithms. My question is due to the varying hyperparameters ...
GNMO11's user avatar
  • 121
2 votes
0 answers
36 views

With EM algorithm, can you infer the location and variance of each "peak" in a pdf? Gaussian Mixture Models?

When I plot my data into bins, there is a frequency of data points per bin, which I can plot with a histogram. Based on this probability density function, I would like to find the maximum likelihood ...
ShanZhengYang's user avatar
2 votes
0 answers
133 views

Using classification to find the best support and confidence measure in associative rule mining

I have been trying the find the best support and confidence values for associative rules mining. I came across the following approach from an answer on Quora - Picking the "appropriate" values for ...
kusur's user avatar
  • 129
2 votes
1 answer
98 views

Decision tree to get difference in rates in two groups?

I have two sample groups of customers, each customer has 100s of features. For a single sample, i would use Decision Trees to find sub-groups that have a high churn rate. Thats easy. However, my ...
Arslán's user avatar
  • 131
1 vote
2 answers
431 views

Confused about the different aspects in Machine Learning [closed]

After reading different articles about ML and algorithms, scientist tends to use different words when describing the different aspects in ML. So now I'm a bit confused myself and I hope you can ...
Anfield_08's user avatar
1 vote
2 answers
49 views

Unsupervised Learning - Using the Outcome of Learning

My understanding in Unsupervised Learning is that -- when you want a computer to learn on its own by examining a large dataset. The goal is to establish some form of cluster or association-based ...
ha9u63a7's user avatar
  • 141
1 vote
3 answers
921 views

Applying and Visualizing k means clustering on a data set that has 9 features

I had a data set of images that I have extracted 9 numerical features that I want to apply k means clustering or hierarchical clustering to. I'm just not sure how to go about it. The tutorials I have ...
somedude1234's user avatar
1 vote
4 answers
2k views

How to evaluate the clustering result when cluster numbers are not equal to data set class

When apply clustering algorithm with the multi-class data set and class numbers are not equal to the result cluster numbers(For example , When we use K-Means algorithm by setting K = 3 apply with "...
tanawatl's user avatar
  • 237
1 vote
1 answer
101 views

How to Justify Anomalies Detected by Unsupervised Anomaly Detection Models? [closed]

I'm working on an unsupervised anomaly detection project involving a large sensor dataset, where I aim to identify anomalies without the aid of labeled data. While I've implemented several ...
Jais Varghese Joseph's user avatar
1 vote
2 answers
88 views

Clustering initialization

I'm running into a problem while working on clustering. I work on data with white Gaussian noise. All of the methods I have come across use some sort of random initialization to set up the mean and ...
Engineer's user avatar
  • 145
1 vote
1 answer
676 views

News topic detection and categorization

If I want to get how many and what kind of topics are covered by New York Times each week from a bag of words model(All the news covered by NYT in a week) how should I approach? Using traditional ...
user33680's user avatar
1 vote
1 answer
689 views

Topic classification on text data with no/few labels

I would like to achieve a classification of a text input into predefined categories. From what I have understand unsupervised approach are unfeasible if my target label is something very rare in ...
Andrea's user avatar
  • 45
1 vote
1 answer
10k views

Clustering based on distance between points [closed]

I am trying to cluster geographical locations in such a way that all the locations inside each cluster are at max within 25 miles of each other. For this, I am using Agglomerative clustering. I am ...
Karthik Katragadda's user avatar
1 vote
1 answer
4k views

Understanding Contrastive Divergence

I’m trying to understand, and eventually build a Restricted Boltzmann Machine. I understand that the update rule - that is the algorithm used to change the weights - is something called “contrastive ...
Finn Williams's user avatar
1 vote
1 answer
95 views

Can clustering my data first help me learn better classifiers?

I was thinking about this lately. Let's say that we have a very complex space, which makes it hard to learn a classifier that can efficiently split it. But what if this very complex space is actually ...
Valentin Calomme's user avatar
1 vote
1 answer
74 views

Stuck implementing k means for big and small dogs - dodgy results

my algortithm isnt working. The code seems to make sense and everything but im just not convinced with the results. I fell like the centroids should be amongst the data more, sort of central, but they ...
Finn Williams's user avatar
1 vote
1 answer
207 views

Approach for unsupervised time series clustering/segmentation

I have a big sample of data on a human's everyday life. The snapshots of the life are taken every 5 minutes. The data include time, the location of the human, accelerometer data, gyroscope data, and ...
Gabriele's user avatar
  • 133
1 vote
2 answers
120 views

unsupervised anomaly detection for univariate fast frequency time series data?

I have a univariate time series (there is a value for each time sampling) (sampling time: 66.66 micro second, number of samples/sampling time=151) coming from a scala customer This time series ...
user10296606's user avatar
  • 1,844
1 vote
1 answer
266 views

what is meant by minimizing and maximizing in GANs?

It is a subtle change that involves the generator maximizing the log of the discriminator probabilities for generated images instead of minimizing the log of the inverted discriminator probabilities ...
Nauman Akram's user avatar
1 vote
1 answer
2k views

Can Shapley/Lime values be used for unsupervised learning?

One thing that is really useful when trying to understand what a machine learning model does, is seeing why some instances got predicted. For that Shapley Values and Lime are really usefull. But can ...
Carlos Mougan's user avatar
1 vote
1 answer
354 views

Effects that Empty cells have in Unsupervised Machine Learning

I have a dataframe in a csv file that I would like to perform different unsupervised machine learning algorithms on. The file itself has some cells that are not ...
E199504's user avatar
  • 605
1 vote
2 answers
310 views

Clustering Small Text Descriptions

Im presented with a unique text classification problem. Im given a list of descriptions each containing 3-8 words. I know that there are some descriptions that are nearly the same, but the majority ...
Chad Van De Hey's user avatar
1 vote
1 answer
48 views

How to find similar points to a positive set when you don't have any negative set?

The task I'm used to do is the following. A client comes to see me with a set of clients (called positive companies) and he wants me to find other similar prospects. Usually, he also gives me a set of ...
Dust009's user avatar
  • 113
1 vote
2 answers
410 views

How to use a deep learning algorithm to cluster image *styles* in an unlabeled data set?

I have a hard problem, and I'd be interested in hearing people's thoughts. I have a set of images depicting a large variety of phenomena, but only a few styles. The images are already labeled by ...
Zorgoth's user avatar
  • 133
1 vote
1 answer
69 views

Machine Learning to predict risk of items [closed]

I'm trying to find out what I need to research and start learning to try and apply machine learning to this problem: In multiple offices I have 20 chairs, all of these chairs will need to have a ...
li x's user avatar
  • 113

1 2 3
4
5
10