Questions tagged [unsupervised-learning]

Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.

Filter by
Sorted by
Tagged with
-1
votes
1answer
118 views

How to use K-Means to detect users anomaly in Access Control

I'm currently working on access control project, Smart Lock to be more spesific. Like the other smart lock system, the system required user's authentication to open the door. I'm using RFID as ...
1
vote
0answers
22 views

Decision tree to get difference in rates in two groups?

I have two sample groups of customers, each customer has 100s of features. For a single sample, i would use Decision Trees to find sub-groups that have a high churn rate. Thats easy. However, my ...
1
vote
1answer
163 views

More weightage to a categorical feature for an Autoencoder model

I am using autoencoder for anomaly detection. I don't have any labels already and so its unsupervised. If I have categorical variables, I usually one hot encode before giving it to the model. I would ...
3
votes
1answer
427 views

ML Models: How to handle categorical feature with over 1000 unique values

I am trying to build an ML Classification model on a data set that contains quite a few categorical columns. However, few of them have over 1000 unique values. I am concerned that if I run one-hot ...
0
votes
1answer
47 views

How to split temporal sequences to sub-sequences in a meaningful yet unsupervised manner?

I have a biological process that undergoes some cellular event which I am observing. I have a series of events, with different temporal gaps between them. For example ...
3
votes
1answer
150 views

How to use a different model to deep neural network with reinforcement learning based on DQN?

Is it possible to implement a reinforcement learning algorithm without using a deep neural network (DNN) as used in deep reinforcement learning e.g. Deep Q-Network (DQN)? How can I replace the DNN in ...
2
votes
1answer
61 views

Given data that is labeled as outliers, how can I classify data as outliers?

I have a dataset that is a mixture of sparse binary features and quantitative features. I only have definite outliers labeled. How should I approach trying to classify unlabeled data? I considered ...
5
votes
3answers
14k views

Anomaly detection on time series

I've just started working on an anomaly detection development in Python. My data sets are a collection of timeseries. More in details, data are coming from some sensors/meters which record and ...
1
vote
3answers
597 views

Cross validation for anomaly detection using autoencoder

I am using autoencoder for anomaly detection in warranty data. I don't have any ground truth labels to confirm whether the anomalies detected by the model is really an anomaly or not. Since I don't ...
0
votes
1answer
639 views

using unsupervised learning algorithms on images

I am working on a project to classify images of types of cloth (shirt, tshirt, pant etc). While this is a standard supervised classification problem, the accuracy of the neural network is not good. ...
5
votes
1answer
688 views

What does it mean by “t-SNE retains the structure of the data”?

I was learning about t-SNE when I was told that t-SNE retains the structure of the data in the embeddings. What exactly does this mean ? How does the algorithm achieve this ? So far I have ...
-1
votes
2answers
58 views

How to identify clusters after multiple runs?

Suppose I run an unsupervised clustering algorithm. After multiple runs, I find clusters and would like to know if the same cluster was found more than once. For example: I can figure out A-orange, ...
1
vote
0answers
61 views

Semi-supervised Learning doubt

I'm reading "Hands on machine learning" by Aurelien Geron. He stated that semi-supervised learning is: Some photo-hosting services, such as Google Photos, are good examples of this. Once you ...
2
votes
3answers
723 views

What machine learning algorithms to use for unsupervised POS tagging?

I am interested in an unsupervised approach to training a POS-tagger. Labeling is very difficult and I would like to test a tagger for my specific domain (chats) where users typically write in lower ...
1
vote
0answers
55 views

Classifying variable types on a list of variables

I have a list of around 700 variables which I need to perform a variable cleanup on. What complicates things is there are different numeric codes which flag an invalid value and these differ by the ...
1
vote
0answers
40 views

Random Training set for GAN's [closed]

I have studies the gans in depth and some of its type like cycle, pix2pix, cgans. Now I want to generate random images from random distribution from generator. So I am creating a dataset with no ...
-1
votes
1answer
37 views

What is the most straightforward way to discover clusters in data? [closed]

I'm planning on extracting a number of word vector distances from a data set, and I want to be able to detect clusters within that set, with an undefined number of clusters that are dynamically ...
1
vote
1answer
348 views

Graph & Network Mining: clustering/community detection/ classification

I am working on graphs/networks where nodes and edges have some attributes. I want to know what algorithm exist for: 1) clustering a graph to k groups: depend only on the structure (edge attribute ...
0
votes
1answer
841 views

Chat Bot Answering based on Data Corpus Self-Training

I have created a very simple chat bot based on RASA NLU. In this case, I manually create some sample input text and create a model for using it against unknown source of input. It's fine for now. As ...
1
vote
1answer
919 views

How can we define missing rating in recommender system?

I was reading about collaborative filtering where we need to pass (user, item and rating) in case of matrix factorisation (SVD). Now, my question is given data of ...
1
vote
1answer
107 views

How to find unknown number of clusters in circular data?

I have some 1 dimensional data. Each record in the data is a specific time of the day. In order to cluster it I projected the data onto a circle of radius 1 unit. Now I need to find clusters in this ...
1
vote
0answers
23 views

What is the difference between K-Means & Self Organized Maps?

It seems they both perform clustering. They both reduce the dimensionality of the input data and classify further inputs based upon their distance/similarity to the center points. These points then ...
3
votes
2answers
2k views

What approach other than Tf-Idf could I use for text-clustering using K-Means?

I am working on a text-clustering problem. My goal is to create clusters with similar context, similar talk. I have around 40 million posts from social media. To start with I have written clustering ...
1
vote
0answers
305 views

How to approach Peak picking with a wide range of peak shapes, sizes, varying noise level, and occasionally shifting baseline?

I am trying write a program that continuously tracks the location a peak. To do that I need a very good peak detection algorithm. It not only has to tell the location of the peak but also the absence ...
0
votes
1answer
47 views

What algorithm can be used to divide a time series

Hi everyone I have a data as on the image. I would like to derive a line that separates the data (like the red line), but not sure what algorithm can be used. I thought about k-means in 1-d data, ...
3
votes
0answers
55 views

Determine the most important documents for supervised learning

I have somewhat of a general/high level question. Assume I'm doing supervised machine learning on some text data (tweets for example) and categorizing the documents to a certain taxonomy (multi-class ...
2
votes
2answers
57 views

Which machine learning algorithm to choose?

I want to choose an unsupervised algorithm which learns to predict $n$ outputs from the data, for eg. 4 coordinates (pixels) in an image. What algorithm should I choose? I think it's a 2-class ...
2
votes
1answer
57 views

Learning Football Player Stats like FIFA's by only the game result [closed]

It is a general question on how to learning representation of one entity but the dataset is mixed with a lot of other entities, which their statis are always waiting to be learnt. The question is ...
1
vote
1answer
1k views

Clustering a labeled data set

I have a large labeled dataset with 29 classes. Is is possible to use a clustering algorithm (like k-means) in this dataset, or it's not possible since clustering algorithms are unsupervised ?
1
vote
1answer
104 views

Annotation tool for classification experiments

What tools are available that provide an interface to present text classification results? I need to keep and compare the results of different classification algorithms for unsupervised data. The tool ...
1
vote
1answer
110 views

Training with data of different shapes. Is padding an alternative?

I have a dataset of about 1k samples and I want to apply some unsuspervised techniques in order to clustering and visualization of these data. The data can be interpreted as a table of a spreadsheet ...
2
votes
0answers
164 views

Exploratory analysis and feature engineering for time till failure prediction using sensor data of engines

I am trying to do some data exploration and analysis on a dataset of engine sensor readings. I would like to determine if the data I have is good enough to predict a time till failure and possibly ...
0
votes
1answer
578 views

How to recreate T-SNE dimensions deterministically?

So I have a set of 3000 features from which I would like to generate clusters. I passed my features through the T-SNE algorithm to reduce dimensionality to 2 features, and clusters are really visible ...
3
votes
1answer
2k views

Good books on unsupervised learning [closed]

I am looking for a good book about unsupervised learning that goes beyond the typical k-means and hierarchical clustering algorithms. Practical implementations in R or Python will be a plus. ...
3
votes
2answers
203 views

which NN should I use for Time-series dataset, whose pattern change as time goes

I am analyzing a time-series dataset using (supervised) tensorflow deep learning. The tensorflow code is given a series of inputs, and based on each input, the NN has to predict output value in near ...
0
votes
1answer
47 views

Bandwidth selection Kernel Density Estimation

I want to do KDE on data that are not necessarily normal using Gaussian kernels. In KDE in wikipedia an expression for the bandwidth is given when the underlying distribution of the data is gaussian. ...
1
vote
2answers
373 views

Confused about the different aspects in Machine Learning [closed]

After reading different articles about ML and algorithms, scientist tends to use different words when describing the different aspects in ML. So now I'm a bit confused myself and I hope you can ...
0
votes
1answer
2k views

Understanding Contrastive Divergence

I’m trying to understand, and eventually build a Restricted Boltzmann Machine. I understand that the update rule - that is the algorithm used to change the weights - is something called “contrastive ...
2
votes
2answers
98 views

Which is the best Machine learning technique for this Load forecasting problem?

I am trying to use Machine Learning to predict the load of a residence at any point in time for a whole year. I have past data pertaining to that house. So I have the training data and I need the ...
1
vote
0answers
71 views

Categorical data with order and blanks, is frequent dataset or k-modes a better option?

I have a dataset that's purely categorical: for each item it's ranked across a set of attributes, whether it's easy, moderate or difficult. But there are blanks if the item doesn't have the ...
0
votes
1answer
63 views

Can You Purposely Bias A Clustering Model?

We have a large amount (Billions) of high cardinality, mixed nominal & numerical data, and are performing some clustering on it as an experiment. There is a small subset of these data, however, ...
1
vote
1answer
565 views

Estimating Predictive Uncertainty for unlabeled data

I am trying to estimate the predictive uncertainty for a deep neural network. While I do have a labeled training set, I´m trying to measure uncertainty for some unlabeled production data. This paper ...
2
votes
3answers
6k views

is it possible to do feature selection for unsupervised machine learning problems?

I started looking for ways to do feature selection in machine learning. By having a quick look at this post , I made the assumption that feature selection is only manageable for supervised learning ...
2
votes
4answers
70 views

Unsupervised clustering without of Data which is supposed to be on a linear function

When I have a dataset where each datum has x and y, and the (x,y) has a relation of one of <...
1
vote
0answers
44 views

Conceptual clustering with sklearn?

How can I perform conceptual clustering in sklearn? My use case is that I have English Wikipedia articles that I'm doing unsupervised learning on (tfidf -> truncated svd -> l2 normalize), and I'd like ...
2
votes
1answer
621 views

Is it possible to make a label automatically in supervised learning(Machine Learning)?

My background knowledge: Basically, supervised learning is based on labeled data. Using the labeled data, the machine can study and determine results for unlabeled data. To do that, for example, if we ...
4
votes
2answers
5k views

Why will the accuracy of a highly unbalanced dataset reduce after oversampling?

I have created a synthetic dataset, with 20 samples in one class and 100 in the other, thus creating an imbalanced dataset. Now the accuracy of classification of the data before balancing is 80% while ...
1
vote
1answer
47 views

Help me understand how word-as-vector representations are constructed

Let's suppose I have a big list of words. I want to turn this list into a vector space of dimension $N$ such that each word is a vector in this vector space. But I have no idea how to go about with ...
4
votes
2answers
75 views

When using an unsupervised alogirthm, what is the “learning” part since it belong to machine learning field?

I had a brief experience with machine learning by using a clustering algorithm, i also read the basic ideas and calculations of a simple classification algorithm. Now, i would read more about "machine ...
1
vote
1answer
261 views

What is the advantage of using Dunn index over other metrics for evaluating clustering algorithm? [closed]

There are many metrics to evaluate clustering algorithm like Calinski-Harabaz Index, Dunn index, Rand index, etc. Are there any advantage of using Dunn index over other metrics for evaluating ...