Questions tagged [unsupervised-learning]

Finding hidden (statistical) structure in unlabelled data, including clustering and feature extraction for dimensionality reduction.

Filter by
Sorted by
Tagged with
0
votes
1answer
36 views

Gibbs sampling (For inference) vs EM

I'm familiar with the Expectation-Maximixation algorithm and, until now, I thought it was the only way to maximize the likelihood of the observed data, assuming a Gaussian mixture model. In the last ...
1
vote
0answers
22 views

I need help interpreting this PCA plot

I have a dataset of 116 observations and 10 numeric variables. The dataset contains information about healthy patients and patients attained with breast cancer. I did a PCA plot showing the cluster of ...
1
vote
0answers
109 views

PCA for unsupervised feature selection [closed]

If I understood correctly, "using results of PCA to select features" (as recommended in this answer) implies visually analysing bi-plots of first two principal components - i.e. the angle between a ...
0
votes
1answer
39 views

what is a good performance measure for comparing different neural network architectures in unsupervised clustering task?

What is a good measure to use when trying to decide between picking unsupervised clustering NN architectures? There seems to some ideas here, but i am trying to find out feedback/suggestions from ...
0
votes
1answer
114 views

Kmeans cluster validation when I have labeled test data

I'm trying to implement the unsupervised k-means algorithm for sentiment analysis of imdb movie dataset created by stanford. The steps that I followed is : 1) Load the comments 2) Apply tokenization ...
1
vote
3answers
484 views

How to create clusters based on sentence similarity?

I have data which looks like following. Data is a group of sentences which are similar, but have few unique words in between like TABLEA, TABLEB etc. ...
3
votes
3answers
151 views

ML algorithm for Music Features

I am a newbie in machine learning topic and I need to create model from music data. It contains features of the songs but it is not labeled. How can I create a model from that ? Do I need to use ...
0
votes
1answer
124 views

Clustering Customers on transactional behavior

Objective: Segment the accounts on their transactional behavior and find the accounts which are more likely to subscribe for loans. Dataset: 1) Account_Number 2-91) Transaction amount ...
2
votes
0answers
249 views

Training detector without bounding box data

From what I can see most object detection NNs (Fast(er) R-CNN, YOLO etc) are trained on data including bounding boxes indicating where in the picture the objects are localized. Is there any model ...
0
votes
1answer
247 views

How can I detect anomalies/outliers in my online streaming data on a real-time basis?

Say, I've a huge set of data(infinite in size) consisting of alternating sine wave and step pulses one after the other. What I want from my model is to parse the data sequence wise or point wise and ...
0
votes
1answer
22 views

Working with few instances of specific target feature over large dataset

I have data over a single, a machine includes different components, all the parts are interacting, the data are tracked for those parts, it tracks power consumption and many other relevant feature ...
1
vote
0answers
24 views

TextRank algorithm for Web content

I am looking for an algorithm that would be able to extract meaningful keyphrases from web articles. Each article has more than 2000 words and information is structured using paragraphs, h1, h2 tags ...
4
votes
2answers
447 views

Is SVD non-linear while PCA (by eigendecompostion) is linear?

I am quite confused because a colleague of mine recently told me that he preferred using SVD instead of PCA (by eigendecomposition) because, contrary to the latter, the former is non-linear so it can ...
2
votes
1answer
68 views

Is PCA (by eigendecomposition) or SVD better in decorrelating the predictors of a machine learning model?

Is there any reason to think that SVD is better than PCA (by eigendecomposition) in decorrelating the predictors of a machine learning model?
0
votes
1answer
105 views

how to build a predictive model without training data neither historical data

I m trying to score "how much a product is expected in the market". I created some features: How much this product is used each year. Where was it used . how many product for each country. the main ...
0
votes
1answer
46 views

Is there an upper bound for k in nearest neighbors-based methods?

When applying a nearest neighbors-based method to a data of, for instance, 2000 points, what is the largest number of neighbors that can be considered ? I am using a nearest neighbors method in an ...
1
vote
0answers
79 views

Time-series clustering Quality Measures

I am clustering time-series datasets which are not labeled (No Ground truth) and I want to measure the quality of the clusters. Could you please suggest any Clustering performance evaluation methods ...
1
vote
1answer
45 views

Is train/test-Split in unsupervised learning of neural network necessary?

I am using autoencoder for anomaly detection in warranty data. It is unsupervised. I calculate the reconstruction error by the model and the records with high reconstruction error value is considered ...
-1
votes
1answer
133 views

How to use K-Means to detect users anomaly in Access Control

I'm currently working on access control project, Smart Lock to be more spesific. Like the other smart lock system, the system required user's authentication to open the door. I'm using RFID as ...
1
vote
0answers
22 views

Decision tree to get difference in rates in two groups?

I have two sample groups of customers, each customer has 100s of features. For a single sample, i would use Decision Trees to find sub-groups that have a high churn rate. Thats easy. However, my ...
1
vote
1answer
166 views

More weightage to a categorical feature for an Autoencoder model

I am using autoencoder for anomaly detection. I don't have any labels already and so its unsupervised. If I have categorical variables, I usually one hot encode before giving it to the model. I would ...
3
votes
1answer
455 views

ML Models: How to handle categorical feature with over 1000 unique values

I am trying to build an ML Classification model on a data set that contains quite a few categorical columns. However, few of them have over 1000 unique values. I am concerned that if I run one-hot ...
0
votes
1answer
47 views

How to split temporal sequences to sub-sequences in a meaningful yet unsupervised manner?

I have a biological process that undergoes some cellular event which I am observing. I have a series of events, with different temporal gaps between them. For example ...
3
votes
1answer
160 views

How to use a different model to deep neural network with reinforcement learning based on DQN?

Is it possible to implement a reinforcement learning algorithm without using a deep neural network (DNN) as used in deep reinforcement learning e.g. Deep Q-Network (DQN)? How can I replace the DNN in ...
2
votes
1answer
65 views

Given data that is labeled as outliers, how can I classify data as outliers?

I have a dataset that is a mixture of sparse binary features and quantitative features. I only have definite outliers labeled. How should I approach trying to classify unlabeled data? I considered ...
5
votes
3answers
16k views

Anomaly detection on time series

I've just started working on an anomaly detection development in Python. My data sets are a collection of timeseries. More in details, data are coming from some sensors/meters which record and ...
1
vote
3answers
682 views

Cross validation for anomaly detection using autoencoder

I am using autoencoder for anomaly detection in warranty data. I don't have any ground truth labels to confirm whether the anomalies detected by the model is really an anomaly or not. Since I don't ...
0
votes
1answer
711 views

using unsupervised learning algorithms on images

I am working on a project to classify images of types of cloth (shirt, tshirt, pant etc). While this is a standard supervised classification problem, the accuracy of the neural network is not good. ...
5
votes
1answer
751 views

What does it mean by “t-SNE retains the structure of the data”?

I was learning about t-SNE when I was told that t-SNE retains the structure of the data in the embeddings. What exactly does this mean ? How does the algorithm achieve this ? So far I have ...
-1
votes
2answers
59 views

How to identify clusters after multiple runs?

Suppose I run an unsupervised clustering algorithm. After multiple runs, I find clusters and would like to know if the same cluster was found more than once. For example: I can figure out A-orange, ...
1
vote
0answers
68 views

Semi-supervised Learning doubt

I'm reading "Hands on machine learning" by Aurelien Geron. He stated that semi-supervised learning is: Some photo-hosting services, such as Google Photos, are good examples of this. Once you ...
2
votes
3answers
773 views

What machine learning algorithms to use for unsupervised POS tagging?

I am interested in an unsupervised approach to training a POS-tagger. Labeling is very difficult and I would like to test a tagger for my specific domain (chats) where users typically write in lower ...
1
vote
0answers
55 views

Classifying variable types on a list of variables

I have a list of around 700 variables which I need to perform a variable cleanup on. What complicates things is there are different numeric codes which flag an invalid value and these differ by the ...
1
vote
0answers
40 views

Random Training set for GAN's [closed]

I have studies the gans in depth and some of its type like cycle, pix2pix, cgans. Now I want to generate random images from random distribution from generator. So I am creating a dataset with no ...
-1
votes
1answer
37 views

What is the most straightforward way to discover clusters in data? [closed]

I'm planning on extracting a number of word vector distances from a data set, and I want to be able to detect clusters within that set, with an undefined number of clusters that are dynamically ...
1
vote
1answer
363 views

Graph & Network Mining: clustering/community detection/ classification

I am working on graphs/networks where nodes and edges have some attributes. I want to know what algorithm exist for: 1) clustering a graph to k groups: depend only on the structure (edge attribute ...
0
votes
1answer
879 views

Chat Bot Answering based on Data Corpus Self-Training

I have created a very simple chat bot based on RASA NLU. In this case, I manually create some sample input text and create a model for using it against unknown source of input. It's fine for now. As ...
1
vote
1answer
990 views

How can we define missing rating in recommender system?

I was reading about collaborative filtering where we need to pass (user, item and rating) in case of matrix factorisation (SVD). Now, my question is given data of ...
1
vote
1answer
109 views

How to find unknown number of clusters in circular data?

I have some 1 dimensional data. Each record in the data is a specific time of the day. In order to cluster it I projected the data onto a circle of radius 1 unit. Now I need to find clusters in this ...
1
vote
0answers
23 views

What is the difference between K-Means & Self Organized Maps?

It seems they both perform clustering. They both reduce the dimensionality of the input data and classify further inputs based upon their distance/similarity to the center points. These points then ...
3
votes
2answers
2k views

What approach other than Tf-Idf could I use for text-clustering using K-Means?

I am working on a text-clustering problem. My goal is to create clusters with similar context, similar talk. I have around 40 million posts from social media. To start with I have written clustering ...
1
vote
0answers
309 views

How to approach Peak picking with a wide range of peak shapes, sizes, varying noise level, and occasionally shifting baseline?

I am trying write a program that continuously tracks the location a peak. To do that I need a very good peak detection algorithm. It not only has to tell the location of the peak but also the absence ...
0
votes
1answer
48 views

What algorithm can be used to divide a time series

Hi everyone I have a data as on the image. I would like to derive a line that separates the data (like the red line), but not sure what algorithm can be used. I thought about k-means in 1-d data, ...
3
votes
0answers
55 views

Determine the most important documents for supervised learning

I have somewhat of a general/high level question. Assume I'm doing supervised machine learning on some text data (tweets for example) and categorizing the documents to a certain taxonomy (multi-class ...
2
votes
2answers
57 views

Which machine learning algorithm to choose?

I want to choose an unsupervised algorithm which learns to predict $n$ outputs from the data, for eg. 4 coordinates (pixels) in an image. What algorithm should I choose? I think it's a 2-class ...
2
votes
1answer
58 views

Learning Football Player Stats like FIFA's by only the game result [closed]

It is a general question on how to learning representation of one entity but the dataset is mixed with a lot of other entities, which their statis are always waiting to be learnt. The question is ...
1
vote
1answer
2k views

Clustering a labeled data set

I have a large labeled dataset with 29 classes. Is is possible to use a clustering algorithm (like k-means) in this dataset, or it's not possible since clustering algorithms are unsupervised ?
1
vote
1answer
112 views

Annotation tool for classification experiments

What tools are available that provide an interface to present text classification results? I need to keep and compare the results of different classification algorithms for unsupervised data. The tool ...
1
vote
1answer
113 views

Training with data of different shapes. Is padding an alternative?

I have a dataset of about 1k samples and I want to apply some unsuspervised techniques in order to clustering and visualization of these data. The data can be interpreted as a table of a spreadsheet ...
2
votes
0answers
166 views

Exploratory analysis and feature engineering for time till failure prediction using sensor data of engines

I am trying to do some data exploration and analysis on a dataset of engine sensor readings. I would like to determine if the data I have is good enough to predict a time till failure and possibly ...

1 2 3
4
5
7