Questions tagged [classification]

An instance of supervised learning that identifies the category or categories which a new instance of dataset belongs.

Filter by
Sorted by
Tagged with
5
votes
1answer
610 views

How to detect overfitting of a stock screener

The project I am working on allows users to create Stock Screeners based on both technical and fundamental criteria. Stock Screeners are then "backtested" by simulating the results of applying in ...
1
vote
1answer
106 views

Splitting binary classification into smaller susbsets

As an example. If you are tying to classify humans from dogs. Is it possible to approach this problem by classifying different kinds of animals (birds, fish, reptiles, mammals, ...) or even smaller ...
1
vote
1answer
236 views

depth/complexity of decision trees

I have used the same methods/parameters to create two decision trees. The trees classify the presence or absence of a medical condition using the presence or absence of various symptoms. There is a ...
3
votes
1answer
98 views

What are the possible ways to handle class unbalance in a large scale image recognition problem with Deep Neural Nets?

I have 22 classes of objects but they have very skewed distributions where max class has 100.000 images and the min class has 1600 images. In that setting I would like to hear some possible solutions ...
1
vote
1answer
1k views

Kaggle Titanic Survival Table an example of Naive Bayes?

is the survival table classification method on the Kaggle Titanic dataset an example of an implementation of Naive Bayes ? I am asking because I am reading up on Naive Bayes and the basic idea is as ...
2
votes
1answer
170 views

Reasons and prevention of trivial (and less trivial) misclassification errors?

I was not sure about posting this question with mentioning the name of the company, which I quite respect and admire. However, I've figured that a wider exposure might help the team to fix this and ...
3
votes
4answers
784 views

Word Frequency Analysis of Document Sets

I'm doing some work trying to extract commonly occurring words from a set of human classified documents and had a couple questions for anyone who might know something about NLP or statistical analysis ...
0
votes
2answers
545 views

minimization with a negative cost function: works in MATLAB, not in Python

I'm trying to use a particular cost function (based on doubling rate of wealth) for a classification problem, and the solution works well in MATLAB. See https://github.com/acmyers/compareCostFXs When ...
2
votes
2answers
7k views

What is a discrimination threshold of binary classifier?

With respect to ROC can anyone please tell me what the phrase "discrimination threshold of binary classifier system" means? I know what a binary classifier is.
7
votes
1answer
1k views

How to extract features and classify alert emails coming from monitoring tools into proper category?

My company provides managed services to a lot of its clients. Our customers typically uses following monitoring tools to monitor their servers/webapps: OpsView Nagios Pingdom Custom shell scripts ...
3
votes
4answers
1k views

How to learn a classifier from a dataset with high imbalance

What are the most useful techniques for learning a binary classifier from a dataset with a high degree of imbalance (i.e., a dataset with the "target" class being much rarer than the "background" ...
13
votes
1answer
14k views

What is the difference between feature generation and feature extraction?

Can anybody tell me what is the purpose of feature generation? and why feature space enrichment is needed before classifying an image? Is it a necessary step? Is there any method to enrich feature ...
2
votes
1answer
125 views

Object Recognition for classification, is it being used in industry?

I'm wondering if e-commerce companies where products are offered by users, such as EBay, are using Object Recognition to ensure that an uploaded image corresponds to an specific type of object (...
2
votes
1answer
50 views

Does a NB wrapper consider feature subset size?

while comparing two different algorithms to feature selection I stumbled upon the follwing question: For a given dataset with a discrete class variable we want to train a naive bayes classifier. We ...
2
votes
2answers
3k views

Improving Naive Bayes accuracy for text classification

I am performing document (text) classification on the category of websites, and use the website content (tokenized, stemmed and lowercased). My problem is that I have an over-represented category ...
1
vote
1answer
250 views

What software is being used in this image recognition system?

I was wondering if anyone knew which piece of software is being used in this video? It is an image recognition system that makes the training process very simple. http://www.ted.com/talks/...
7
votes
1answer
12k views

How does the naive Bayes classifier handle missing data in training?

Naive Bayes apparently handles missing data differently, depending on whether they exist in training or testing/classification instances. When classifying instances, the attribute with the missing ...
4
votes
1answer
517 views

K nearest neighbour

Is the k-nearest neighbour algorithm a discriminative or a generative classifier? My first thought on this was that it was generative, because it actually used Bayes's theorem to compute the posterior....
4
votes
3answers
3k views

What cost function and penalty are suitable for imbalanced datasets?

For an imbalanced data set, is it better to choose an L1 or L2 regularization? Is there a cost function more suitable for imbalanced datasets to improve the model score (...
8
votes
1answer
1k views

Document classification: tf-idf prior to or after feature filtering?

I have a document classification project where I am getting site content and then assigning one of numerous labels to the website according to content. I found out that tf-idf could be very useful ...
1
vote
3answers
2k views

kNN - what happens if more than K observation have the same distance to the centroid of the cluster

EDIT It was pointed out in the Answers-section that I am confusing k-means and kNN. Indeed I was thinking about kNN but wrote k-means since I'm still new to this topic and confuse the terms quite ...
8
votes
2answers
2k views

Using NLP to automate the categorization of user description

I have a huge file of customer complaints about the products my company owns and I would like to do a data analysis on those descriptions and tag a category to each of them. For example: I need to ...
4
votes
2answers
102 views

Method for solving problem with variable number of predictors

I've been toying with this idea for a while. I think there is probably some method in the text mining literature, but I haven't come across anything just right... What is/are some methods for ...
2
votes
0answers
76 views

Decision trees, categorizacion and oversampling

I want to create a model to predict the propensity to buy a certain product. As my proportion of 1's is very low, I decided to apply oversampling (to get a 10% of 1's and a 90% of 0's). Now, I want to ...
1
vote
2answers
263 views

Graphlab vs Mahout [closed]

I have some question regarding to the choice of the better implementation. I would know the differences and advantages of Mahout Apache (Java implementation) versus Graphlab (Python implementation) in ...
10
votes
3answers
1k views

What regression to use to calculate the result of election in a multiparty system?

I want to make a prediction for the result of the parliamentary elections. My output will be the % each party receives. There is more than 2 parties so logistic regression is not a viable option. I ...
4
votes
1answer
406 views

Using the Datumbox Machine Learning Framework for website classification - guidelines?

A short while ago, I came across this ML framework that has implemented several different algorithms ready for use. The site also provides a handy API that you can access with an API key. I have need ...
30
votes
5answers
21k views

Deep Learning vs gradient boosting: When to use what?

I have a big data problem with a large dataset (take for example 50 million rows and 200 columns). The dataset consists of about 100 numerical columns and 100 categorical columns and a response column ...
3
votes
1answer
375 views

What is the best practice to classify category of named entity in sentence

I have 1-4 gram text data from wikipedia for 14 categories, which I am using for NE classification. I feed named entity from sentence to lucene indexer which searches named entity from these 14 ...
3
votes
3answers
3k views

Visualizing Support Vector Machines (SVM) with Multiple Explanatory Variables

I was wondering if anyone was aware of any methods for visualizing an SVM model where there are more than three continuous explanatory variables. In my particular situation, my response variable is ...
2
votes
3answers
788 views

Choosing the right data mining method to find the effect of each parameter over the target

I am dealing with a lot of categorical data right now and I would like to use an appropriate data mining method in any tool [preferably R] to find the effect of each parameter [categorical parameters] ...
2
votes
1answer
202 views

Modeling when the response variable has too many 0's and few continuous values?

For problems where the data represents online fraud or insurance (where each row represents a transaction), it is typical for the response variable to denote the value of fraud committed in dollars. ...
2
votes
0answers
105 views

FUZZY ARTMAP for continuous data

I was going through an IEEE Research paper which has used Fuzzy ARTMAP for predicting the price of electricity given some highly correlated data. As per my basic understanding about Fuzzy ARTMAP it ...
6
votes
2answers
247 views

The meaning of multi-class classification rules

The meaning of multi-class classification rules ...
2
votes
1answer
90 views

Ranked tag recommendation for university courses

Our system allows an admin to manage a database of university courses. These courses have multiple fields, like the department, a title, and a description. I am adding the ability to add learning ...
4
votes
1answer
175 views

Rough vs Fuzzy vs Granular Computing

For my Computational Intelligence class, I'm working on classifying short text. One of the papers that I've found makes a lot of use of granular computing, but I'm struggling to find a decent ...
1
vote
2answers
2k views

How to calculate classification accuracy with confusion matrix?

I have Train and Test data, how to calculate classification accuracy with confusion matrix ? Thanks ...
4
votes
2answers
237 views

Distributed Scalable Decision Trees

Are there any good sources that explain how decision trees can be implemented in a scalable way on a distributed computing system. Where in a given source is this explained?
6
votes
1answer
4k views

Bayes Optimal Decision Boundaries for Gaussian Data with Equal Covariance

I am drawing samples from two classes in the two-dimensional Cartesian space, each of which has the same covariance matrix $[2, 0; 0, 2]$. One class has a mean of $[1.5, 1]$ and the other has a mean ...
5
votes
2answers
1k views

Machine Learning - Where is the difference between one-class, binary-class and multinominal-class classification?

Where is the difference between one-class, binary-class and multinominal-class classification? If I like to classify text in lets say four classes and also want the system to be able to tell me that ...
1
vote
0answers
47 views

Machine Learning - Where is the difference between one-class, binary-class and multinominal-class classification? [duplicate]

Where is the difference between one-class, binary-class and multinominal-class classification? If I like to classify text in lets say four classes and also want the system to be able to tell me that ...
3
votes
1answer
4k views

R Script to generate random dataset in 2d space

I want to analyze the effectiveness and efficiency of kernel methods for which I would require 3 different data-set in 2 dimensional space for each of the following cases: BAD_kmeans: The data set ...
2
votes
1answer
169 views

what is buyer classification problem?

Recently in a data analytic job interview for an e-commerce site, they asked me, do i have some knowledge of buyer classification problem. Unfortunately i heard this term for the first time. After ...
8
votes
3answers
2k views

How can I classify text considering word order, instead of just using a bag-of-words approach?

I've made a Naive Bayes classifier that uses the bag-of-words technique to classify spam posts on a message board. It works, but I think I could get much better results if my models considered the ...
2
votes
1answer
130 views

How to classify test objects with this ruleset in order of priority?

I'm coding a program that tests several classifiers over a database weather.arff, I found rules below, I want classify test objects. I do not understand how the classification, it is described: "In ...
9
votes
2answers
264 views

How much time do scikit classifiers take to classify?

I am planning to use scikit linear support vector machine (SVM) classifier for text classification on a corpus consisting of 1 million labeled documents. What I am planning to do is, when a user ...
6
votes
3answers
296 views

Distance calculation/vector range significance

I'm trying to implement item based collaborative filtering. Do any distance calculations allow for weighting of certain ranges of values within each vector? For example, I would like to be able to ...
3
votes
3answers
10k views

How to classify and cluster this time series data

I have post already the question few months ago about my project that I'm starting to work on. This post can be see here: Human activity recognition using smartphone data set problem Now, I know ...
6
votes
1answer
2k views

Choosing a window size for DTW

I have time series data from mobile sensors for different motions such as walking, pushups, dumbellifts, rowing and so on. All these motions have different length of time series. For classifying them ...
8
votes
1answer
2k views

Difference between tf-idf and tf with Random Forests

I am working on a text classification problem using Random Forest as classifiers, and a bag-of-words approach. I am using the basic implementation of Random Forests (the one present in scikit), that ...