Questions tagged [classification]

An instance of supervised learning that identifies the category or categories which a new instance of dataset belongs.

Filter by
Sorted by
Tagged with
1
vote
0answers
17 views

Multilabel Classification; which network design?

I have a hard time thinking about how I can build this network with the following problem: I want to build a CNN to classify notes from sheet music. I have tried several models with and without ...
1
vote
1answer
13 views

When would not normalizing input values have higher accuracy?

Right now I'm training a deep neural network for a binary classification problem, with a feature set of winrates. As such, each winrate is bigger or equal to 0 but smaller than 100. I've been getting ...
0
votes
2answers
19 views

Class Imbalance and Cost-Sensitive Learning XGBoost

I'm fairly new to data science and machine learning and have been trying to read a bit more on methods like boosting for one of the projects I am working on. The investigator on this project is ...
0
votes
1answer
7 views

Which ML method for multiclass (non-binary) text classification should I choose (from SparkML)?

I am working on a quite big dataset that will be processed on the cluster, so this is why I am using PySpark for that purpose. The presentable records of this dataset have a such structure: ...
0
votes
0answers
9 views

Stratified sampling for large imbalanced data set

I have a very large data set (64 Million Rows) and I want to understand the best approach to sample the data for explorative data analysis before proceeding to perform any classification modeling. ...
0
votes
2answers
16 views

How to find whether a dataset is blanced or imbalanced?

I have few dataset to experiment classification(Multi-class). These datasets are about 400GB. I wanted to know whether the dataset is balanced or imbalanced. How to know that dataset is balance or ...
0
votes
0answers
16 views

Bayes Classifier and Unavoidable Error

Consider the following two-class classi cation problem, where $Y = 0 \text{ or } 1$. Suppose $Y$ is a random variable satisfying$ P(Y = 0) = \frac{1}{3}$ and $P(Y = 1) = \frac{2}{3}$: Moreover, we ...
0
votes
0answers
5 views

Threshold for overfitted models

It's common knowledge in DS that overfitted models perform well on training data and poorly on test data. But how do you decide if a model is really overfitting? I have nowhere (books, online courses, ...
0
votes
1answer
12 views

Classifier design for website screenshots

I'm working on a project that requires determining if the page representing a hosted file on a third party platform (such as rapidgator or nitroflare) is still up or not. For example, here is a file ...
1
vote
1answer
26 views

How to choose solution - Neural Neworks or Scikit-Learn/Numpy/Pandas?

I am trying to solve a problem - categorising and routing service desk emails to concerned teams for resolution. Created and tested a model using Scikit-Learn, Numpy and Pandas. - Tokenized the email ...
0
votes
0answers
26 views

Predict Data Linking

I've two different datasets A(x1, x2, x3...xm) and B(y1, y2, y3...yn). Each unique row in A is linked with a unique row in B i.e. only one unique pair exists. If I get a new row in A and a new row in ...
0
votes
1answer
18 views

How to Split And Resample Imbalanced Dataset Into Train, Validation and Test

I want to understand how to split the imbalanced data set with a binary target variable where 87% of the samples are negative and 13% of the samples are positive. Now, I know that you should always ...
0
votes
0answers
5 views

Splitting a pdf containing batch of scanned documents

My question is primarily: is there any ML research paper about splitting a pdf containing a batch of scanned documents (eg bank statements) into individual documents? I have searched for this but I ...
1
vote
2answers
27 views

class_weight on sklearn's DecisionTreeClassifier

Can class_weight='balanced' on scikit-learn's DecisionTreeClassifier be interpreted as having identical duplicate data points for the minority classes? I know that doesn't work that way, class_weight ...
0
votes
1answer
25 views

General approach on time series for customer retention/churn in retail

I have a time series of data in the following form: ...
0
votes
0answers
6 views

Combining decision trees and neural networks for classifying text with metadata . How to combine and train?

I have a multi-label classification problem where the input consist of free text, with metadata such as categories (from a fixed, limited set) associated with each text. The output consist of a set of ...
0
votes
1answer
28 views

Error encoding categorical features using sklearn pipelines

I am new to sklearn pipelines and am using this post as a guide for my code: https://www.codementor.io/bruce3557/beautiful-machine-learning-pipeline-with-scikit-learn-uiqapbxuj I am trying to encode ...
0
votes
1answer
17 views

Similar objects same labels

Example: red paprica and green paprica. The output I need is just paprica, should i label them the same or give them two labels "green paprica" and "red paprica" and leater on just treat them as one? ...
1
vote
1answer
83 views

Impose similar metric on segments to model

I am training a binary classifier in a dataset using AUC as a score. The dataset has two main groups (we will refer to them as good and bad population). A property that this dataset has is having a ...
3
votes
2answers
113 views

XGBoost validation for number of trees

I have a simple Question: I am using XGBoost to classify some data: 1.) With 100 estimators I have following scores(roc_score): train_data : 98.5 validation_data : 97.2 2.) With 500 ...
1
vote
1answer
23 views

How to control the amount of positives in classification?

I have a basic, yet quite complex problem to solve right now. Let's say we have a training set of 20,000 samples in my training set, out of which 3 to 4% is flagged as "True", the rest is flagged as "...
0
votes
1answer
19 views

How Does Cross-Entropy Work With Softmax Activation Function?

I found online that the derivative of a cross-entropy activation function with a softmax activation is (output - expected), which had me very confused. If for example, the expected value is 1, and ...
2
votes
0answers
35 views

Why decision tree algorithm performs better in small dataset in compare to other classification algorithm?

I read in some paper that decision tree algorithms have better result in small dataset like(400-600 records) when compared to other classification algorithms like SVM, Naive Bayes and KNN. I want to ...
1
vote
2answers
104 views

How to find “regions” with high purity

I am applying a ML model (LGBM binary classifier) to data and would now like to identify the part of data where I have a low ratio of false-negatives (false-postives are not such a problem) and as ...
0
votes
1answer
34 views

Non-Convex Constraints for Classification Problems

I am willing to create a hypothetical non-convex constraints for the purpose of practising nonlinear classification using an algorithm. I thought of such constraints in the form: $x^TAx + Bx \leq c$. ...
0
votes
0answers
14 views

How to build a 2-d bayes classifier

I'm working on a 2-d bayes classifier and I'm a little confused on how to start exactly. My attribute space is 2-d. There are 3 classes. The data is assumed to be normally distributed. p(x | y1): P(...
1
vote
1answer
30 views

Predict correct answer among ten answers for a given question

I have a case study to solve where I am given a dataset of questions and its answers, there are ten answers for a particular question. It's a classification problem where correct answer is having <...
1
vote
1answer
7 views

Growth function of a 6-dimensional linear classifier

In our course, we are dealing with a d-dimensional classification problem ($\chi = \mathbb{R}^{d}$ as our input space, and $y = \{-1,+1\}$). Our hypothesis class $H$ consists of all hypotheses of the ...
0
votes
2answers
45 views

Feature extraction; similarity and classification of accelerometer data

I have several expert persons performing the same specific action (for example, squat or leap forward) multiple times. Say 5 persons do 100 squats each. They have an accelerometer attached to the same ...
1
vote
1answer
22 views

Can a decision in a node of a decision tree be based on comparison between 2 columns of the dataset?

Assume the features in the dataframe are columns - A,B,C and my target is Y Can my decision tree have a decision node which looks for say, ...
0
votes
2answers
25 views

Classify if someone is home based on time

I have a dataset with locations and a timestamp of a subject. For each location and timestamp I determined by comparing the location to the home address if the subject was at home or not (0/1) and ...
0
votes
1answer
19 views

How to choose the features for an algorithm from the given attached screenshot?

How to choose the features from the given attached heat map & correlation factor for the classification algorithm? I have 6 different features i.e., ac233fc01403, ac233fc02eaa, ac233fc015f6, ...
0
votes
1answer
29 views

How to approach data prediction problem

I'm new to ML and data science. I would really like high level advice how to approach the following problem. I need to predict if an engine will fail, what I've is a sensor that give a certain value ...
0
votes
1answer
25 views

Classifying Letters using CNN - Help

so some context, I'm trying to develop an OCR (for fun) and for that reason I decided to first find text within a page, parse it in to letters within the text and from there try and classify the ...
0
votes
1answer
27 views

SMOTE vs SMOTENC for binary classifier with categorical and numeric data

I have a problem that I am having trouble thoroughly understanding. I am using Xgboost for classification. My y is 0 or 1 (true or false). I have categorical and numeric features, so theoretically, I ...
1
vote
1answer
69 views

Machine learning method to predict event date

Let's say I have a big dataset consisting of variables including but not limited to the start/end date of loans, their notional amount, a loan prepayment indicator etc. My goal is to create a model ...
1
vote
1answer
34 views

how to use word embedding to do document classification etc?

I just start learning NLP technology, such as GPT, Bert, XLnet, word2vec, Glove etc. I try my best to read papers and check source code. But I still cannot understand very well. When we use word2vec ...
1
vote
1answer
15 views

Interpreting fraction of zero weights in TensorFlow

I am using the TensorFlow to do a simple linear classification using logistic regression. The graph included from the TensorBoard displays what they call the fraction of zero weights. How do I ...
0
votes
0answers
5 views

Estimating “child” level probability based on historical proportion?

I have a classification model that predicts whether a product (eg chair) will be a champion. A champion product is defined as products exceeding a certain monthly sales threshold. However, if i want ...
0
votes
1answer
16 views

Explanation behind the calculation of training loss in deep learning model

I am trying to model an image classification problem using convolution neural network. I came across a code on Github in which I am not able to understand the meaning of following line for loss ...
0
votes
1answer
12 views

How to Proceed with Tokenized text content which is given in number?

I have one data set of customer review, but the text data is given is tokenized text number. I am unable to proceed thinking about how to proceed? As I am encountering such data set the first time, ...
0
votes
0answers
9 views

Feature Analysis to Maximize Classification

I keep coming across a pattern in problems I need to solve.  Perhaps one of you might have a suggestion on the best method to solve this one: Assume 3 features, X1, X2, X3.   X1, X2 are real valued ...
0
votes
0answers
21 views

Bayesian test for classification problems?

I am using two classification algorithms in Weka i.e. Logistic Regression and Naive Bayes and want to know which algorithm has better performance? I need a statistical test like Bayesian, so which ...
0
votes
0answers
13 views

Finding the closest neighbour of multidimensial data point

I have a test point with 15 attributes. I want to find the closest data point to this test point from 10,000 data points. I'm thinking using something like this: ...
2
votes
1answer
22 views

Confusion for considering accuracy or standard deviation in selecting the best parameters

I have a model with a various parameters to test. The size of the dataset I have is not really large (~500 documents). My issue is that when I test the parameters using 10 CV, some of them produce ...
0
votes
3answers
45 views

Classification vs Regression Model what should I choose?

I am working on a problem like 'customers next month revenue prediction'. Here revenue will be the target variable. Again we actually segment the customers based on there revenue(like if they give ...
1
vote
0answers
8 views

Evolution of classification methods

In the book "Deep Learning with Python" by Francois Chollet (2018), in section 1.2.4 one can find: Decisions trees learned from data began to receive significant research interest in the 2000s, and ...
0
votes
2answers
47 views

Using LSTM for binary text Classification, getting almost same accuracy at each epoch

I am doing Twitter sentiment classification. For that I am using LSTM with pretrained 50d GloVe word embeddings(not training them as of now, might do in future). The tweets are of variable lengths ...
1
vote
1answer
27 views

Checking if ML model is possible

How can I check if a machine learning model is feasible on a given dataset? What techniques like EDA, correlation etc. can be used to judge if a model is possible i.e. data and predictor variables ...
1
vote
3answers
34 views

The effect of imbalanced distribution of data

I read on Google's ML website if I have classification dataset with a ratio of 90% for one classification and 10% of the data for another classification. In that case, should I use the exact same ...