Questions tagged [classification]

An instance of supervised learning that identifies the category or categories which a new instance of dataset belongs.

Filter by
Sorted by
Tagged with
0
votes
1answer
31 views

Deep model ensemble giving different results

I am making an ensemble of deep models for solving a classification problem. The initial weights follow the default distribution of keras layers. Each time I run ...
1
vote
1answer
26 views

Dealing with irrelevant features in dataset (Homework)

I have a specific question pertaining to one of my machine learning homeworks. Basically, we are required to build a model that takes a 5000*10000 dataset X (5000 examples each with 10000 features), ...
1
vote
1answer
15 views

Same probability for all classes

I implemented a fully connected MLP of shape [783 (input), 128 (hidden layer) and 10 (output)] the hidden layer had a sigmoid activation function and the output a sofmax. I tested with the dataset of ...
2
votes
2answers
30 views

model predicting probability close to 50 for positive classes in imbalanced training dataset

I have a binary classification model where I am predicting the positive class which is only 10% of whole training data set. The issue with this imbalanced data set is my model is predicting ...
0
votes
1answer
35 views

Clause type classification

We would like identify similar text (clauses) on a contract based on a trained corpus. For instance: Contract - small sample ...
0
votes
0answers
19 views

Neural Network for classifying input data to maximize accuracy

Recently I've been working on some deep neural networks for binary classification, and in developing the appropriate features, I found the necessity to classify the input data. For example, if the ...
1
vote
2answers
108 views

A generalized quadratic loss for deep neural network for multi-class classification

I'm evaluating the possibility to introduce a new loss for the subject described above. Let $l$ be the number of examples, $q$ the number of classes, $p_{i,r}$ the $r$ classifier output on example $i$...
0
votes
0answers
18 views

Mean encoding and covariate shift

I am working on a binary classification problem whose two main issues are categorical variables with many levels and, since the process, I am modeling is not yet stable, the proportion among the two ...
0
votes
0answers
29 views

Recommend ML algorithm

I have objects of two nominal classes that are described by an order list of numbers with the same list size for each object where the list contains both positive and negative numbers. They are real ...
0
votes
1answer
23 views

extract document topic vectors from lda model

how can I extract document-topic matrix from LDA model and use it as input features an svm classifier? I am using gensim for implementation
0
votes
2answers
36 views

How to form and minimise custom features for classification in supervised learning [closed]

I am having an issue in understanding how to form the features based on particular math formula, and how to adjust the weights with. The aim is to draw ellipses for each unique category of points. ...
0
votes
0answers
10 views

Class balancing in Weka

Hi I am using software fault prediction datasets, which has class values Yes (faulty) and No (non faulty). I have to do class balancing at ratio of 20-80% (faulty=20, non faulty=80). So for that,do I ...
0
votes
1answer
14 views

RNN LSTM input conflict due to generator

I am trying to do text classification with LSTM RNN on 255 length padded sequences. My classification data looks like this 1, 'sequence 1' The first column is ...
1
vote
0answers
34 views

Multilabel Classification; which network design?

I have a hard time thinking about how I can build this network with the following problem: I want to build a CNN to classify notes from sheet music. I have tried several models with and without ...
1
vote
1answer
23 views

When would not normalizing input values have higher accuracy?

Right now I'm training a deep neural network for a binary classification problem, with a feature set of winrates. As such, each winrate is bigger or equal to 0 but smaller than 100. I've been getting ...
0
votes
2answers
72 views

Class Imbalance and Cost-Sensitive Learning XGBoost

I'm fairly new to data science and machine learning and have been trying to read a bit more on methods like boosting for one of the projects I am working on. The investigator on this project is ...
0
votes
1answer
13 views

Which ML method for multiclass (non-binary) text classification should I choose (from SparkML)?

I am working on a quite big dataset that will be processed on the cluster, so this is why I am using PySpark for that purpose. The presentable records of this dataset have a such structure: ...
0
votes
0answers
16 views

Stratified sampling for large imbalanced data set

I have a very large data set (64 Million Rows) and I want to understand the best approach to sample the data for explorative data analysis before proceeding to perform any classification modeling. ...
0
votes
2answers
64 views

How to find whether a dataset is blanced or imbalanced?

I have few dataset to experiment classification(Multi-class). These datasets are about 400GB. I wanted to know whether the dataset is balanced or imbalanced. How to know that dataset is balance or ...
0
votes
0answers
9 views

Threshold for overfitted models

It's common knowledge in DS that overfitted models perform well on training data and poorly on test data. But how do you decide if a model is really overfitting? I have nowhere (books, online courses, ...
0
votes
2answers
29 views

Classifier design for website screenshots

I'm working on a project that requires determining if the page representing a hosted file on a third party platform (such as rapidgator or nitroflare) is still up or not. For example, here is a file ...
1
vote
1answer
31 views

How to choose solution - Neural Neworks or Scikit-Learn/Numpy/Pandas?

I am trying to solve a problem - categorising and routing service desk emails to concerned teams for resolution. Created and tested a model using Scikit-Learn, Numpy and Pandas. - Tokenized the email ...
0
votes
0answers
29 views

Predict Data Linking

I've two different datasets A(x1, x2, x3...xm) and B(y1, y2, y3...yn). Each unique row in A is linked with a unique row in B i.e. only one unique pair exists. If I get a new row in A and a new row in ...
0
votes
3answers
129 views

How to Split And Resample Imbalanced Dataset Into Train, Validation and Test

I want to understand how to split the imbalanced data set with a binary target variable where 87% of the samples are negative and 13% of the samples are positive. Now, I know that you should always ...
3
votes
2answers
42 views
+50

Splitting a pdf containing batch of scanned documents

My question is primarily: is there any ML research paper about splitting a pdf containing a batch of scanned documents (eg bank statements) into individual documents? I have searched for this but I ...
1
vote
2answers
99 views

class_weight on sklearn's DecisionTreeClassifier

Can class_weight='balanced' on scikit-learn's DecisionTreeClassifier be interpreted as having identical duplicate data points for the minority classes? I know that doesn't work that way, class_weight ...
0
votes
1answer
47 views

General approach on time series for customer retention/churn in retail

I have a time series of data in the following form: ...
0
votes
0answers
6 views

Combining decision trees and neural networks for classifying text with metadata . How to combine and train?

I have a multi-label classification problem where the input consist of free text, with metadata such as categories (from a fixed, limited set) associated with each text. The output consist of a set of ...
0
votes
1answer
87 views

Error encoding categorical features using sklearn pipelines

I am new to sklearn pipelines and am using this post as a guide for my code: https://www.codementor.io/bruce3557/beautiful-machine-learning-pipeline-with-scikit-learn-uiqapbxuj I am trying to encode ...
0
votes
1answer
17 views

Similar objects same labels

Example: red paprica and green paprica. The output I need is just paprica, should i label them the same or give them two labels "green paprica" and "red paprica" and leater on just treat them as one? ...
1
vote
1answer
86 views

Impose similar metric on segments to model

I am training a binary classifier in a dataset using AUC as a score. The dataset has two main groups (we will refer to them as good and bad population). A property that this dataset has is having a ...
3
votes
2answers
160 views

XGBoost validation for number of trees

I have a simple Question: I am using XGBoost to classify some data: 1.) With 100 estimators I have following scores(roc_score): train_data : 98.5 validation_data : 97.2 2.) With 500 ...
1
vote
1answer
26 views

How to control the amount of positives in classification?

I have a basic, yet quite complex problem to solve right now. Let's say we have a training set of 20,000 samples in my training set, out of which 3 to 4% is flagged as "True", the rest is flagged as "...
0
votes
1answer
24 views

How Does Cross-Entropy Work With Softmax Activation Function?

I found online that the derivative of a cross-entropy activation function with a softmax activation is (output - expected), which had me very confused. If for example, the expected value is 1, and ...
2
votes
0answers
37 views

Why decision tree algorithm performs better in small dataset in compare to other classification algorithm?

I read in some paper that decision tree algorithms have better result in small dataset like(400-600 records) when compared to other classification algorithms like SVM, Naive Bayes and KNN. I want to ...
1
vote
2answers
107 views

How to find “regions” with high purity

I am applying a ML model (LGBM binary classifier) to data and would now like to identify the part of data where I have a low ratio of false-negatives (false-postives are not such a problem) and as ...
0
votes
1answer
39 views

Non-Convex Constraints for Classification Problems

I am willing to create a hypothetical non-convex constraints for the purpose of practising nonlinear classification using an algorithm. I thought of such constraints in the form: $x^TAx + Bx \leq c$. ...
0
votes
0answers
15 views

How to build a 2-d bayes classifier

I'm working on a 2-d bayes classifier and I'm a little confused on how to start exactly. My attribute space is 2-d. There are 3 classes. The data is assumed to be normally distributed. p(x | y1): P(...
1
vote
1answer
41 views

Predict correct answer among ten answers for a given question

I have a case study to solve where I am given a dataset of questions and its answers, there are ten answers for a particular question. It's a classification problem where correct answer is having <...
1
vote
1answer
26 views

Growth function of a 6-dimensional linear classifier

In our course, we are dealing with a d-dimensional classification problem ($\chi = \mathbb{R}^{d}$ as our input space, and $y = \{-1,+1\}$). Our hypothesis class $H$ consists of all hypotheses of the ...
0
votes
2answers
65 views

Feature extraction; similarity and classification of accelerometer data

I have several expert persons performing the same specific action (for example, squat or leap forward) multiple times. Say 5 persons do 100 squats each. They have an accelerometer attached to the same ...
1
vote
1answer
23 views

Can a decision in a node of a decision tree be based on comparison between 2 columns of the dataset?

Assume the features in the dataframe are columns - A,B,C and my target is Y Can my decision tree have a decision node which looks for say, ...
0
votes
2answers
27 views

Classify if someone is home based on time

I have a dataset with locations and a timestamp of a subject. For each location and timestamp I determined by comparing the location to the home address if the subject was at home or not (0/1) and ...
0
votes
1answer
19 views

How to choose the features for an algorithm from the given attached screenshot?

How to choose the features from the given attached heat map & correlation factor for the classification algorithm? I have 6 different features i.e., ac233fc01403, ac233fc02eaa, ac233fc015f6, ...
0
votes
1answer
35 views

How to approach data prediction problem

I'm new to ML and data science. I would really like high level advice how to approach the following problem. I need to predict if an engine will fail, what I've is a sensor that give a certain value ...
0
votes
1answer
30 views

Classifying Letters using CNN - Help

so some context, I'm trying to develop an OCR (for fun) and for that reason I decided to first find text within a page, parse it in to letters within the text and from there try and classify the ...
0
votes
1answer
105 views

SMOTE vs SMOTENC for binary classifier with categorical and numeric data

I have a problem that I am having trouble thoroughly understanding. I am using Xgboost for classification. My y is 0 or 1 (true or false). I have categorical and numeric features, so theoretically, I ...
1
vote
1answer
139 views

Machine learning method to predict event date

Let's say I have a big dataset consisting of variables including but not limited to the start/end date of loans, their notional amount, a loan prepayment indicator etc. My goal is to create a model ...
1
vote
1answer
42 views

how to use word embedding to do document classification etc?

I just start learning NLP technology, such as GPT, Bert, XLnet, word2vec, Glove etc. I try my best to read papers and check source code. But I still cannot understand very well. When we use word2vec ...
1
vote
1answer
20 views

Interpreting fraction of zero weights in TensorFlow

I am using the TensorFlow to do a simple linear classification using logistic regression. The graph included from the TensorBoard displays what they call the fraction of zero weights. How do I ...