Questions tagged [performance]

For Question about Performance of a data science, statistical or machine learning model. Performace is a direct way to measure the efficiency of model. The Performance measure deals with time, accuracy and scalability for improve the model.

Filter by
Sorted by
Tagged with
0 votes
0 answers
15 views

Comparing a classfiers performance across distinct testing datasets

I have a dataset split into training, validation, and testing sets. I trained this model on the training data and evaluated on the validation and testing sets. Now I have an additional set of data ...
tensormoby's user avatar
0 votes
0 answers
11 views

Interpreting large discrepancies between Specificities & the # of Extraneous Variable Models selected by a variable selection algorithm

I am going to preface my question by saying that this problem of interpretation I have run into is in the context of me doing my part as a collaborator on a statistical learning paper for the first ...
Marlen's user avatar
  • 157
0 votes
0 answers
18 views

Why does the first call to a TensorFlow function execute much slower than the second call?

I was doing an Image Classification problem using TensorFlow. I was generating the mean images for two image datasets having the same size. The dataset was generated using the tf.data API. Thereafter ...
Harsh Khare's user avatar
0 votes
0 answers
27 views

why is there no research on machine learning algorithms to determine optimal hyperparameters for metaheuristics?

I am not shure if I am in the correct forum for this question. I'm sorry, if I'm in the wrong place here. Question: Why is there no research on machine learning algorithms to determine hyperparameters ...
Andre's user avatar
  • 101
0 votes
0 answers
18 views

How does TensorFlow method tf.data.Dataset.reduce() work?

I was trying to compute the mean of the images that I fetched from a GCS bucket using TensorFlow's tf.data input pipeline. For this, I came across two methods: ...
Harsh Khare's user avatar
0 votes
0 answers
23 views

Different size of deep learning models but similar inference-time

I have three different semantic segmentation models with large differences in size. The first one includes 30,000,000 trainable parameters, the second one about 20,000,000 and the third one about 200,...
Capdi's user avatar
  • 111
0 votes
0 answers
12 views

One class confusion matrix notation for model evaluation

A one class classification set-up for a set of rules acting as a model, where each input is a whole dataset model makes some decision within the dataset for each entry output is decisions made for ...
zaira's user avatar
  • 101
1 vote
1 answer
40 views

Seeking guidance on understanding graphics card parameters for deep learning training

I am currently in the process of purchasing a new Nvidia graphics card for training deep learning models, and I have a few questions regarding the parameters involved and their relationship to the ...
ja1ba6's user avatar
  • 11
0 votes
0 answers
48 views

This model is too slow. I'm looking for a good, fast-enough, out-of-the-box, pre-trained image classifier. Any tip?

I have been using this on a laptop without a GPU: https://github.com/pharmapsychotic/clip-interrogator Currently it takes about 10s to classify a single image on my own computer. I use ...
jokoon's user avatar
  • 101
0 votes
0 answers
13 views

why performance values for passive aggressive model changing every time?

i havq question, that am a little confused about PassiveAggressive performance values after i fitted tf-idf vectorization on it(bigram and trigram), the values are changed every time i run the code ...
joj abd's user avatar
0 votes
0 answers
16 views

how to analyze model performance without any ML libraries using a BI tool?

I have a dataset that has the ML model predictions along with the true labels column of the predictions and ...
Adem Youssef's user avatar
0 votes
1 answer
55 views

How to reduce the false positives to improve the models performance?

I am currently building a binary classification model to predict order return rates. I used the GradientBoostingClassifier for training the model and also performed hyperparameter tuning using ...
Kedharnath Kb's user avatar
1 vote
1 answer
58 views

Imbalanced performance metrics in binary classification

I am developing a binary classification model using sklearn pipeline for preprocessing and a soft voting classifier (Adaboost and Extratrees with 50 estimators). The dataset (3 million rows) contains ...
fendrbud's user avatar
0 votes
0 answers
14 views

Why are decision trees driven by the Gini impurity as opposed to the accuracy? [duplicate]

It seems that most implementations of decision trees use the Gini impurity as their partitioning criterion. Why isn't accuracy used instead, since it's a more widespread metric across different ...
Tfovid's user avatar
  • 203
0 votes
1 answer
20 views

Applying the model on validation data achieves higher performance than on test set. Is this possible?

I trained a binary cross-validated classification model and got high performance (about 90) on the test data but when I apply the model to new unseen data to see how to performs, i get even higher ...
Din's user avatar
  • 11
0 votes
1 answer
17 views

Total Retention Rate Calculated from Categories

I am calculating retention for 3 categories and then total, and I am trying to double check my total, but my check formula isn't working. I am comparing the last 14 days (let's call it Period 1) to ...
user485656's user avatar
0 votes
0 answers
30 views

How to multiply the boolean/binary values in 1 row in a dataframe by the chr values in another row with the product landing in the original row

Suppose that I already know which subset out of a set of 30 candidate regressor columns are the true regressors included in the structural equation describing that dataset (because I do by ...
Marlen's user avatar
  • 157
0 votes
1 answer
72 views

Quantifying the performance of Stepwise Regression ran on Monte Carlo generated datasets & comparing them to your method of interest

The source data files and scripts referenced here and from whom lines of code are included here can be found in my GitHub Repository for this collaborative research project exploring the properties of ...
Marlen's user avatar
  • 157
0 votes
1 answer
374 views

Why does Adam outperform SGD in logistic regression?

I am training a logistic regression model. In case it matters, the features are 1376-dimensional embeddings output from a neural network. I tried both SGD and Adam with a learning rate of $10^{-3}$ ...
nalzok's user avatar
  • 113
0 votes
0 answers
27 views

Trying to make a visualization for training performance

I am using scikit learn's BayesianRidge model to fit a regression to tabular data of d features and N sample. I have already tested how well my model performs using a repeated kfold cross validation ...
lambdaChops's user avatar
0 votes
1 answer
58 views

Ordering training text data by length

If I have text data where the length of documents greatly varies and I'd like to use it for training where I use batching, there is a great chance that long strings will be mixed with short strings ...
Badr Jaidi's user avatar
1 vote
1 answer
49 views

How do well informed labels for ordinal encoding improve model performance?

From Kaggle's intermediate machine learning tutorial, it was stated that for each column, we randomly assign each unique value to a different integer. This is a common approach that is simpler than ...
Terrarium's user avatar
  • 113
1 vote
1 answer
73 views

Overall acurracy +/- E (with 90% C.I.)

I am assessing the accuracy of my classification model. I performed a 4-folds cross-validation and I obtained the following Overall Accuracy: OA = (0.910, 0.920, 0.880, 0.910). So, the average OA is 0....
sermomon's user avatar
0 votes
1 answer
50 views

Do I need to use AUPRC for reporting classification results on an imbalanced dataset when the model was trained using upsampling and CV

I am working on a binary classification problem which dataset has about 5% of positive class samples. I split the dataset, 70% for training and 30% for testing. I used the test data only once for ...
Paul's user avatar
  • 1
0 votes
1 answer
511 views

precision and recall at k for movielens dataset

I wanted to recreate a very simple collaborative filtering example with the 1M movielens dataset I have from Kaggle (https://www.kaggle.com/datasets/odedgolden/movielens-1m-dataset) and then ...
corianne1234's user avatar
1 vote
1 answer
32 views

Doubt about comparison of Machine Learning algorithm

I read this article about the comparison of Machine Learning algorithm. According to this article, there is some characteristics that define the ML models performance. I know that there are some ...
Inuraghe's user avatar
  • 481
0 votes
1 answer
327 views

Which classification_report metrics are appropriate to report/interpret for a binary label? Individual or macro average for both classes? scikit-learn

First, please forgive my ignorance; I am a newbie but dedicated to learning more. Example: I have a using a random forest classifier to predict a binary outcome. The binary outcome equals 1 if people ...
iamtheonewhoknocks's user avatar
2 votes
1 answer
1k views

Imbalanced data set with Sample weighting - How to interpret the performance metrics?

Consider a binary classification scenario whereby the True class (5%) is severely outbalanced to the False class (95%). My data set contains numeric data. I am using SKLearn and trying some different ...
Jurgen Cuschieri's user avatar
1 vote
1 answer
112 views

The behavior of the cross validation error and training error in underfitting case is not clear

I currently study the "Machine Learning" course on Coursera.org by Andrew Ng, it comes to a topic that discusses the performance of learning algorithms under different conditions. Here, we ...
Ahmed Hesham's user avatar
4 votes
1 answer
72 views

What could be good Perfomance Evaluation Metrics for a Data Scientist?

Background I'm a Data Scientist and am being asked to come up with a set of metrics/KPIs to assess my annual performance, and things like bonuses (and in the worst case being fired) depend on that. ...
Hasan Shaukat's user avatar
1 vote
0 answers
70 views

How to implement kfold and cv into Hybrid feature selection and evaluate the classification model performance?

I have been working on a Hybrid feature selection combined with hyperopt package for hyperparameter tuning and I am thinking about evaluating the performance of several model classifiers. I looked ...
WDpad159's user avatar
  • 111
0 votes
1 answer
528 views

Is sensitivity the same as recall in multiclass classification?

In Wikipedia, it is stated "In binary classification, recall is called sensitivity" under the Recall section. Are they both different in case of multi-class classification?
penguin_smasher's user avatar
2 votes
1 answer
532 views

How to run hdbscan clustering faster?

I'm using hdbscan to cluster embedding output from BERT, which took in a data file of >150k chat messages. The embedding process took a little over 4 minutes, but as of this writing the hdbscan ...
reedvoid's user avatar
  • 121
0 votes
0 answers
30 views

What should be the ratio between training time and accuracy?

What should be the ratio between time and accuracy? I mean when should you drop the accuracy a little bit but it will take less time for the Classifier/Regressor to run? Edit: As part of my studies I ...
L. breitman's user avatar
1 vote
1 answer
36 views

Measure performance of classification model for training on different snapshots

I am trying to do binary classification on some chronological data. Let's assume we have weekly data from the first week of 2017 through the last week of 2020. Now we have found out that 26 weeks of ...
Ricky's user avatar
  • 189
1 vote
1 answer
20 views

How to compute performance of a detection-classification system?

I use a yolo (y) to detect only one object and a multiclassifier (mc) that classifies that object. Now, the problem is: what I have to do with yolo's false positive and false negative, if I want to ...
Vincenzo's user avatar
-1 votes
1 answer
456 views

nested cross validation vs. train-test split

I am trying to understand the main benefits of conducting a nested cross-validation compared to a simpler train-test split. Let us say I would like to build a prediction model. I initially split my ...
dean's user avatar
  • 103
1 vote
1 answer
25 views

Performance measurement of an event extraction system

I have developed an event extraction system from text documents. It first clusters the data corpus and extracts answers for what, when and where questions. Final answers are determined by using a ...
Nilani Algiriyage's user avatar
0 votes
1 answer
272 views

What is the Most Efficient Tool in Python for row-wise manipulation of data?

I'm doing a lot of work that requires operations to be performed across rows, using the data in that rows's columns on other columns in the row. I recently had to do some processing on a 1.2 million ...
Scott's user avatar
  • 1
2 votes
1 answer
26 views

Create new performance indicators (error metrics)

I am wondering if any of you happen to know of a procedure/approach/rationale to develop new performance indicators (error metrics) that can be used to evaluate the prediction capability (say, ...
The Guy's user avatar
  • 121
1 vote
0 answers
23 views

correlated variables & model performance: optimal trade-off

on the back of this topic (When to remove correlated variables) I feel a follow up is needed, with the focus here being on raw performance and risk of distribution shift. assuming little to medium ...
Asher11's user avatar
  • 159
1 vote
0 answers
63 views

Creating a new feature from an existing one using decision trees

Is it possible to create a new feature out of two, or more than two existing features using a decision tree? If so, how, and can it produce features with good information value that can better help ...
Soumyajit Sarkar's user avatar
0 votes
0 answers
167 views

2D Z-score/Mahalanobis distance that includes a penalty for uncertainty

I have some 2D points and I want to assess their performance against the target point. When I was doing this in 1D, I took the Z-score Z = (x- mu)/sigma, but that ...
Lizardinablizzard's user avatar
2 votes
2 answers
242 views

KNN efficient implementation

The KNN algorithm is very handy and particularly suited to some of my problems, but I can't find any resources on how to implement it in production. As a comparative example, when I use a neural ...
Nathan Jodo's user avatar
0 votes
0 answers
58 views

How to increase accuracy and decrease loss of my model

https://jovian.ai/casella0798/badmodel I created the model above to predict red wine quality. I have 6 classes, from 3 to 8. Dataset is unbalanced, with a lot of classes 5 and 6. My model performs ...
CasellaJr's user avatar
  • 229
2 votes
2 answers
67 views

Choosing best model produced from different algorithms. Metric produced by cross-validation on the train set or metric produced on the test set?

I know that choosing between models produced by one algorithm with different hyperparameters the metric for choosing the best one should be the cross-validation on train set. But what about choosing ...
vasili111's user avatar
  • 157
2 votes
2 answers
77 views

Once a predictive model is in production, how it can be evaluated?

I have a data science project, predicting customer's next purchase day. Customer's one year behavioral data was split to 9 and 3 months for train and test, using RFM analysis, I trained a model with ...
Mahsaem's user avatar
  • 23
1 vote
1 answer
1k views

CNN inference is slow on Jetson Nano

I'm running what I believe is a pretty lightweight CNN on an nVidia Jetson Nano with Jetpack 4.4. nVidia claims the Nano can run a ResNet-50 at 36fps, so I expected my much smaller network to run at ...
couka's user avatar
  • 121
1 vote
1 answer
91 views

How do data types influence hardware (CPU / GPU / TPU) performance?

I am currently dealing with a relatively big data set, for which I have some memory usage concerns. I am dealing with most of the different data types : floats, integers, Booleans, characters strings ...
Lucas Morin's user avatar
  • 2,567
2 votes
1 answer
863 views

What is the appropriate statistical significance test for multi-class classification?

I have a multi-class classification problem. I am primarily using macro-average F1 measure to evaluate the performance of models and want to verify if the results are statistically significant. I have ...
Ashwin Geet D'Sa's user avatar