Questions tagged [performance]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
0answers
12 views

Improve performance of my CNN model

I am working on an image classification problem. There are 876 images in the training and 600 in the test dataset. It is a multi class classification for plants. Since this is my first CNN problem, ...
2
votes
0answers
31 views

Python library to process large files

I'm looking for a Python library to process large files without performance problems. For example I want to transfer large files with hash calculating. Python faces memory problems with such processes....
0
votes
0answers
10 views

How to calculate “minimal confidence for the correct label”

In RapidMiner one of the decision tree performance measures is called the margin. The margin is defined as "minimal confidence for the correct label". Can someone explain to me what it means and how ...
1
vote
0answers
13 views

Is there a cost associated with converting Koalas dataframe to Spark dataframe?

I know that pandas works "under the hood" with numpy arrays stored in dictionaries. In contrast, Koalas works with the underlying Spark framework. Does that mean that there is no extra cost associated ...
1
vote
1answer
23 views

what metrics to evaluate rank order results?

I have searched on stackexchange and found a couple of topics like this and this but they are not quite relevant to my problem (or at least I don't know how to make them relevant to my problem). ...
1
vote
0answers
13 views

Multiclass Classification Task - Performance on Each Class Compared to Chance?

As a part of a classification task, a classifier has decided whether different books belong to class A, B or C (which are imbalanced) by looking at certain feature of the book. I have calculated ...
5
votes
1answer
244 views

Why ML model produces different results despite random_state defined? And how to set global random seed for sklearn

I have been running few ML models on same set of data for a binary classification problem with class proportion of 33:67. I had the same algorithms and same set of hyperparamters during yesterday and ...
2
votes
3answers
72 views

Is over-fitting a matter of features engineering or a matter of parameters set?

I am using sklearn package to make models. I tried randomly to set some paramater to a sklearn.ensemble.RandomForestClassifier ...
0
votes
1answer
24 views

Customer Intelligence - How to measure success?

we are creating models that aim to filter new leads from our current customer base. We started to create propensity models that calculate a percentage for each customer for a certain product group. I ...
3
votes
2answers
28 views

Does Sampling size matters in Multi classification Model

I am working on a multi class classification model where few of the class are with less data compare to other classes. I used random sampling technique to create a sample from the population keeping ...
2
votes
1answer
31 views

Relating ROC curves with class statistics

I have three neural net models that I am running on the same dataset (of 7 classes) and calculate their class performance and also ROC curves. The firs tmodel is a 4-layer model with 8 neurons in each ...
0
votes
0answers
24 views

How to improve computation speed on Orange?

I need to process huge data sets on Orange 3.23. My Orange workflow contains several widgets in parallel. It seems that Orange launches each process at a time. Is there a way to run several processes ...
2
votes
1answer
32 views

How can we conclude that an optimization algorithm is better than another one

When we test a new optimization algorithm, what the process that we need to do?For example, do we need to run the algorithm several times, and pick a best performance,i.e., in terms of accuracy, f1 ...
0
votes
0answers
55 views

Doc2vec doesn't use all the CPU power

I have a big dataset that trying to train with a Doc2vec model. I am working on a 8 CPU, 32GB RAM, but as I can see on the monitoring tools, it only uses about 66-67% of the CPU. I am not sure if it ...
0
votes
0answers
13 views

Evaluating randomness in a model performance

I'm evaluating the variability in performance (AUC) in the test set of a machine learning model with an intrinsic random component (xgboost). How many sources of variation should I use? Just ...
0
votes
1answer
571 views

How label smoothing and label flipping increases the performance of a machine learning model

I have seen posts and research papers mentioning these techniques for improving the performance of a machine learning model. These techniques certainly make some sense in the case when we are not ...
1
vote
2answers
85 views

AUC ROC Threshold Setting in heavy imbalance

I am doing binary logistic regression on a dataset with very heavy class imbalance. Class 1 is only 1% of data. When I train logistic regressor without class weights I get ROC AUC Score of 0.6269. ...
2
votes
1answer
104 views

Xgboost multiple class predictive performance beats one versus rest

I have an NLP task I'm tackling with xgboost (R implementation). Before describing my doubt I'll give you some background: I have a corpus of documents for which I did topic discovery, using a term ...
1
vote
1answer
52 views

fast ML algorithms for binary classification with (large+sparse) binary input data

I'm sorry that this is so very broad, but as a non-ML scientist it feels to be almost impossible to keep up with recent developments (esp. in deep learning etc.). Hence, I'm asking for guidance on how ...
0
votes
0answers
10 views

Performant alternatives to Matrix package in R that requires minimal effort by end users to install

Background I'm going to be releasing a package for R that does calculations involving extremely large sparse matrices (1,000,000 x 1,000,000 is the minimum for what we consider useful). For this ...
0
votes
2answers
52 views

How can i test the performance of a model when the test data contains seen and unseen data

To test the performance of my model based on some selected features, i try to use unseen and seen data. However, when choosing the accuracy based on all data, the model is almost overfitting since ...
0
votes
1answer
18 views

Reordering feature and its impact

How does reordering the features impact model training and its performance? Per my understanding, it should not impact the model performance as weights get tuned according to feature value and not ...
0
votes
0answers
104 views

For a multiclass classification problem, how do we find the cohen kappa score?

So I have a multiclass classification problem and I have found the Matthews Correlation Coefficient of that (https://scikit-learn.org/stable/modules/model_evaluation.html#matthews-correlation-...
0
votes
1answer
110 views

Normalized metric for comparing regression models performance

I was recently trying to explain to someone whether performance of my estimation approach is good or bad. For instance, whether a model with Mean Absolute Error (MAE) of 17000 is a bad solution. It ...
0
votes
0answers
31 views

Changing the performance metric used to optimize with random-forest

I'm looking to change the performance metric used to optimize the training for my data set because it is highly unbalance. Because it's highly unbalanced, I don't feel like accuracy is the appropriate ...
2
votes
1answer
206 views

why does performance of machine learning models plateau after certain amount of training data

I'm referring to the below image which I came across. The explanation seemed intuitive at first but I don't think I understand how it works. The image says that the performance of traditional machine ...
0
votes
0answers
219 views

How to evaluate reinforcement learning model?

I am relatively new to reinforcement learning and have been experiencing with a reinforcement learning model to make decisions based on human activities (dynamic environment). Appreciate if someone ...
5
votes
1answer
263 views

GPU performance is about 50% slower than benchmarks

Running this benchmark I get 50% slower performance than the author on practically all deep learning sub problems (SINGLE precision and on TRAINING only): I tested this on a GeForce 1080 GTX Ti and ...
0
votes
1answer
30 views

Difference Between Performance Scores

I need some help to understand the meaning between these different scores. Currently, I am doing the classification problems using machine learning, and I have obtained the results for the ...
1
vote
1answer
172 views

Accuracy improving but, val_acc oscillating in ConvNet. What does it mean?

In my ConvNet model, i'm trying to classify some images. It is malware images and it doesn't contain complex features (i think), as expected model learn to classify images easily. You can see my ...
1
vote
0answers
52 views

Model Performance using Precision as evaluation metric

I am dealing with an imbalanced class with the following distribution : (Total dataset size : 10763 X 20) 0 : 91% 1 : 9% To build model on this dataset having class imbalance, I have compared ...
1
vote
0answers
256 views

How can I calculate perplexity for a bigram model?

I didn't find any function in nltk to calculate the perplexity. There are some codes I found: ...
0
votes
0answers
48 views

Cross-validation and out-of-bag bootstrap applications

I have a question regarding steps on which a specific resample method should be used in general. As far as I know: out-of-bag bootstrap is the resample method with replacement, which has lower ...
4
votes
1answer
256 views

CNN computing time on good CPU vs cheap GPU

I am a researcher working on my first deep learning project, which consists of using a CNN (pre-trained VGG16+2 densely connected layers) to classify drone imagery of vegetation. In trying to hack ...
2
votes
0answers
154 views

How to choose best model checkpoint when training deep learning model on all the data?

When training a final model for production, it's often recommended to train on all available data (train + dev + test), as discussed here. I'm training a deep learning model. I typically save and use ...
0
votes
1answer
31 views

How reliable are model performance reportings?

I had a conceptual doubt about estimating and reporting a classification model's performance. Say my model works with range of depth values and gives out different readings of test errors. We choose ...
4
votes
1answer
2k views

AUC ROC in keras is different when using tensorflow or scikit functions.

Two solutions for using AUC-ROC to train keras models, proposed here worked for me. But using tensorflow or scikit rocauc functions I get different results. ...
2
votes
4answers
240 views

Neural Network unseen data performance

I started dabbling in neural networks quite recently and encountered a situation which is quite strange (at least with my limited knowledge). The problem I'm using a NN is a regression problem which ...
3
votes
4answers
606 views

Metrics to determine K in K-cross fold validation

Consider a scenario where the dataset in hand is quite large, let's assume 50000 samples (quite well balanced between two classes). What metrics can be used to decide the K value in a K-fold cross-...
0
votes
2answers
345 views

How can I perform backpropagation directly in matrix form?

I had made a neural network library a few months ago, and I wasn't too familiar with matrices. So, instead of performing matrix dot products (between weights and inputs, then adding a bias matrix), I ...
5
votes
4answers
1k views

Log loss vs accuracy for deciding between different learning rates?

While model tuning using cross validation and grid search I was plotting the graph of different learning rate against log loss and accuracy separately. Log loss When I used log loss as score in ...
1
vote
0answers
75 views

Time-series clustering Quality Measures

I am clustering time-series datasets which are not labeled (No Ground truth) and I want to measure the quality of the clusters. Could you please suggest any Clustering performance evaluation methods ...
2
votes
1answer
51 views

“Each agent was evaluated every 250,000 training frames for 135,000 validation frames” What does this sentences stands for? in DQN nature paper?

In nature paper of DQN by DeepMind, DQN is compared to linear function but they does not said what is this linear function? They compared with some linear functions? 0- What is the meaning of this ...
1
vote
1answer
137 views

How to decide optimal threshold for my classification model from FPR, TPR and threshold

I am building my model in Python to classify customer in buyer/ non-buyer category. I used mutiple agorithms for this problem and then after evaluation selecting the best out of all. sklearn package ...
1
vote
1answer
287 views

bad regression performance on imbalanced dataset

My current dataset has a shape of 5300 rows by 160 columns with a numeric target variable range=[641, 3001]. That’s no big dataset, but should in general be enough for decent regression quality. The ...
0
votes
1answer
48 views

Confidence of this particual prediction

I am looking for a confidence of model to predict well in a given situation. So I have a model $f$ (generic, let's exemplify with a regression model of explicit form for brevity). It well fits on the ...
0
votes
1answer
81 views

Regression model performance with noisy dependent variable

I'm doing a support vector regression with the dependent variable representing measurements from an uncalibrated sensor (measurement error between 2% and 20%) and I want to study the effect of this ...
2
votes
1answer
295 views

how to improve searching index in dataframe

Given a pandas dataframe with a timestamp index, sorted. I have a label and I need to find the closest index to that label. Also, I need to find a smaller timestamp, so the search should be computed ...
8
votes
1answer
763 views

What is the best performance metric used in balancing dataset using SMOTE technique

I used smote technique to oversample my dataset and now I have a balanced dataset. The problem I faced is that the performance metrics; precision, recall, f1 measure, accuracy in the imbalanced ...
0
votes
3answers
39 views

classification performance metric for high risk medical decisions

What is the best classification performance metric for risky medical treatments like surgery? for example a patient should NOT suggest a surgery (negative) if he/she can be treated by medicine (...