Questions tagged [performance]

For Question about Performance of a data science, statistical or machine learning model. Performace is a direct way to measure the efficiency of model. The Performance measure deals with time, accuracy and scalability for improve the model.

Filter by
Sorted by
Tagged with
1
vote
1answer
49 views

How to select a model based on statistical analyses?

I've been statistically validating the performance of different Deep Learning models in classifying parasitized and normal cells. In the process, I could find that there is no statistically ...
14
votes
3answers
27k views

Improve Pandas dataframe filtering speed

I have a dataset with 19 columns and about 250k rows. I have worked with bigger datasets, but this time, Pandas decided to play with my nerves. I tried to split the original dataset into 3 sub-...
0
votes
1answer
841 views

Why use the .idx data format?

The MNIST handwritten digit dataset uses a file format .idx. What are the advantages of this file format over alternatives such as CSV, TSX and ODS?
0
votes
1answer
31 views

How to evaluate the performance based on rate data

I have the following data: Goal Achieved 100 90 150 130 200 175 ... The first column "Goal" is the number which should be done that ...
0
votes
2answers
194 views

How do I compare traditional classifiers performance with my proposed method?

I want to compare my proposed method with traditional machine learning classifiers like Multilayer Perceptron(MLP) and SVM to check the classification accuracy.How do I compare different classifiers?. ...
1
vote
1answer
653 views

How to compare performance of Cosine Similarity and Manhatten Distance?

I'm doing clustering of documents by applying k-Means on the word-vectors. To measure the cluster quality, I calculate David Bouldin Index for different k's. I tried two different distance measures, ...
3
votes
1answer
4k views

Performance issues when merging two dataframe columns into one on millions rows with Pandas

I am trying to merge two address columns into one and separate the resulting string with '--'. The dataset has 10 million rows and 33 columns - but the number of rows grows for a million or so a month....
1
vote
2answers
163 views

ML models: average of all versus average of averages?

I have data for 20 different people and am training a model (e.g. a neural network with the same hyperparameters) on the data from each person; so this gives me 20 models. I chose to use RMSE to ...
4
votes
1answer
151 views

Estimating Titan X graphics card impact on performance

I'm currently training CNNs using Tensorflow (Python) on my GTX 970 (specs here). I recently took a look at the new pascal based Titan Xs and I'm wondering what an estimated performance/speed gain ...
0
votes
6answers
15k views

Which train test split performs better: 50:50 or 60:40? [closed]

I have 10,000 customer data of a supermarket. And I want to split the data into training set and testing set. So, which train test split gives me a better accuracy: 50:50 or 60:40?
2
votes
1answer
291 views

Comparing Non-deterministic Binary Classifiers

I have two classifiers which I am implementing, and they are both non-deterministic in the sense that they can each give different results (FPR and TPR) when you run them multiple times. I would like ...
2
votes
1answer
88 views

Vectorizing/Parallelizing DataFrame indexing

I want to make the following Python data-processing code more efficient by replacing for loops. Is there any way to vectorize code like this? I have a DataFrame object ...
15
votes
4answers
3k views

How to compare the performance of feature selection methods?

There are several feature selection / variable selection approaches (see for example Guyon & Elisseeff, 2003; Liu et al., 2010): filter methods (e.g., correlation-based, entropy-based, random ...
1
vote
2answers
2k views

Using the trainbr function for classification in Matlab

I am training a neural network for classification using Matlab, and I don't understand if I can use the trainbr training function (Bayesian Regularization ...
1
vote
0answers
62 views

Is there a machine learning framework that supports partial evaluation ie can return a function?

Is there a machine learning framework that supports partial evaluation? For example: We train on [model, year, km, ..., colour, price]. Today we call ...
8
votes
1answer
8k views

Why doesn't training RNNs use 100% of the GPU?

I wonder why training RNNs typically doesn't use 100% of the GPU. For example, if I run this RNN benchmark on a Maxwell Titan X on Ubuntu 14.04.4 LTS x64, the GPU utilization is below 90%: The ...
1
vote
1answer
181 views

Optimal parameter estimation for a classifier with multiple parameters

The image on the left shows a standard ROC curve formed by sweeping a single threshold and recording the corresponding True Positive Rate (TPR) and False Positive Rate (FPR). The image on the right ...
1
vote
2answers
1k views

Benchmarks based on neural networks libraries to compare the performance between different GPUs

I am looking for benchmarks based on neural networks libraries (Theano/TensorFlow/Torch/Caffe/…) to compare the performance between different GPUs. I am aware of: https://github.com/jcjohnson/cnn-...
2
votes
2answers
271 views

ARIMA(X) Validation

I'm in the process of developing a new spark-based ARIMA(X) tool, and have reached the point where I need to know whether my coefficient estimates and forecasts are sensible. I can compare my results ...
0
votes
2answers
75 views

What do this Classification evaluation results mean to you? Do they are suspicious or not?

I have collected dataset with two class labels and used the SVM Method to classify the dataset, and this is the results. Does this appear suspicious or not? scikit-learn classifiers with SVM SVC ...
0
votes
1answer
38 views

Measuring performance of different classifiers without class type in data

To measure the performance of a classification algorithms on a dataset that has an attribute for class type, I divide my dataset to training and test samples and then create a ...
0
votes
1answer
63 views

Cross-validation strategy

I have a regression problem and I am in doubt about how I can calculate RMSE in my life-cycle. I deal with time-series and for every prediction, I want to look N points in the future. It is apparent ...
1
vote
0answers
62 views

Analytics oriented database

I'm analyzing a single large table (about 50GB 16M rows, 15 columns perhaps more) currently using PostgreSQL 9.4. Most of my queries doesn't involve more tables. Queries run time are very large. I'm ...
1
vote
1answer
678 views

Accuracy value constant even after different runs

I am using the neural network toolbox of Matlab to train a network. Now my code is as follows: ...
3
votes
1answer
176 views

Understanding ROCs in imbalanced data-sets

A response variable (label) $B$ can either be $0$ or $1$. In the training set, $B_i = 1$ is an extremely rare event at only $0.26\%$ occurrences. Which makes the prediction of this label on a test ...
13
votes
4answers
1k views

Performance measure: Why is it called recall and sensitivity?

precision is the fraction of retrieved instances that are relevant, while recall (also known as sensitivity) is the fraction of relevant instances that are retrieved. I know their meaning but I don'...
0
votes
1answer
3k views

python pandas optimization: filtering on text index values

I have to filter a pandas data frame by matching a complex regular expression on a text index. The data frame is multi level indexed, and contains more than 2 million records. The way I'm doing is: ...
1
vote
1answer
3k views

Training And Testing Error Curves caret package in r

I am Running the following models Logistic regression Decision Trees SVM Naive Bayes Random Forest On the same data set. I am using Caret package in r. Its My dream to plot Training error and ...
1
vote
1answer
507 views

ROC curves/AUC values as a performance metric

I want to plot ROC curves using R. I have a prediction matrix, where each column shows the prediction values corresponding to different approaches. Also, I have a label vector. The column names of ...
1
vote
1answer
360 views

Testing Model Performance

Once the Model is built we want to check its performance, i did the following Predicted it on training set. Compute confusion matrix and ROC curve on training set. Predicted on test set Computed ...
4
votes
1answer
234 views

Benchmarking Theano

I sometimes want to benchmark Theano, either to compare different versions of Theano, or to compare two different computing environment with the same version of Theano. Is there any Theano code I ...
1
vote
4answers
3k views

What can I do with a Decision Tree with poor ROC

Let's say I do a Decision Tree analysis. But the performance characteristics are nothing great (e.g. ROC is nothing great). Is there anything I can do with this "not so great" tree. Or do I ...
6
votes
1answer
153 views

Geometric Interpretation of Whether SVMs are performing well or not

I came across this research paper which contained this figure which talks about the center of mass (presumably, of the training dataset's datapoints?) and represents the solution of an SVM as ...
4
votes
2answers
20k views

What does RMSE points about performance of a model in machine learning?

I am working on Decision Tree algorithm and at the end I calculate RMSE value based on actual labels and predicted values (for ...
1
vote
2answers
1k views

PCA on MNIST dataset results in very poor performance

I am trying to build a model for classifying MNIST dataset using SVM. With raw features I am getting accuracy of around 94% (using linear kernel). When I tried it with PCA, with different number of ...
2
votes
2answers
3k views

Slow convergence with rpart

I'm building a decision tree in R using the rpart function, available in the library of the same name, but am experiencing some serious performance issues when ...
2
votes
1answer
141 views

Case when Out of Bag Error and Test error differs a lot in Random Forest

I'm using random forest and the out of bag error for the level of one class is very different to its test error. I'm working with a cutt-of equal to c(0.2,0.8). Here's the case: ...
3
votes
1answer
416 views

Online/incremental unsupervised dimensionality reduction for use with classification for event prediction

Consider the application: We have a set of users and items. Users can perform different action types (think browsing, clicking, upvoting etc.) on different items. Users and items accumulate a "...
3
votes
1answer
3k views

How is H2O faster than R or SAS?

I am trying to understand the abstract details that explain how h2o is faster than R and SAS for data science computations.
0
votes
1answer
437 views

Python library to compute some metrics for multioutput-multiclass classification task

Is there any Python library that provides ready-to-use metrics to analyze the performance of a classifier for a multioutput-multiclass classification task? scikit-learn doesn't have this option yet (...
0
votes
1answer
4k views

What are the performance measures in the neural networks field?

I constructed a neural networks in R using neuralnet package. I want to test that using cross-validation, that is a technique based on using 4/5 of the dataset to ...
2
votes
1answer
86 views

Which open-source sgdb for kind of large data

I have a 7 giga confidential dataset which I want to use for a machine learning application. I tried : Every package recommanded for efficient dataset management in R like : data.table, ff and ...
23
votes
3answers
493 views

How do you manage expectations at work?

With all the hoopla around Data Science, Machine Learning, and all the success stories around, there are a lot of both justified, as well as overinflated, expectations from Data Scientists and their ...
0
votes
1answer
1k views

Why is the Naive Bayes classifier of sklearn faster than sklearns SVM?

I've used scikit-learn in Python to compare results of naive Bayes and SVM. I've found that naive Bayes is quicker than SVM. Could anyone shed some light on reasons ...
4
votes
2answers
4k views

R and Python, memory differences yielding performance differences

I think there are numerous posts regarding which one to use: R or Python. However, I'm curious about how their architecture differences yield differences in speed performance, not which one to use. ...
4
votes
2answers
3k views

Accuracy of Stanford NER

I am performing Named Entity Recognition using Stanford NER. I have successfully trained and tested my model. Now I want to know: 1) What is the general way of measuring accuracy of NER model ?? For ...
0
votes
1answer
297 views

When it is time to use Hadoop? [closed]

Hadoop is a buzzword now. A lot of start-ups use it (or just say, that they use it), a lot of widely known companies use it. But when and what is the border? When person can say: "Better to solve it ...
3
votes
1answer
35 views

Timing sequence in MapReduce

I'm running a test on MapReduce algorithm in different environments, like Hadoop and MongoDB, and using different types of data. What are the different methods or techniques to find out the execution ...
5
votes
3answers
443 views

What technologies are fastest at performing joins on large datasets?

By "large", I mean in the range of 100m to 10b rows. I'm currently using both Hadoop MapReduce and Amazon RedShift. MapReduce has been a little disappointing here. Redshift works very well if the ...
4
votes
1answer
2k views

Scikit Learn Logistic Regression Memory Leak

I'm curious if anyone else has run into this. I have a data set with about 350k samples, each with 4k sparse features. The sparse fill rate is about 0.5%. The data is stored in a ...