A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

Questions tagged [performance]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
0answers
89 views

Performance Metric for topic extraction when there is no ground truth

I am extracting topics from text using a predefined ontology containing 2690 concepts, wordnet(to expand concept terms with their synsets, and other morphological forms of the same word) and lucene to ...
1
vote
0answers
27 views

How to measure the performance of a deep learning model in parasite detection?

I’m working with segmented cells from thin blood smear images and using deep learning models to classify parasitic and uninfected cells. I obtained the following values: Accuracy: 0.986, AUC: 0.99, ...
1
vote
1answer
1k views

how should I measure performance if there is no test data?

I have 'practice' data set which I can split into training, validation, and test set and I will play with data to make a machine learning model. But in real situation, I will be given a very small ...
1
vote
1answer
44 views

How to select a model based on statistical analyses?

I've been statistically validating the performance of different Deep Learning models in classifying parasitized and normal cells. In the process, I could find that there is no statistically ...
9
votes
3answers
13k views

Improve Pandas dataframe filtering speed

I have a dataset with 19 columns and about 250k rows. I have worked with bigger datasets, but this time, Pandas decided to play with my nerves. I tried to split the original dataset into 3 sub-...
0
votes
1answer
579 views

Why use the .idx data format?

The MNIST handwritten digit dataset uses a file format .idx. What are the advantages of this file format over alternatives such as CSV, TSX and ODS?
0
votes
1answer
31 views

How to evaluate the performance based on rate data

I have the following data: Goal Achieved 100 90 150 130 200 175 ... The first column "Goal" is the number which should be done that ...
0
votes
2answers
140 views

How do I compare traditional classifiers performance with my proposed method?

I want to compare my proposed method with traditional machine learning classifiers like Multilayer Perceptron(MLP) and SVM to check the classification accuracy.How do I compare different classifiers?. ...
1
vote
1answer
535 views

How to compare performance of Cosine Similarity and Manhatten Distance?

I'm doing clustering of documents by applying k-Means on the word-vectors. To measure the cluster quality, I calculate David Bouldin Index for different k's. I tried two different distance measures, ...
3
votes
1answer
3k views

Performance issues when merging two dataframe columns into one on millions rows with Pandas

I am trying to merge two address columns into one and separate the resulting string with '--'. The dataset has 10 million rows and 33 columns - but the number of rows grows for a million or so a month....
1
vote
2answers
93 views

ML models: average of all versus average of averages?

I have data for 20 different people and am training a model (e.g. a neural network with the same hyperparameters) on the data from each person; so this gives me 20 models. I chose to use RMSE to ...
4
votes
1answer
129 views

Estimating Titan X graphics card impact on performance

I'm currently training CNNs using Tensorflow (Python) on my GTX 970 (specs here). I recently took a look at the new pascal based Titan Xs and I'm wondering what an estimated performance/speed gain ...
2
votes
1answer
260 views

Comparing Non-deterministic Binary Classifiers

I have two classifiers which I am implementing, and they are both non-deterministic in the sense that they can each give different results (FPR and TPR) when you run them multiple times. I would like ...
2
votes
1answer
77 views

Vectorizing/Parallelizing DataFrame indexing

I want to make the following Python data-processing code more efficient by replacing for loops. Is there any way to vectorize code like this? I have a DataFrame object ...
10
votes
4answers
1k views

How to compare the performance of feature selection methods?

There are several feature selection / variable selection approaches (see for example Guyon & Elisseeff, 2003; Liu et al., 2010): filter methods (e.g., correlation-based, entropy-based, random ...
1
vote
2answers
2k views

Using the trainbr function for classification in Matlab

I am training a neural network for classification using Matlab, and I don't understand if I can use the trainbr training function (Bayesian Regularization ...
1
vote
0answers
47 views

Is there a machine learning framework that supports partial evaluation ie can return a function?

Is there a machine learning framework that supports partial evaluation? For example: We train on [model, year, km, ..., colour, price]. Today we call ...
8
votes
1answer
7k views

Why doesn't training RNNs use 100% of the GPU?

I wonder why training RNNs typically doesn't use 100% of the GPU. For example, if I run this RNN benchmark on a Maxwell Titan X on Ubuntu 14.04.4 LTS x64, the GPU utilization is below 90%: The ...
1
vote
1answer
167 views

Optimal parameter estimation for a classifier with multiple parameters

The image on the left shows a standard ROC curve formed by sweeping a single threshold and recording the corresponding True Positive Rate (TPR) and False Positive Rate (FPR). The image on the right ...
1
vote
2answers
1k views

Benchmarks based on neural networks libraries to compare the performance between different GPUs

I am looking for benchmarks based on neural networks libraries (Theano/TensorFlow/Torch/Caffe/…) to compare the performance between different GPUs. I am aware of: https://github.com/jcjohnson/cnn-...
2
votes
2answers
182 views

ARIMA(X) Validation

I'm in the process of developing a new spark-based ARIMA(X) tool, and have reached the point where I need to know whether my coefficient estimates and forecasts are sensible. I can compare my results ...
0
votes
2answers
66 views

What do this Classification evaluation results mean to you? Do they are suspicious or not?

I have collected dataset with two class labels and used the SVM Method to classify the dataset, and this is the results. Does this appear suspicious or not? scikit-learn classifiers with SVM SVC ...
0
votes
1answer
33 views

Measuring performance of different classifiers without class type in data

To measure the performance of a classification algorithms on a dataset that has an attribute for class type, I divide my dataset to training and test samples and then create a ...
0
votes
1answer
53 views

Cross-validation strategy

I have a regression problem and I am in doubt about how I can calculate RMSE in my life-cycle. I deal with time-series and for every prediction, I want to look N points in the future. It is apparent ...
1
vote
0answers
60 views

Analytics oriented database

I'm analyzing a single large table (about 50GB 16M rows, 15 columns perhaps more) currently using PostgreSQL 9.4. Most of my queries doesn't involve more tables. Queries run time are very large. I'm ...
1
vote
1answer
538 views

Accuracy value constant even after different runs

I am using the neural network toolbox of Matlab to train a network. Now my code is as follows: ...
3
votes
1answer
165 views

Understanding ROCs in imbalanced data-sets

A response variable (label) $B$ can either be $0$ or $1$. In the training set, $B_i = 1$ is an extremely rare event at only $0.26\%$ occurrences. Which makes the prediction of this label on a test ...
10
votes
4answers
376 views

Performance measure: Why is it called recall?

precision is the fraction of retrieved instances that are relevant, while recall (also known as sensitivity) is the fraction of relevant instances that are retrieved. I know their meaning but I don'...
0
votes
1answer
2k views

python pandas optimization: filtering on text index values

I have to filter a pandas data frame by matching a complex regular expression on a text index. The data frame is multi level indexed, and contains more than 2 million records. The way I'm doing is: ...
1
vote
1answer
3k views

Training And Testing Error Curves caret package in r

I am Running the following models Logistic regression Decision Trees SVM Naive Bayes Random Forest On the same data set. I am using Caret package in r. Its My dream to plot Training error and ...
1
vote
1answer
445 views

ROC curves/AUC values as a performance metric

I want to plot ROC curves using R. I have a prediction matrix, where each column shows the prediction values corresponding to different approaches. Also, I have a label vector. The column names of ...
0
votes
1answer
230 views

Testing Model Performance

Once the Model is built we want to check its performance, i did the following Predicted it on training set. Compute confusion matrix and ROC curve on training set. Predicted on test set Computed ...
4
votes
1answer
210 views

Benchmarking Theano

I sometimes want to benchmark Theano, either to compare different versions of Theano, or to compare two different computing environment with the same version of Theano. Is there any Theano code I ...
1
vote
4answers
2k views

What can I do with a Decision Tree with poor ROC

Let's say I do a Decision Tree analysis. But the performance characteristics are nothing great (e.g. ROC is nothing great). Is there anything I can do with this "not so great" tree. Or do I ...
6
votes
1answer
150 views

Geometric Interpretation of Whether SVMs are performing well or not

I came across this research paper which contained this figure which talks about the center of mass (presumably, of the training dataset's datapoints?) and represents the solution of an SVM as ...
3
votes
2answers
14k views

What does RMSE points about performance of a model in machine learning?

I am working on Decision Tree algorithm and at the end I calculate RMSE value based on actual labels and predicted values (for ...
1
vote
2answers
1k views

PCA on MNIST dataset results in very poor performance

I am trying to build a model for classifying MNIST dataset using SVM. With raw features I am getting accuracy of around 94% (using linear kernel). When I tried it with PCA, with different number of ...
2
votes
2answers
2k views

Slow convergence with rpart

I'm building a decision tree in R using the rpart function, available in the library of the same name, but am experiencing some serious performance issues when ...
2
votes
1answer
129 views

Case when Out of Bag Error and Test error differs a lot in Random Forest

I'm using random forest and the out of bag error for the level of one class is very different to its test error. I'm working with a cutt-of equal to c(0.2,0.8). Here's the case: ...
3
votes
1answer
334 views

Online/incremental unsupervised dimensionality reduction for use with classification for event prediction

Consider the application: We have a set of users and items. Users can perform different action types (think browsing, clicking, upvoting etc.) on different items. Users and items accumulate a "...
3
votes
1answer
2k views

How is H2O faster than R or SAS?

I am trying to understand the abstract details that explain how h2o is faster than R and SAS for data science computations.
0
votes
1answer
228 views

Python library to compute some metrics for multioutput-multiclass classification task

Is there any Python library that provides ready-to-use metrics to analyze the performance of a classifier for a multioutput-multiclass classification task? scikit-learn doesn't have this option yet (...
0
votes
1answer
3k views

What are the performance measures in the neural networks field?

I constructed a neural networks in R using neuralnet package. I want to test that using cross-validation, that is a technique based on using 4/5 of the dataset to ...
2
votes
1answer
78 views

Which open-source sgdb for kind of large data

I have a 7 giga confidential dataset which I want to use for a machine learning application. I tried : Every package recommanded for efficient dataset management in R like : data.table, ff and ...
23
votes
3answers
453 views

How do you manage expectations at work?

With all the hoopla around Data Science, Machine Learning, and all the success stories around, there are a lot of both justified, as well as overinflated, expectations from Data Scientists and their ...
0
votes
1answer
882 views

Why is the Naive Bayes classifier of sklearn faster than sklearns SVM?

I've used scikit-learn in Python to compare results of naive Bayes and SVM. I've found that naive Bayes is quicker than SVM. Could anyone shed some light on reasons ...
3
votes
2answers
3k views

R and Python, memory differences yielding performance differences

I think there are numerous posts regarding which one to use: R or Python. However, I'm curious about how their architecture differences yield differences in speed performance, not which one to use. ...
4
votes
2answers
3k views

Accuracy of Stanford NER

I am performing Named Entity Recognition using Stanford NER. I have successfully trained and tested my model. Now I want to know: 1) What is the general way of measuring accuracy of NER model ?? For ...
0
votes
1answer
255 views

When it is time to use Hadoop? [closed]

Hadoop is a buzzword now. A lot of start-ups use it (or just say, that they use it), a lot of widely known companies use it. But when and what is the border? When person can say: "Better to solve it ...
3
votes
0answers
33 views

Timing sequence in MapReduce

I'm running a test on MapReduce algorithm in different environments, like Hadoop and MongoDB, and using different types of data. What are the different methods or techniques to find out the execution ...