Questions tagged [performance]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
87
votes
12answers
14k views

How big is big data?

Lots of people use the term big data in a rather commercial way, as a means of indicating that large datasets are involved in the computation, and therefore potential solutions must have good ...
25
votes
4answers
13k views

Is there a straightforward way to run pandas.DataFrame.isin in parallel?

I have a modeling and scoring program that makes heavy use of the DataFrame.isin function of pandas, searching through lists of facebook "like" records of ...
23
votes
3answers
455 views

How do you manage expectations at work?

With all the hoopla around Data Science, Machine Learning, and all the success stories around, there are a lot of both justified, as well as overinflated, expectations from Data Scientists and their ...
13
votes
1answer
199 views

When a relational database has better performance than a no relational

When a relational database, like MySQL, has better performance than a no relational, like MongoDB? I saw a question on Quora other day, about why Quora still uses MySQL as their backend, and that ...
12
votes
3answers
277 views

Measuring performance of different classifiers with different sample sizes

I'm currently using several different classifiers on various entities extracted from text, and using precision/recall as a summary of how well each separate classifier performs across a given dataset. ...
10
votes
4answers
188 views

Why is it hard to grant efficiency while using libraries?

Any small database processing can be easily tackled by Python/Perl/... scripts, that uses libraries and/or even utilities from the language itself. However, when it comes to performance, people tend ...
10
votes
4answers
411 views

Performance measure: Why is it called recall?

precision is the fraction of retrieved instances that are relevant, while recall (also known as sensitivity) is the fraction of relevant instances that are retrieved. I know their meaning but I don'...
10
votes
4answers
1k views

How to compare the performance of feature selection methods?

There are several feature selection / variable selection approaches (see for example Guyon & Elisseeff, 2003; Liu et al., 2010): filter methods (e.g., correlation-based, entropy-based, random ...
9
votes
3answers
14k views

Improve Pandas dataframe filtering speed

I have a dataset with 19 columns and about 250k rows. I have worked with bigger datasets, but this time, Pandas decided to play with my nerves. I tried to split the original dataset into 3 sub-...
8
votes
1answer
759 views

What is the best performance metric used in balancing dataset using SMOTE technique

I used smote technique to oversample my dataset and now I have a balanced dataset. The problem I faced is that the performance metrics; precision, recall, f1 measure, accuracy in the imbalanced ...
8
votes
3answers
123 views

How to compare experiments run over different infrastructures

I'm developing a distributed algorithm, and to improve efficiency, it relies both on the number of disks (one per machine), and on an efficient load balance strategy. With more disks, we're able to ...
8
votes
1answer
7k views

Why doesn't training RNNs use 100% of the GPU?

I wonder why training RNNs typically doesn't use 100% of the GPU. For example, if I run this RNN benchmark on a Maxwell Titan X on Ubuntu 14.04.4 LTS x64, the GPU utilization is below 90%: The ...
7
votes
2answers
169 views

What makes you confident in your results? At what point do you think you can present your work to tech illiterate superiors?

I understand that the models are only as good as the data you get, and bad design can generate really bad data. Nonrandom sampling, unbalanced/incomplete designs, confounding, can make data analysis ...
6
votes
1answer
520 views

What is the Best NoSQL backend for a mobile game

What is the best noSQL backend to use for a mobile game? Users can make a lot of servers requests, it needs also to retrieve users' historical records (like app purchasing) and analytics of usage ...
6
votes
1answer
150 views

Geometric Interpretation of Whether SVMs are performing well or not

I came across this research paper which contained this figure which talks about the center of mass (presumably, of the training dataset's datapoints?) and represents the solution of an SVM as ...
5
votes
2answers
768 views

How to speedup message passing between computing nodes

I'm developing a distributed application, and as it's been designed, there'll be a great load of communication during the processing. Since the communication is already as much spread along the entire ...
5
votes
1answer
236 views

Why ML model produces different results despite random_state defined? And how to set global random seed for sklearn

I have been running few ML models on same set of data for a binary classification problem with class proportion of 33:67. I had the same algorithms and same set of hyperparamters during yesterday and ...
5
votes
4answers
941 views

Log loss vs accuracy for deciding between different learning rates?

While model tuning using cross validation and grid search I was plotting the graph of different learning rate against log loss and accuracy separately. Log loss When I used log loss as score in ...
5
votes
3answers
303 views

What technologies are fastest at performing joins on large datasets?

By "large", I mean in the range of 100m to 10b rows. I'm currently using both Hadoop MapReduce and Amazon RedShift. MapReduce has been a little disappointing here. Redshift works very well if the ...
5
votes
1answer
257 views

GPU performance is about 50% slower than benchmarks

Running this benchmark I get 50% slower performance than the author on practically all deep learning sub problems (SINGLE precision and on TRAINING only): I tested this on a GeForce 1080 GTX Ti and ...
4
votes
1answer
129 views

Estimating Titan X graphics card impact on performance

I'm currently training CNNs using Tensorflow (Python) on my GTX 970 (specs here). I recently took a look at the new pascal based Titan Xs and I'm wondering what an estimated performance/speed gain ...
4
votes
2answers
14k views

What does RMSE points about performance of a model in machine learning?

I am working on Decision Tree algorithm and at the end I calculate RMSE value based on actual labels and predicted values (for ...
4
votes
1answer
2k views

Scikit Learn Logistic Regression Memory Leak

I'm curious if anyone else has run into this. I have a data set with about 350k samples, each with 4k sparse features. The sparse fill rate is about 0.5%. The data is stored in a ...
4
votes
1answer
2k views

AUC ROC in keras is different when using tensorflow or scikit functions.

Two solutions for using AUC-ROC to train keras models, proposed here worked for me. But using tensorflow or scikit rocauc functions I get different results. ...
4
votes
2answers
3k views

Accuracy of Stanford NER

I am performing Named Entity Recognition using Stanford NER. I have successfully trained and tested my model. Now I want to know: 1) What is the general way of measuring accuracy of NER model ?? For ...
4
votes
1answer
250 views

CNN computing time on good CPU vs cheap GPU

I am a researcher working on my first deep learning project, which consists of using a CNN (pre-trained VGG16+2 densely connected layers) to classify drone imagery of vegetation. In trying to hack ...
4
votes
1answer
211 views

Benchmarking Theano

I sometimes want to benchmark Theano, either to compare different versions of Theano, or to compare two different computing environment with the same version of Theano. Is there any Theano code I ...
3
votes
1answer
2k views

How is H2O faster than R or SAS?

I am trying to understand the abstract details that explain how h2o is faster than R and SAS for data science computations.
3
votes
4answers
591 views

Metrics to determine K in K-cross fold validation

Consider a scenario where the dataset in hand is quite large, let's assume 50000 samples (quite well balanced between two classes). What metrics can be used to decide the K value in a K-fold cross-...
3
votes
2answers
28 views

Does Sampling size matters in Multi classification Model

I am working on a multi class classification model where few of the class are with less data compare to other classes. I used random sampling technique to create a sample from the population keeping ...
3
votes
1answer
165 views

Understanding ROCs in imbalanced data-sets

A response variable (label) $B$ can either be $0$ or $1$. In the training set, $B_i = 1$ is an extremely rare event at only $0.26\%$ occurrences. Which makes the prediction of this label on a test ...
3
votes
1answer
337 views

Online/incremental unsupervised dimensionality reduction for use with classification for event prediction

Consider the application: We have a set of users and items. Users can perform different action types (think browsing, clicking, upvoting etc.) on different items. Users and items accumulate a "...
3
votes
1answer
3k views

Performance issues when merging two dataframe columns into one on millions rows with Pandas

I am trying to merge two address columns into one and separate the resulting string with '--'. The dataset has 10 million rows and 33 columns - but the number of rows grows for a million or so a month....
3
votes
2answers
3k views

R and Python, memory differences yielding performance differences

I think there are numerous posts regarding which one to use: R or Python. However, I'm curious about how their architecture differences yield differences in speed performance, not which one to use. ...
3
votes
0answers
197 views

How much text is enough to train a good embedding model?

I need to train a word2vec embedding model on Wikipedia articles using Gensim. Eventually, I will use the entire Wikipedia for that but for the moment, I'm doing some experimentation/optimization to ...
3
votes
0answers
33 views

Timing sequence in MapReduce

I'm running a test on MapReduce algorithm in different environments, like Hadoop and MongoDB, and using different types of data. What are the different methods or techniques to find out the execution ...
2
votes
4answers
237 views

Neural Network unseen data performance

I started dabbling in neural networks quite recently and encountered a situation which is quite strange (at least with my limited knowledge). The problem I'm using a NN is a regression problem which ...
2
votes
2answers
183 views

ARIMA(X) Validation

I'm in the process of developing a new spark-based ARIMA(X) tool, and have reached the point where I need to know whether my coefficient estimates and forecasts are sensible. I can compare my results ...
2
votes
1answer
51 views

“Each agent was evaluated every 250,000 training frames for 135,000 validation frames” What does this sentences stands for? in DQN nature paper?

In nature paper of DQN by DeepMind, DQN is compared to linear function but they does not said what is this linear function? They compared with some linear functions? 0- What is the meaning of this ...
2
votes
1answer
291 views

how to improve searching index in dataframe

Given a pandas dataframe with a timestamp index, sorted. I have a label and I need to find the closest index to that label. Also, I need to find a smaller timestamp, so the search should be computed ...
2
votes
1answer
999 views

Interpreting confusion matrix and validation results in convolutional networks

I need some help in the assessment of the training results of a convolutional neural network. Here is my setup: Architecture: InceptionV3 Pre-trained InceptionV3 with weights from image net ...
2
votes
1answer
79 views

Vectorizing/Parallelizing DataFrame indexing

I want to make the following Python data-processing code more efficient by replacing for loops. Is there any way to vectorize code like this? I have a DataFrame object ...
2
votes
2answers
2k views

Slow convergence with rpart

I'm building a decision tree in R using the rpart function, available in the library of the same name, but am experiencing some serious performance issues when ...
2
votes
1answer
130 views

Case when Out of Bag Error and Test error differs a lot in Random Forest

I'm using random forest and the out of bag error for the level of one class is very different to its test error. I'm working with a cutt-of equal to c(0.2,0.8). Here's the case: ...
2
votes
1answer
78 views

Which open-source sgdb for kind of large data

I have a 7 giga confidential dataset which I want to use for a machine learning application. I tried : Every package recommanded for efficient dataset management in R like : data.table, ff and ...
2
votes
3answers
72 views

Is over-fitting a matter of features engineering or a matter of parameters set?

I am using sklearn package to make models. I tried randomly to set some paramater to a sklearn.ensemble.RandomForestClassifier ...
2
votes
1answer
31 views

Relating ROC curves with class statistics

I have three neural net models that I am running on the same dataset (of 7 classes) and calculate their class performance and also ROC curves. The firs tmodel is a 4-layer model with 8 neurons in each ...
2
votes
1answer
95 views

Xgboost multiple class predictive performance beats one versus rest

I have an NLP task I'm tackling with xgboost (R implementation). Before describing my doubt I'll give you some background: I have a corpus of documents for which I did topic discovery, using a term ...
2
votes
1answer
195 views

why does performance of machine learning models plateau after certain amount of training data

I'm referring to the below image which I came across. The explanation seemed intuitive at first but I don't think I understand how it works. The image says that the performance of traditional machine ...
2
votes
0answers
151 views

How to choose best model checkpoint when training deep learning model on all the data?

When training a final model for production, it's often recommended to train on all available data (train + dev + test), as discussed here. I'm training a deep learning model. I typically save and use ...