Community Digest

Top new questions this week:

Why removing rows with NA values from the majority class improves model performance

I have an imbalanced dataset like so: df['y'].value_counts(normalize=True) * 100 ...

python pandas class-imbalance missing-data smote  
asked by sums22 4 votes
answered by Erwan 2 votes

Machine Learning with intended missing values

I have a dataset relating to humans completing reviews, the target variable is whether the review decision is correct / incorrect and one of my features is a trailing 4 week accuracy score for the ...

machine-learning statistics feature-engineering  
asked by Stats DUB01 4 votes
answered by Erwan 2 votes

Remove outliers from a noisy curve

Allow me to present some images so that I may explain my problem. The images on the left contain a smooth curve surrounded by lots of outliers/noise. The image on the right depict the desired curve. ...

image-preprocessing  
asked by Sau001 3 votes
answered by Nikos M. 0 votes

What to replace outliers with? (supermarket transaction data)

I have a transaction dataset from a supermarket. Let's say the average spend is $50. I want to get each customer's average spend and rank them based on where they fall based on this $50 average spend. ...

python pandas statistics outlier  
asked by SCool 2 votes
answered by Pluviophile 0 votes

References/tutorials about data mining and machine learning

I am learning data analytics and I wonder if there are some good references and tutorials about machine learning, data analytics and data mining? What I'm searching for is an understandable reference/...

machine-learning reference-request  
asked by plpm 2 votes
answered by Peter 0 votes

Big Data clustering

I've got some big datasets of images (a few million each), and I would like to cluster them according to images' visual similarities. I've extracted a feature vector for each image; the space of ...

machine-learning clustering bigdata apache-spark scalability  
asked by Overloop 2 votes

When to use which multiple testing correction?

There are a large number multiple testing p-value correction methods. e.g.: ...

hypothesis-testing  
asked by lordy 2 votes

Greatest hits from previous weeks:

What are deconvolutional layers?

I recently read Fully Convolutional Networks for Semantic Segmentation by Jonathan Long, Evan Shelhamer, Trevor Darrell. I don't understand what "deconvolutional layers" do / how they work. The ...

neural-network convnet convolution  
asked by Martin Thoma 240 votes
answered by David Dao 262 votes

What is "experience replay" and what are its benefits?

I've been reading Google's DeepMind Atari paper and I'm trying to understand the concept of "experience replay". Experience replay comes up in a lot of other reinforcement learning papers (...

reinforcement-learning q-learning  
asked by Ryan Zotti 34 votes
answered by Neil Slater 40 votes

How to count the number of missing values in each row in Pandas dataframe?

How can I get the number of missing value in each row in Pandas dataframe. I would like to split dataframe to different dataframes which have same number of missing values in each row. Any suggestion?...

python pandas  
asked by Kaggle 26 votes
answered by Jan van der Vegt 24 votes

Cross-entropy loss explanation

Suppose I build a neural network for classification. The last layer is a dense layer with Softmax activation. I have five different classes to classify. Suppose for a single training example, the <...

machine-learning neural-network deep-learning softmax  
asked by enterML 56 votes
answered by Neil Slater 78 votes

How can I check the correlation between features and target variable?

I am trying to build a Regression model and I am looking for a way to check whether there's any correlation between features and target variables? This is my ...

machine-learning scikit-learn regression linear-regression  
asked by user_6396 20 votes
answered by JahKnows 14 votes

How do you visualize neural network architectures?

When writing a paper / making a presentation about a topic which is about neural networks, one usually visualizes the networks architecture. What are good / simple ways to visualize common ...

machine-learning neural-network deep-learning visualization  
asked by Martin Thoma 124 votes
answered by Alex Lenail 54 votes

How to get correlation between two categorical variable and a categorical variable and continuous variable?

I am building a regression model and I need to calculate the below to check for correlations Correlation between 2 Multi level categorical variables Correlation between a Multi level categorical ...

r statistics correlation  
asked by GeorgeOfTheRF 98 votes
answered by Alexey Grigorev 97 votes

Can you answer these questions?

Liveness Detection System - Suggestions

I am trying to build a liveness detection system but till now I haven't been able to generate any significant results. I read the article by pyimagesearch this on the same topic but no luck. Tried ...

machine-learning computer-vision  
asked by Shaikh Abuzar 1 vote

Keras model with LSTM quantization aware training

I would like to run quantization aware training with a keras model which has an LSTM layer. However, just the LSTM layer seems to not be supported. Alan Chiao seems to suggest here that it is possible ...

keras lstm binary  
asked by Karl Haebler 1 vote

Can elastic net l1 ratio be greater than 1?

I have multiple datasets that I trained with ElasticNetCV (sklearn), and I noticed that many of them selected l1_ratio = 1 as ...

scikit-learn linear-regression regularization elastic-net  
asked by Oren Matar 1 vote
You're receiving this message because you subscribed to the Data Science community digest.
Unsubscribe from this community digest       Edit email settings       Leave feedback       Privacy
Stack Overflow

Stack Overflow, 110 William Street, 28th floor, New York, NY 10038

<3