Data Science Stack Exchange Community Digest

Top new questions this week:

Python (S)ARIMA models completely wrong

I have some time series, like this one: I want to predict future values, so I splitted in train/test (70/30) and I created several ARIMA models, however they are all completely wrong (or maybe I am ...

python time-series arima

asked by CasellaJr Score of 3

answered by bayes2021 Score of 5

Is there a way to map words to their synonyms in tfidf?

I have the following code: ...

nlp scikit-learn nltk tfidf spacy

asked by james pow Score of 2

answered by Andy Score of 4

Discussion about modern deep learning training strategies

Previously I have put a lot of effort into training networks appropriately. However, talking to colleagues, a lot of the things I did may be redundant due to novel optimizers and the theory of deep ...

deep-learning training optimization overfitting

asked by Zador Pataki Score of 2

Stacking: Use predictions of train or test to create features for level 1 classifier

The question is pretty simple. In stacking, the predictions of level 0 models are being used as features to train a level 1 model. However, the predictions of what data? Intuitively it makes more ...

machine-learning classification data-leakage stacking

asked by liakoyras Score of 2

answered by Erwan Score of 4

Finding useful noun 2-grams?

Q: How can I find noun 2-grams in the English language (e.g., "roller coaster", "test tube")? Better yet, how can I find them with proportions? Ultimate goal: Generate distinct ...

text-mining

asked by lowndrul Score of 2

answered by Brian Spiering Score of 2

Question about non linearity of activation function

I have a basic question about activation functions. It is told that they are added to the network to introduce non linearity. However, the neural network itself is non linear. Isn' it? If we see any ...

deep-learning neural-network activation-function

asked by Sandeep Bhutani Score of 2

answered by Dave Score of 1

Can you do the math for this simple treeSHAP example (decisionTree)?

I've been trying to understand the math behind shap values. So far I understand all the concepts in SHAP but not to get to the shap values that are in this example (coming from the last example of ...

machine-learning python shap

asked by Tom Score of 1

answered by Ben Reiniger Score of 0

Greatest hits from previous weeks:

Dataframe has no column names. How to add a header?

I am using a dataset to practice for building a decision tree classifier. Here is my code: ...

python pandas

asked by user633599 Score of 17

answered by Gyan Ranjan Score of 18

Keras difference beetween val_loss and loss during training

What is the difference between val_loss and loss during training in Keras? E.g. ...

machine-learning deep-learning keras

asked by Vladimir Shebuniayeu Score of 36

answered by Green Falcon Score of 25

When is precision more important over recall?

Can anyone give me some examples where precision is important and some examples where recall is important?

machine-learning evaluation

asked by Rajat Score of 41

answered by SmallChess Score of 63

What is the difference between Gradient Descent and Stochastic Gradient Descent?

What is the difference between Gradient Descent and Stochastic Gradient Descent? I am not very familiar with these, can you describe the difference with a short example?

machine-learning neural-network deep-learning gradient-descent

asked by Developer Score of 72

answered by Sociopath Score of 80

Calculation and Visualization of Correlation Matrix with Pandas

I have a pandas data frame with several entries, and I want to calculate the correlation between the income of some type of stores. There are a number of stores with income data, classification of ...

python statistics visualization pandas

asked by gdlm Score of 38

answered by AN6U5 Score of 28

Choosing a learning rate

I'm currently working on implementing Stochastic Gradient Descent, SGD, for neural nets using back-propagation, and while I understand its purpose I have some ...

machine-learning neural-network deep-learning optimization hyperparameter

asked by ragingSloth Score of 113

answered by indico Score of 85

Opening a 20GB file for analysis with pandas

I am currently trying to open a file with pandas and python for machine learning purposes it would be ideal for me to have them all in a DataFrame. Now The file is 18GB large and my RAM is 32 GB but I ...

python bigdata pandas anaconda

asked by Hari Prasad Score of 46

answered by Olel Daniel Score of 40

Can you answer these questions?

Filter relevant geojson data based on a selected datapoint

I'll outline the situation and challenge I have a bit more clearly. I have multiple different datasets: A set of locations where products are sold (name, long, lat) A geojson dataset with geometry (...

geospatial tableau

asked by Lars Kuijlenburg Score of 1

What is the best way for me to classify this audio data?

I have a set of audio data. I would like to classify each audio file based on a half-second of data from a give time period. The audio data is given as counts as a function of time $s(t)$. Right now ...

classification dataset machine-learning-model preprocessing model-selection

asked by user3517167 Score of 1

Why is max_features ordered by term frequency instead of inverse document frequency

In the docs: https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html it is explained that max_features is ordered by ...

nlp scikit-learn tfidf

asked by james pow Score of 1

answered by Erwan Score of 0