238
votes
Accepted
Train/Test/Validation Set Splitting in Sklearn
You could just use sklearn.model_selection.train_test_split twice. First to split to train, test and then split train again into validation and train. Something ...
- 2,732
144
votes
How to draw Deep learning network architecture diagrams?
I recently found this online tool that produces publication-ready NN-architecture schematics. It is called NN-SVG and made by Alex Lenail.
You can easily export these to use in, say, LaTeX for ...
- 1,541
99
votes
Accepted
Cross-entropy loss explanation
The cross entropy formula takes in two distributions, $p(x)$, the true distribution, and $q(x)$, the estimated distribution, defined over the discrete variable $x$ and is given by
$$H(p,q) = -\sum_{\...
- 28.9k
92
votes
How do you visualize neural network architectures?
I recently created a tool for drawing NN architectures and exporting SVG, called NN-SVG
- 1,021
90
votes
How to draw Deep learning network architecture diagrams?
I wrote some latex code to draw Deep networks for one of my reports. You can find it here: https://github.com/HarisIqbal88/PlotNeuralNet
With this, you can draw networks like these:
- 909
87
votes
What is the difference between Gradient Descent and Stochastic Gradient Descent?
For a quick simple explanation:
In both gradient descent (GD) and stochastic gradient descent (SGD), you update a set of parameters in an iterative manner to minimize an error function.
While in GD, ...
- 1,243
81
votes
Train/Test/Validation Set Splitting in Sklearn
There is a great answer to this question over on SO that uses numpy and pandas.
The command (see the answer for the discussion):
...
- 955
79
votes
Accepted
In softmax classifier, why use exp function to do normalization?
It is more than just numerical. A quick reminder of the softmax:
$$
P(y=j | x) = \frac{e^{x_j}}{\sum_{k=1}^K e^{x_k}}
$$
Where $x$ is an input vector with length equal to the number of classes $K$. ...
- 916
79
votes
When should I use Gini Impurity as opposed to Information Gain (Entropy)?
Gini impurity and Information Gain Entropy are pretty much the same. And people do use the values interchangeably. Below are the formulae of both:
$\textit{Gini}: \mathit{Gini}(E) = 1 - \sum_{j=1}^{c}...
- 8,296
77
votes
Accepted
In supervised learning, why is it bad to have correlated features?
Correlated features in general don't improve models (although it depends on the specifics of the problem like the number of variables and the degree of correlation), but they affect specific models in ...
- 1,267
73
votes
Accepted
What does Logits in machine learning mean?
Logits interpreted to be the unnormalised (or not-yet normalised) predictions (or outputs) of a model. These can give results, but we don't normally stop with logits, because interpreting their raw ...
- 14.9k
73
votes
Accepted
Why mini batch size is better than one single "batch" with all training data?
The key advantage of using minibatch as opposed to the full dataset goes back to the fundamental idea of stochastic gradient descent1.
In batch gradient descent, you compute the gradient over the ...
- 1,360
73
votes
Accepted
Why do cost functions use the square error?
Your loss function would not work because it incentivizes setting $\theta_1$ to any finite value and $\theta_0$ to $-\infty$.
Let's call $r(x,y)=\frac{1}{m}\sum_{i=1}^m {h_\theta\left(x^{(i)}\right)} ...
- 1,051
70
votes
Accepted
Should we apply normalization to test data as well?
Yes you need to apply normalisation to test data, if your algorithm works with or needs normalised training data*.
That is because your model works on the representation given by its input vectors. ...
- 28.9k
67
votes
Accepted
GBM vs XGBOOST? Key differences?
Quote from the author of xgboost:
Both xgboost and gbm follows the principle of gradient boosting. There are however, the difference in modeling details. ...
- 4,326
66
votes
Accepted
When is precision more important over recall?
For rare cancer data modeling, anything that doesn't account for false-negatives is a crime. Recall is a better measure than precision.
For YouTube recommendations, false-negatives is less of a ...
- 3,540
65
votes
Accepted
What does from_logits=True do in SparseCategoricalcrossEntropy loss function?
The from_logits=True attribute inform the loss function that the output values generated by the model are not normalized, a.k.a. logits. In other words, the softmax ...
- 854
65
votes
Data scientist vs machine learning engineer
Good question. Actually there is a lot of confusion on this subject, mainly because both are quite new jobs. But if we focus on the semantics, the real meaning of the jobs become clear.
Beforehand is ...
- 1,774
64
votes
Accepted
How to get accuracy, F1, precision and recall, for a keras model?
Metrics have been removed from Keras core. You need to calculate them manually. They removed them on 2.0 version. Those metrics are all global metrics, but Keras works in batches. As a result, it ...
- 3,920
59
votes
Accepted
What is the advantage of keeping batch size a power of 2?
This is a problem of alignment of the virtual processors (VP) onto the physical processors (PP) of the GPU. Since the number of PP is often a power of 2, using a number of VP different from a power of ...
- 706
59
votes
Adding Features To Time Series Model LSTM
For RNNs (e.g., LSTMs and GRUs), the layer input is a list of timesteps, and each timestep is a feature tensor. That means that you could have a input tensor like this (in Pythonic notation):
...
- 1,106
58
votes
What is the Q function and what is the V function in reinforcement learning?
$V^\pi(s)$ is the "state" value function of an MDP (Markov Decision Process). It's the expected return starting from state $s$ following policy $\pi$:
$$V^\pi(s) = E_{\pi} \{G_t \vert s_t = ...
- 907
58
votes
Accepted
Why are Machine Learning models called black boxes?
The black box thing has nothing to do with the level of expertise of the audience (as long as the audience is human), but with the explainability of the function modelled by the machine learning ...
- 26.6k
57
votes
Accepted
How to set batch_size, steps_per epoch, and validation steps?
batch_size determines the number of samples in each mini batch. Its maximum is the number of all samples, which makes gradient descent accurate, the loss will decrease towards the minimum if the ...
- 761
57
votes
Accepted
Why is ReLU used as an activation function?
In mathematics (linear algebra) a function is considered linear whenever a function$f: A \rightarrow B$ if for every $x$ and $y$ in the domain $A$ has the following property: $f(x) + f(y) = f(x+y)$. ...
- 2,420
57
votes
Accepted
How to interpret the output of XGBoost importance?
From your question, I'm assuming that you're using xgboost to fit boosted trees for binary classification. The importance matrix is actually a data.table object with the first column listing the names ...
- 2,537
53
votes
Accepted
The difference between `Dense` and `TimeDistributedDense` of `Keras`
Let's say you have time-series data with $N$ rows and $700$ columns which you want to feed to a SimpleRNN(200, return_sequence=True) layer in Keras. Before you feed ...
- 2,206
52
votes
Accepted
Why do we need XGBoost and Random Forest?
It's easier to start with your second question and then go to the first.
Bagging
Random Forest is a bagging algorithm. It reduces variance.
Say that you have very unreliable models, such as ...
- 3,420
52
votes
How do you visualize neural network architectures?
Tensorflow, Keras, MXNet, PyTorch
If the neural network is given as a Tensorflow graph, then you can visualize this graph with TensorBoard.
Here is how the MNIST CNN looks like:
You can add names / ...
- 18.9k
50
votes
Accepted
Why should the data be shuffled for machine learning tasks
Based on What should we do when a question posted on DataScience is a duplicate of a question posted on CrossValidated?, I am reposting my answer to the same question asked on CrossValidated (https://...
- 623
Only top scored, non community-wiki answers of a minimum length are eligible