Questions tagged [machine-learning]
Machine Learning is a subfield of computer science that draws on elements from algorithmic analysis, computational statistics, mathematics, optimization, etc. It is mainly concerned with the use of data to construct models that have high predictive/forecasting ability. Topics include modeling building, applications, theory, etc.
11,497
questions
202
votes
18
answers
430k
views
Train/Test/Validation Set Splitting in Sklearn
How could I randomly split a data matrix and the corresponding label vector into a X_train, X_test, ...
- 8,687
199
votes
6
answers
157k
views
What is the "dying ReLU" problem in neural networks?
Referring to the Stanford course notes on Convolutional Neural Networks for Visual Recognition, a paragraph says:
"Unfortunately, ReLU units can be fragile during training and can
"die". For ...
- 4,095
198
votes
6
answers
377k
views
How to draw Deep learning network architecture diagrams?
I have built my model. Now I want to draw the network architecture diagram for my research paper. Example is shown below:
- 2,487
182
votes
21
answers
262k
views
How do you visualize neural network architectures?
When writing a paper / making a presentation about a topic which is about neural networks, one usually visualizes the networks architecture.
What are good / simple ways to visualize common ...
- 19.1k
152
votes
6
answers
166k
views
The cross-entropy error function in neural networks
In the MNIST For ML Beginners they define cross-entropy as
$$H_{y'} (y) := - \sum_{i} y_{i}' \log (y_i)$$
$y_i$ is the predicted probability value for class $i$ and $y_i'$ is the true probability ...
- 19.1k
150
votes
17
answers
126k
views
Best python library for neural networks
I'm using Neural Networks to solve different Machine learning problems. I'm using Python and pybrain but this library is almost discontinued. Are there other good alternatives in Python?
127
votes
14
answers
121k
views
Python vs R for machine learning
I'm just starting to develop a machine learning application for academic purposes. I'm currently using R and training myself in it. However, in a lot of places, I have seen people using Python.
What ...
115
votes
11
answers
128k
views
Choosing a learning rate
I'm currently working on implementing Stochastic Gradient Descent, SGD, for neural nets using back-propagation, and while I understand its purpose I have some ...
- 1,834
115
votes
5
answers
71k
views
Why do cost functions use the square error?
I'm just getting started with some machine learning, and until now I have been dealing with linear regression over one variable.
I have learnt that there is a hypothesis, which is:
$h_\theta(x)=\...
- 1,323
112
votes
9
answers
140k
views
When should I use Gini Impurity as opposed to Information Gain (Entropy)?
Can someone practically explain the rationale behind Gini impurity vs Information gain (based on Entropy)?
Which metric is better to use in different scenarios while using decision trees?
- 1,221
99
votes
4
answers
106k
views
Advantages of AUC vs standard accuracy
I was starting to look into area under curve(AUC) and am a little confused about its usefulness. When first explained to me, AUC seemed to be a great measure of performance but in my research I've ...
- 1,093
91
votes
7
answers
125k
views
In supervised learning, why is it bad to have correlated features?
I read somewhere that if we have features that are too correlated, we have to remove one, as this may worsen the model. It is clear that correlated features means that they bring the same information, ...
- 1,289
87
votes
6
answers
154k
views
strings as features in decision tree/random forest
I am doing some problems on an application of decision tree/random forest. I am trying to fit a problem which has numbers as well as strings (such as country name) as features. Now the library, scikit-...
- 1,005
83
votes
5
answers
127k
views
GBM vs XGBOOST? Key differences?
I am trying to understand the key differences between GBM and XGBOOST. I tried to google it, but could not find any good answers explaining the differences between the two algorithms and why xgboost ...
- 997
81
votes
9
answers
32k
views
Data scientist vs machine learning engineer
What are the differences, if any, between a "data scientist" and a "machine learning engineer"?
Over the past year or so "machine learning engineer" has started to show up a lot in job postings. ...
- 4,169
78
votes
7
answers
167k
views
What is the difference between Gradient Descent and Stochastic Gradient Descent?
What is the difference between Gradient Descent and Stochastic Gradient Descent?
I am not very familiar with these, can you describe the difference with a short example?
- 1,099
76
votes
6
answers
153k
views
Cross-entropy loss explanation
Suppose I build a neural network for classification. The last layer is a dense layer with Softmax activation. I have five different classes to classify. Suppose for a single training example, the <...
- 3,051
73
votes
7
answers
83k
views
Open source Anomaly Detection in Python
Problem Background:
I am working on a project that involves log files similar to those found in the IT monitoring space (to my best understanding of IT space). These log files are time-series data, ...
- 933
71
votes
5
answers
53k
views
Adding Features To Time Series Model LSTM
have been reading up a bit on LSTM's and their use for time series and its been interesting but difficult at the same time. One thing I have had difficulties with understanding is the approach to ...
- 1,235
69
votes
11
answers
104k
views
Why should the data be shuffled for machine learning tasks
In machine learning tasks it is common to shuffle data and normalize it. The purpose of normalization is clear (for having same range of feature values). But, after struggling a lot, I did not find ...
- 14.1k
69
votes
5
answers
79k
views
Why mini batch size is better than one single "batch" with all training data?
I often read that in case of Deep Learning models the usual practice is to apply mini batches (generally a small one, 32/64) over several training epochs. I cannot really fathom the reason behind this....
- 8,687
68
votes
6
answers
86k
views
What is the Q function and what is the V function in reinforcement learning?
It seems to me that the $V$ function can be easily expressed by the $Q$ function and thus the $V$ function seems to be superfluous to me. However, I'm new to reinforcement learning so I guess I got ...
- 19.1k
68
votes
5
answers
32k
views
In softmax classifier, why use exp function to do normalization?
Why use softmax as opposed to standard normalization? In the comment area of the top answer of this question, @Kilian Batzner raised 2 questions which also confuse me a lot. It seems no one gives an ...
- 783
67
votes
9
answers
107k
views
Clustering geo location coordinates (lat,long pairs)
What is the right approach and clustering algorithm for geolocation clustering?
I'm using the following code to cluster geolocation coordinates:
...
- 813
65
votes
5
answers
204k
views
How to get accuracy, F1, precision and recall, for a keras model?
I want to compute the precision, recall and F1-score for my binary KerasClassifier model, but don't find any solution.
Here's my actual code:
...
- 1,057
63
votes
10
answers
68k
views
Machine learning - features engineering from date/time data
What are the common/best practices to handle time data for machine learning application?
For example, if in data set there is a column with timestamp of event, such as "2014-05-05", how you can ...
- 1,071
63
votes
6
answers
67k
views
Should a model be re-trained if new observations are available?
So, I have not been able to find any literature on this subject but it seems like something worth giving a thought:
What are the best practices in model training and optimization if new observations ...
- 1,783
62
votes
4
answers
54k
views
Difference between OrdinalEncoder and LabelEncoder
I was going through the official documentation of scikit-learn learn after going through a book on ML and came across the following thing:
In the Documentation it is given about ...
- 763
62
votes
5
answers
74k
views
RNN vs CNN at a high level
I've been thinking about the Recurrent Neural Networks (RNN) and their varieties and Convolutional Neural Networks (CNN) and their varieties.
Would these two points be fair to say:
Use CNNs to break ...
- 745
61
votes
9
answers
18k
views
Is there any domain where Bayesian Networks outperform neural networks?
Neural networks get top results in Computer Vision tasks (see MNIST, ILSVRC, Kaggle Galaxy Challenge). They seem to outperform every other approach in Computer Vision. But there are also other tasks:
...
- 19.1k
61
votes
3
answers
62k
views
LeakyReLU vs PReLU
I thought both, PReLU and Leaky ReLU are:
$$f(x) = \max(x, \alpha x) \qquad \text{ with } \alpha \in (0, 1)$$
Keras, however, has both functions in the docs.
Leaky ReLU
Source of LeakyReLU:
...
- 19.1k
60
votes
5
answers
54k
views
Neural networks: which cost function to use?
I am using TensorFlow for experiments mainly with neural networks. Although I have done quite some experiments (XOR-Problem, MNIST, some Regression stuff, ...) now, I struggle with choosing the "...
- 723
59
votes
4
answers
45k
views
What is the advantage of keeping batch size a power of 2?
While training models in machine learning, why is it sometimes advantageous to keep the batch size to a power of 2? I thought it would be best to use a size that is the largest fit in your GPU memory /...
- 1,215
59
votes
6
answers
16k
views
Should I go for a 'balanced' dataset or a 'representative' dataset?
My 'machine learning' task is of separating benign Internet traffic from malicious traffic. In the real world scenario, most (say 90% or more) of Internet traffic is benign. Thus I felt that I should ...
- 693
59
votes
3
answers
115k
views
How to set batch_size, steps_per epoch, and validation steps?
I am starting to learn CNNs using Keras. I am using the theano backend.
I don't understand how to set values to:
batch_size
...
- 693
58
votes
5
answers
35k
views
Is it always better to use the whole dataset to train the final model?
A common technique after training, validating and testing the Machine Learning model of preference is to use the complete dataset, including the testing subset, to train a final model to deploy it on, ...
- 3,970
57
votes
3
answers
63k
views
What does Logits in machine learning mean?
"One common mistake that I would make is adding a non-linearity to my logits output."
What does the term "logit" means here or what does it represent ?
- 1,137
57
votes
8
answers
18k
views
Why Is Overfitting Bad in Machine Learning?
Logic often states that by overfitting a model, its capacity to generalize is limited, though this might only mean that overfitting stops a model from improving after a certain complexity. Does ...
- 1,932
57
votes
2
answers
70k
views
How to interpret the output of XGBoost importance?
I ran a xgboost model. I don't exactly know how to interpret the output of xgb.importance.
What is the meaning of Gain, Cover, and Frequency and how do we ...
user14204
51
votes
7
answers
46k
views
Deep Learning vs gradient boosting: When to use what?
I have a big data problem with a large dataset (take for example 50 million rows and 200 columns). The dataset consists of about 100 numerical columns and 100 categorical columns and a response column ...
- 1,615
51
votes
3
answers
71k
views
StandardScaler before or after splitting data - which is better?
When I was reading about using StandardScaler, most of the recommendations were saying that you should use StandardScaler before ...
- 725
49
votes
7
answers
57k
views
What is the difference between model hyperparameters and model parameters?
I have noticed that such terms as model hyperparameter and model parameter have been used interchangeably on the web without prior clarification. I think this is incorrect and needs explanation. ...
- 2,157
49
votes
4
answers
34k
views
Early stopping on validation loss or on accuracy?
I am currently training a neural network and I cannot decide which to use to implement my Early Stopping criteria: validation loss or a metrics like accuracy/f1score/auc/whatever calculated on the ...
- 1,309
48
votes
4
answers
34k
views
Why is ReLU used as an activation function?
Activation functions are used to introduce non-linearities in the linear output of the type w * x + b in a neural network.
Which I am able to understand ...
- 603
48
votes
3
answers
105k
views
What is Ground Truth
In the context of Machine Learning, I have seen the term Ground Truth used a lot. I have searched a lot and found the following definition in Wikipedia:
In machine learning, the term "ground ...
- 14.1k
47
votes
5
answers
70k
views
Does gradient descent always converge to an optimum?
I am wondering whether there is any scenario in which gradient descent does not converge to a minimum.
I am aware that gradient descent is not always guaranteed to converge to a global optimum. I am ...
- 573
46
votes
10
answers
95k
views
When is precision more important over recall?
Can anyone give me some examples where precision is important and some examples where recall is important?
- 1,137
46
votes
12
answers
55k
views
Data Science in C (or C++)
I'm an R language programmer. I'm also in the group of people who are considered Data Scientists but who come from academic disciplines other than CS.
This works ...
- 1,919
46
votes
3
answers
54k
views
What does from_logits=True do in SparseCategoricalcrossEntropy loss function?
In the documentation it has been mentioned that y_pred needs to be in the range of [-inf to inf] when from_logits=True. I truly ...
- 573
46
votes
2
answers
75k
views
Merging two different models in Keras
I am trying to merge two Keras models into a single model and I am unable to accomplish this.
For example in the attached Figure, I would like to fetch the middle layer $A2$ of dimension 8, and use ...
- 1,033