New answers tagged neural-network
0
votes
Why is online learning 'much faster' than full-batch? - Neural Networks
It depends on the definition of full-batch and online learning. If full-batch is defined as n > 1 and online learning is defined as n = 1, then full-batch will be slower than online. Additional ...
- 16.5k
0
votes
What does this statement relative to neural network weight initialization mean?
If all the weights contribute equally, there is no way to single out some specific weights to penalise during backpropagation. Therefore the change in weights is just a trivial global rescaling.
- 196
0
votes
Accepted
What does this statement relative to neural network weight initialization mean?
Consider the following image of a simple neural network. Note that the network uses a linear activation function and that there are no bias terms (this makes the intuition easier).
Each path from the ...
- 116
0
votes
how to handle soft weight constraints in neural network
Maybe instead of mapping nn weights Wi to your problem, map Constant * Softmax(Wi) instead for each layer?
- 176
5
votes
Accepted
MLOps for beginner
a. For a beginner I would suggest the fullstackdeeplearning course, it's a modern overview of tools and best practices for ML in production. As you can see below, there are a lot of moving pieces.
b. ...
- 196
5
votes
MLOps for beginner
You can do live learning but most models don't require it, because many businesses don't need to learn directly from new input.
Nevertheless, you can apply an automated task every time range (day, ...
- 2,168
0
votes
evaluation of gradient for the subset of parameters using backpropagation
If you are using Tensorflow, you can use the tf.GradientTape function:
tf.GradientTape(
persistent=False, watch_accessed_variables=True
)
For example:
...
- 2,168
0
votes
Newly discovered learning rule
I don't know how this algorithm performs, even with comments, because many functions are connected and use frequent loops, and they include also the output values.
I have to track every step, and it ...
- 2,168
0
votes
Odd error when training neural network with Keras - Error occurred when finalizing GeneratorDataset iterator
If it stops always after 36 epochs, it could be due to the data in input.
You should check if the generated data has always the same dimension and same format, without any outlier (like an empty ...
- 2,168
2
votes
What what will happen if all the layers of a MLP or any DL architecture are set as same in the beginning?
There is no correlation between underfitting/overfitting with the initialization of the weights.
The problem with initialization of the weight is related to 2 main things:
Back propagation: if you ...
0
votes
Small difference in metrics in KERAS for the same model
I found explanation here:
https://github.com/tensorflow/tensorflow/issues/29964
https://stackoverflow.com/questions/59118430/keras-model-evaluate-on-training-and-val-set-differ-from-the-acc-and-val-...
- 1
2
votes
Scalar predictor - is it better to have a lot of training data that is less precise? Or fewer training data that is more precise?
super-interesting question!
My approach to the problem would be not to do any preprocessing on the data. This is, feed all the experiments to the network with the target being the 0/1 variable ...
- 489
1
vote
Scalar predictor - is it better to have a lot of training data that is less precise? Or fewer training data that is more precise?
I would say that it's better to have more data, since noise in the data is reduced by the optimization algorithm when optimizing (and therefore cause no problem in the optimization phase), and you can ...
0
votes
change parameterization to eliminate weight constraints in neural networks
For other people asking this question: In keras / tensorflow I observed way better results using custom constraints instead of a parameterization. When using a parameterization, the weights stayed ...
- 101
1
vote
Building machine learning models whilst penalizing them for complexity
Decision trees have many options for reducing overfitting.
Examples include:
Maximum depth
Minimum node size
Pruning
Another method to reduce overfitting is adding a cost function penalty. L1 or L2 ...
- 16.5k
3
votes
Accepted
How does gradient descent avoid local minimums?
It does not. Gradient descent is not immune to local minima in non-convex function optimization.
Nevertheless, the noise introduced by stochastic gradient descent (SGD) helps escaping local minima. ...
- 15.4k
1
vote
All classification models except neural network giving 100% accuracy
It could be due to a lack of initialization of your neurons.
Did you initialize them randomly?
For instance:
...
- 2,168
3
votes
Accepted
How do researchers actually code novel architectures and layers?
In this particular case, I don't know how are they implementing these complex layers, but in Keras/TensorFlow you can define your own layers by inheriting from ...
- 489
1
vote
Why ReLU is better than the other activation functions
I think normalization issues are minor issues in numerical computations in general and machine learning in particular if you do not encounter significant numerical overflow or underflow.
The expected ...
- 111
1
vote
Accepted
Loss function to prevent estimator bias
Thank you @Nikos M. for your suggestions. I was about to use your post-applied factor but then gave it another try. And found what caused this. It was that the final layer was using a ...
- 31
0
votes
How to deal with different amounts of data every day?
One common option is to aggregate the embeddings of all the headlines. For example, you can compute the average of the embeddings and use the resulting 300-dim vector as input of the model.
- 489
Top 50 recent answers are included