2
Stacked LSTM is a special version of hierachical recurrent neural networks, where hard-wired memory and gating units help long-term preservation of state information.
Hierarchy and recurrence have been explored in many works.
One early example is the Neural Abstraction Pyramid, which introduced recurrent computation to hierarchical convolutional neural ...
2
The simple option is to design your features so that they represent the distribution of the values: every feature $f_i$ represents a bin and its value for a particular instance is the frequency of the corresponding range for this instance.
Example: let's consider 10 bins between 0 and 1, i.e. $f_1=[0,0.1), f_2=[0.1,0.2),..., f_{10}=[0.9,1]$:
$x_1=[0.2, 0.25,...
2
Most deep learning frameworks have APIs that are significantly similar to NumPy. I recommend you take a look at PyTorch as it will let you refactor your code reasonably intuitively to make use of your GPU via Cuda. Speaking as someone who has coded a neural network in NumPy, I would highly recommend learning a popular deep learning framework. It will be ...
1
In fact these involve different aspects of parameters in a CNN.
Parameter sharing, means one parameter may be shared by more than one input/connection. So this reduces total amount of independent parameters. Parameters shared are non-zero.
Sparsity of connections means that some parameters are simply missing (ie are zero), nothing to do with sharing same ...
1
Have you looked into focal loss?
The idea seems to be similar to what you are describing - If predictions (~0.8) is close to GT Label of 1, it does not add to the loss value.
1
Thanks for updating the post, this level of fluctuation in the validation set is a lot less dramatic than before and appears to be similar to regular fluctuation I have seen in my experience. Kudos that you have also managed to prevent the model from overfitting.
1
X[train].shape[0] - This is the number of instances. Let's say it is M
X[train].shape[1] - This is the shape of each instance. Each instance is (1 x N)
Since input instances are of 1-D, the input data become m x N. Had it been 2-D, it would have been m x Nx x Ny
And one more question, I know that CNN required fixed input size. But I split my data into ...
1
I would not say there is such a convention for it per se (if anyone has anything to comment on this, I would also like to know).
I think to make it clearer how the layer's input x interacts with the weights W, it might better to define the dimensions as the following:
x: (m x n)
W: (n x k)
bias term b: (k)
m remains as the number of examples. n represents ...
Only top voted, non community-wiki answers of a minimum length are eligible
