New answers tagged deep-learning
0
You have to use 1-D CNN.
Before that you must prepare your dataset in a sequential format. e.g.
Below is a depiction of how the Convolution will work [Source - D2L],
Study these references for Conv1D:
Machine learning mastery
Dive into Deep Learning
0
Convolutional 1D layers require their input in 3 dimensions:
[batch], [features], [channels]. This makes good sense for 2D Convolutional layers, which are often called upon to process color images, but it can be a little confusing for 1D convolutions.
The simplest solution may be to reshape your data so that it's, ([samples], 6, 1).
Edit: how to reshape your ...
0
I might not get what you meant by that. I have attached the arhitecture of the model which i am trying to do and the input shape which is my input variables.
1
Sliding window in this context is regarding what is given as input to the CNN. It is a sliding window of the input image. I have seen it being used in medical domain where the images are too large to fit into a network and reshaping them into smaller sizes doesn't help. So, a sliding window is done on the bigger image and the sliding window is fed into CNN ...
0
Each sample of your dataframe is a 1D vector, you need to Conv1D (1D convolutions) with filter sizes of 3 or 5.
If you want to convert each sample of yours into an image, you need to do some pre-processing. Here's a paper which does that: paper.
Also, since this a tabular data, if you want to stick to neural networks then consider TabNet. It has ...
0
This is an anomaly detection problem. In your case, we would refer to it as supervised anomaly detection problem as you have the labels of categories.
This typically involves taking a large "normal" dataset, in this case, this would be receipts which are valid. And then using a machine learning method to learn features from this dataset (e.g. the ...
0
You can do structured pruning for MLPs. You would remove things like whole neurons or layers (not specific weights).
0
You can use tf.image.resize, as follows:
(x_train, y_train), (_, _) = tf.keras.datasets.mnist.load_data()
print(x_trian.shape) # (60000, 28, 28)
# train set / data
x_train = np.expand_dims(x_train, axis=-1)
x_train = tf.image.resize(x_train, [32,32]) # if we want to resize
print(x_train.shape) # (60000, 32, 32, 1)
2
For the latest practices of deep learning architectures, I follow the Kaggle latest Competition's notebooks.
Say, for example, In the recent finished Cassava Leaf Deasese Computer Vision Competition, People are sharing the experimental notebooks on different State of the art Architectures Like Vision Transformer(Various versions pretrained - ...
1
Policies found by Deep Q-Learning, even after convergence, are not guaranteed to be optimal. The reason is that the neural networks that approximate the Q function in DQN inherently come with a statistical error (bias and variance), a pointer can be found here.
Furthermore, convergence to the optimal policy for tabular Q-learning is only guaranteed when ...
0
What you are describing does not require a change in the deep learning models you would use but a change in how to estimate them: you need an online estimation procedure rather than an off-line one. You can have a look at the seminal work of Botou [1998], or Chapter 8 of the book by Goodfellow et al. (2016).
0
Given you already have the tf.data.Dataset, one way to do it would be to iterate over the dataset and each time you come across a new label, save that e.g. to a dictionary, otherwise skip an already seen label.
Here is a short example just using the MNIST dataset that comes with tensorflow:
import matplotlib.pyplot as plt
import tensorflow as tf
(x_train, ...
0
According to the Cambridge Dictionary saturation means
the act or result of filling a thing or place completely so that no
more can be added
In this context, it refers to a function for which a bigger input will not lead to a relevant increase in output. So if the gradient is saturated (meaning it is extremely close to zero), a bigger upstream gradient ...
0
Well, yes there are. One of the applications of GANs is image inpainting which is widely used in Photoshop-like applications. You can omit an obstacle that prohibits you to see the entire image and use GANs to see, generate, the original object without any other disturbing objects that may not let you see the desired object entirely.
0
Your data is a good candidate for a mixed effect model.
You have two potential random effects that look to be crossed: I see ID as one and state as another, as it seems that any ID can belong to any state.
Essentially, you can leverage the fact that some states and some IDs will have less data than others (and in general, some states and IDs will vary more ...
1
In various ways suchs as
momentum: think of momentum as a means of dampening oscillations and speeding up the iterations, leading to faster convergence. This means replacing gradients with a leaky average over past gradients.
sparse features and preconditioning (Adagrad): decrease the learning rate dynamically on a per-coordinate basis. This means, use the ...
0
This might help if what you're asking is related to merging models:
Merging two different models in Keras
0
Most Imagenet pretrained CNNs were trained on 224x224 image resolution. It is a common misconception, that when using these pretrained CNN, images need to be resized to 224x224. On the contrary, popular CNN are fully convolutional nets that can accept any input size.
You can input any image size and these CNN output feature maps that are 32x times smaller. ...
0
There was a Contest in Kaggle - SIIM-ISIC Melanoma Classification. In this contest also tasks for participants to classify two class: malignant & benign.
But in the dataset, there were the same images in both the class. And this produces the Dataleakage.
Dataleakage, also can produce in the same class images both in the training and in the validation ...
1
For starters, it depends on why you're trying to create Document Embeddings.
There's the TF-IDF algorithm; it's simple to grasp and implement, and it facilitates using cosine distance as a metric. To train in parallel, you could probably feed both documents to the model and have a synchronized counter doing the calculations such as Term Frequency and Inverse ...
0
Having N encoders would multiply the number of parameters of the encoder side N times, which would lead to different learning dynamics, so the results would probably not the same, and you may incur in overfitting. I don't think there is a case where that would make more sense than having a single encoder.
1
If the image will always be the same and your ROI will always be in the same location, then an RNN with CV2 is not necessary, you can mark the areas that you need to extract the texts. I even recommend you more like JaidedAI
/
EasyOCR.
Here is the google colab of the papers you sent: Information extraction
1
Why is potentially hidden in your feature explanations.
Given that your Features give some indication on why, like Patient demographics data, or some other Features that you can include here, you can use them to answer the why.
Like this using shapley and or eli5 you can integrate it with your Standard predictor classes and for the explanations with user ...
0
Might your CNN architecture is not suitable for this data, but just slight modification can be better. You need to try other CNNs, vary parameters, and try.
1
I understand that you are doing only inference, so you shouldn't need your model to be trainable, and therefore you can set the parameter trainable to False when invoking hub.Module.
1
In multiclass problem use softmax activation function.
For example, in Keras you put 3 neurons:
model.add(Dense(3, activation='softmax'))
As a loss function you can choose Categorical Crossentropy:
loss=tf.keras.losses.CategoricalCrossentropy()
So that you compile model:
model.compile(loss=tf.keras.losses.CategoricalCrossentropy(),
optimizer=...
2
Binary cross-entropy is only a suitable loss function if you are performing binary (two-class) classification problems. If you add a third "neutral" class, it's no longer appropriate. There are two ways you could frame your problem:
Multi-class classification. In that case, the suggestion to use an output layer with three neurons and softmax ...
4
If you’re going to have more than two labels, you need to go with a softmax activation and a loss for multi class classification, ie cross entropy loss.
Also, be cautious for multi-class versus multi-label (below).
Multi-class
One-of-many classification. Each sample can belong to ONE of $C$ classes. The model will have $C$ output neurons that can be ...
0
The solution for my problem was implementing Batch Renormalization: BatchNormalization(renorm=True). In addition normalizing the inputs helped a lot improving the overall performance of the neural network.
0
These numbers refer to minibatches, not individual samples. The data is not fed to the model one by one, but in small groups called "minibatches" or simply "batches". The size of the minibatch (the number of elements to be included in each minibatch) can be specified as a parameter to the fit method. As you did not provide any value, it ...
2
Disclaimer: i am one of the authors of the referenced link.
If you are doing pure image classification/segmentation, I would say no.
I would not store the output of convolution layers, noramlly. In principle, the output of models can be stored in the feature store, however.
Feature stores store data in tabular file formats (like parquet) and are used to ...
0
Line search may help with exploding/vanishing gradient problem. However, line search does not work well with mini-batches and most training uses mini-batches. One of the main advantages of line search is that it tells you whether you have stepped too far. That advantage disappears when you subsample the data.
0
I've finally found a way to forecast values based on predicted values from the earlier observations. As expected, the predictions were rather accurate in the short-term, slightly becoming worse in the long term. It is not so surprising that the future predictions digress over time, as they no longer depend on the actual values. Reflecting on my results and ...
0
This is because the calculation of the loss and accuracy are done before the first weight update (i.e. the model with the initialized parameters). After the loss is calculated the first time the loss is used to backpropagate the error throughout the network and to update the parameters. The loss is not calculated again during training (since this would just ...
2
The teacher/student model approach (at least as I understand it) could be used. It is normally used to replace all layers, but there is nothing stopping you applying it to a subset of layers.
First you create training data, by running your training data through the (trained) network, and recording the inputs to layer i, as your new training data (x) and the ...
1
The choice for the number of neurons in the last two dense layers and the number of filter is somewhat arbitrary and most of the time determined by trying different configurations (using something like a hyperparameter grid search). See also this answer on stats stackexchange. If you want to change the size of the 5x5 kernels you will only have to change the ...
1
All these parameters are trainable.
Note that in normal Transformers it is typical to have fixed (non-trainable) positional embeddings, but in BERT they are learned.
Note also the "pooler" component, which is an extra projection that was not mentioned in the paper, but which the authors commented on later.
1
I think that the "neurons" analogy is not very helpful to understand what is going on with artificial neural networks.
Neural networks are not comprised by "neurons", but by differentiable operations. These operations are arbitrary, e.g. convolutions, indexing (in embeddings), pooling, etc.
What you proposed is a perfectly valid building ...
0
There can be many answers for this question but probably cause it is just an unnecessary complexity.
You can achieve same result (x2) with the current architecture (i.e. using multiplication layer; not to mention just squaring your input features). Why would you use something more specific and not something more general?
Beside that why would your final ...
0
Following your example:
The source sequence would be How are you <EOS>
The input to the encoder would be How are you <EOS>. Note that there is no <start> token here.
The target sequence would be I am fine <EOS> . The output of the decoder will be compared against this in the training.
The input to the decoder would be <start> I ...
1
Backpropagation extends to the full model, through all decoder and encoder layers up to the embedding tables.
0
Why don't you use a lower number of filters in the last convolution? Instead of 128 you can just choose whatever number you want, e.g. 10.
Also, normally after the convolutional (and pooling layers), you flatten the output (therefore losing the spatial information) and then project with a dense layer onto the final representation space. You can control the ...
1
The example you cited (using x^2 instead of x) is the idea more popular outside deep learning community, called feature engineering. The trend in neural network modeling is instead to,
Play with weights (w) and fine tune them.
Not change the input vector (x) but feed it to the network directly.
If a single layer neural network is not good enough, add more ...
0
More gpu means more data in a batch. And the gradients of a batch data is averaged for back-propagation.
If the learning rate of a batch is fixed, then the learning rate of a data is smaller.
If the learning rate of a data is fixed, then the learning rate of a batch is larger.
https://github.com/guotong1988/BERT-GPU
0
Usually, the input for LSTMs is a sequence that already happened regardless of train or test set.
Of course you could also try inputting past predictions, but I think this would probably lead to bad results.
If you want to predict further into the future you could increase the output dimension of your LSTM, however this will be more difficult for the model ...
1
I did a project with the CheXpert dataset which includes metadata and labels for uncertainty. I don't know if it's the right kind of metadata you are looking for. In general, medical datasets usually contain metadata. A similar one would be from the NIH.
Otherwise, this is a decent index of high quality datasets.
0
Doesn't ConvNets allow parameters to be shared, detecting the similar features of different images?
Your final statement holds true for convolutional layers which are in early layers, but final layers detect more abstract features, e.g. a full object. Consequently, the last layers of a CNN that is trained with images of flowers will not be helpful for a ...
1
I've used ECS (Fargate) to train models, the retraining trigger could be the start of an ECS service. While ECS has a little latency, it handles well long runtimes.
You can then serve the model via a lambda.
1
Another option would be to apply an established Optical Character Recognition (OCR) system to the raw images. After converting the raw images to plain text, put the relevant data into a tabular dataframe. Once in a tabular dataframe, fit a deep learning or traditional machine learning model can be fit.
It can be tempting to build an end-to-end deep learning ...
0
Keras should be accessed as tf.keras now with tf2 ,so your import should be written as
tf.keras.callbacks.ModelCheckpoint
Keras own documentation as well as tf api documentation can be easily accessed for this purpose. Keras ModelCheckpoint class mentions the following arguments in official docs:
save_best_only: if save_best_only=True, it only saves when ...
Top 50 recent answers are included
