Questions tagged [deep-learning]
a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.
4,843
questions
2
votes
1
answer
525
views
Deep learning test loss curve won't go down
I've been working with Deep Learning projects for this current project that I am working on and it's basically a time series classification problem. Where given an array of time series data I need to ...
CommunityBot
- 1
0
votes
0
answers
8
views
Align Vectors are Easy to Learn?
I have three vectors $x,y_1,y_2\in\mathbb{R}^{n\times 1}$, where $x=y_1$, $x\perp y_2$. If I use $x$ as input of a 2-layer perceptron, will regressing $y_1$ be easier than $y_2$ (i.e., when fully ...
- 1
0
votes
1
answer
56
views
Deep Q-Learning: How are network parameters updated, and why consider episodes in the first place?
I'm trying to wrap my head around the implementation of deep $Q$-learning, and why we even consider episodes in the first place. The usual set-up is that we initialize some starting state $s_0$, then ...
- 101
0
votes
1
answer
186
views
How do I feed my keras model in batches?
I am trying to feed a Sequential model in batches. To be reproducible my example, suppose my data is:
X=np.random.rand(24,432)
Y=np.random.rand(24,432)
My goal is ...
CommunityBot
- 1
0
votes
1
answer
117
views
How to increase model's test accuracy?
I am using the InceptionV3 model for training. Here is the link for the code (https://github.com/maxmelnick/tensorflow/blob/no_random/tensorflow/examples/image_retraining/retrain.py) Initially I have ...
CommunityBot
- 1
2
votes
1
answer
212
views
What are the key differences between a MLP with lagged features and a RNN
I've been working with MLP's for a while. Whenever I assumed that the past values of a feature might be useful for predicting the future values of Y, I would just create a new column in my data frame ...
CommunityBot
- 1
1
vote
1
answer
133
views
Predicting a next word from a sentence of a different lenght than seen in training
I am building a custom Decoder-only transformer model, which is being trained on the task of Next Word Prediction. The training procedure is analogous to that of chat GPT models - the input to the ...
CommunityBot
- 1
0
votes
0
answers
9
views
【NLP】Is there a model or task that determines contextual similarity?
I am trying to work on an engagement detection task in which I have to determine if a student is engaged in class.
I am looking for an NLP approach where I can calculate the similarity score of a ...
- 1
2
votes
1
answer
114
views
What doese 'v' mean in GoogLeNet?
In GoogLeNet (this link), there is 'v' notation in Figure3 like '1X1+1(v)'.
I don't know the meaning of 'v'.
Also, I understood 's' as stride. But, I don't know the reason why plus operation is used ...
CommunityBot
- 1
0
votes
0
answers
16
views
decision tree limitation VS deep learning
I wonder if decision trees (and their derivatives like Random Forest and Gradient Boosting) have interpolation power as deep learning based model. Most of my experience is with deep learning model.
...
- 101
1
vote
1
answer
38
views
How to use additional features in image captioning?
I have the following question - is it possible to train a model based on Transformer architecture to use additional attributes to generate a caption for an image?
For example, I have a dataset with ...
- 121
2
votes
1
answer
165
views
How to run tensorflow model twice before computing the loss
I want to compute a loss function which uses output of the network twice on different inputs. For example as hypothetically,
...
CommunityBot
- 1
0
votes
1
answer
281
views
What the difference between a flattening validation curve and one that increases again?
I know that we monitor the validation loss to investigate overfitting. I am familiar with the validation curve that first decreases and then increases again. The increasing part means the model starts ...
CommunityBot
- 1
3
votes
1
answer
666
views
Why are parameter updates downscaled by uncentered variance (instead of centered variance) in Adam optimizer?
In Adam optimizer algorithm, parameter updates are computed as follows:
$\theta_t \leftarrow \theta_{t-1} - \alpha \frac{\hat{m}_t}{\sqrt{\hat{v}_t}+\epsilon}$
Where $\hat{m}_t$ is a bias-corrected ...
CommunityBot
- 1
5
votes
4
answers
119
views
Reasoning behind using Deep Learning on non-local data
I understand the using of deep learning for data that have "local" structure, for example, images/videos/texts, as the convolutional layers reduce the amount of dimensions.
However, I saw ...
CommunityBot
- 1
0
votes
1
answer
112
views
after overcoming the overfitting, how to increase training accuracy?
I am building a CNN using keras for a classification task. I started with a simple model as a starting point and as almost all ML problems go, especially if the dataset is not very big, I faced an ...
CommunityBot
- 1
0
votes
2
answers
109
views
How do I predict a class for each time step using the information from previous timsteps
I have a classification problem but different than usual. I have to provide 3 outputs (each of them either 0 or 1) for every input of 3 timesteps and 10 features.
What model architecture or approach ...
CommunityBot
- 1
0
votes
1
answer
275
views
Training the document page layout and classifying good/bad layouts
I have a use case where I am supposed to get the coordinates of each block element in a page (whether its paragraph, image, table) where I train a model to understand how they are placed in a given ...
CommunityBot
- 1
0
votes
1
answer
139
views
Can the performance of a CNN be dependent on the train-test-val split random seed?
I am doing multi-class classification and comparing the effects of 2 image enhancement techniques (IET).
IET 1 performs better than IET 2 at random seed x (for train-test-val split)
IET 2 performs ...
CommunityBot
- 1
0
votes
1
answer
2k
views
how to use tensorflow graphs in multithread？valueerror：tensor a must be from the same graph as tensor b！
I am doing instance detect and image retrieval task by Keras and Tensorflow as backend.
I plan to use multi thread to load two model, I load maskrcnn in a thread and load mobile net in another one.
...
CommunityBot
- 1
0
votes
0
answers
14
views
Which image classification methods/models could suit my (product) image classification problem?
Say you are a potato chips company. The goal is to have consumers upload images of the product they are having issues with and be able to identify the product by brand/variant using machine learning. ...
0
votes
1
answer
69
views
AdaGrad: Intuition
The update formula for Adagrad is:
\begin{equation}
w^i(t)=w^i(t-1) -\frac{\eta}{\sqrt{\epsilon +\sum_{1}^t |\nabla_i\mathcal{L}}|^2} \nabla_i\mathcal{L}
\end{equation}
It indicates that if the ...
CommunityBot
- 1
0
votes
1
answer
244
views
Can I use one-hot encoded output for segmentation in Pytorch, with focal and dice losses?
know that for classification using a neural network and CrossEntropy Loss, we need one-hot encoded output, but in PyTorch, the CrossEntropy loss does not accept one-hot encoded targets, and we should ...
CommunityBot
- 1
0
votes
1
answer
167
views
How to use hierarchical variable in a ML model
I am working on a binary classification problem with 1000 rows and 20 variables.
I have variables like product_id, city, ...
CommunityBot
- 1
1
vote
1
answer
173
views
predict a binary vector of size 40
I have a dataset of shape (2600, 95) with first 55 columns are features and 40 columns are label.
Label is a binary matrix of size 10x4 that flattened, and features are real valued numbers ranging (0....
CommunityBot
- 1
3
votes
2
answers
749
views
Is it possible to make a label automatically in supervised learning(Machine Learning)?
My background knowledge: Basically, supervised learning is based on labeled data. Using the labeled data, the machine can study and determine results for unlabeled data. To do that, for example, if we ...
CommunityBot
- 1
0
votes
2
answers
133
views
Can Transformer Models be used for Training Chatbots?
Can Transformer Models be used for Training Chatbots?
Note - I am talking about the transformer model google released on the paper 'Attention is all you need'
CommunityBot
- 1
1
vote
1
answer
227
views
Tips for improving multitask learning based on multiple outputs
I'm currently trying to use multi-task learning based on a multi-output model that both allows to get an output for classification and regression. However, at the moment it's staying at around 20% ...
CommunityBot
- 1
0
votes
1
answer
219
views
False positive in Multi class Image classification
I am training a neural network with some convolution layers for multi class image classification. I am using keras to build and train the model. I am using 1600 images for all categories for training. ...
CommunityBot
- 1
4
votes
3
answers
905
views
How to determine the number of the training images in Keras after data augmentaion?
I want to create a CNN model and I am using data augmentation. I want know the number of augmented images in Keras.
How to determine the number of the training images in Keras after data augmentation?...
CommunityBot
- 1
1
vote
0
answers
28
views
ML paper reproducibility
How can I reproduce results in an ML paper if I don't have the identical resources to train the models as in the paper ? (in my case I only have a laptop spec NVidia gpu and in most of the papers I ...
- 11
0
votes
0
answers
10
views
Meaning of mean squared error in multistep prediction
In multistep prediction with LSTM(keras), say we had this kind of result:
target = [[1,2,3] ,[4,5,6] ]
predictions = [[1.1,2.2,3.3] , [4.4,5.5,6.6]]
When we choose mean_squared_error as the loss ...
- 23
3
votes
2
answers
136
views
How can we create an label, value detector?
I am trying to implement an text detector using MaskRCNN such that the model detects the label and value as shown in the image below.
Detecting the same is easier for fields like page date and order ...
CommunityBot
- 1
2
votes
1
answer
110
views
Using the first 3 layers of a pretrained network in Keras
I want to use the 3rd layer's output of the VGG16 network. The error is like below:
...
CommunityBot
- 1
0
votes
2
answers
401
views
Is Siamese network rotation invariant?
Is Siamese network rotation invariant which means if I train my siamese network on the different rotated versions of the same image so will it treat each image as different image or same.
Also if I ...
CommunityBot
- 1
0
votes
0
answers
18
views
3D Design file labelling and classification for manufacturing
I have ~1 million 3D design (.STP and/or .OBJ) files of various parts for medical devices, aerospace, automotive or defense systems. I'd like to label them based on appropriate manufacturing methods ...
- 1
114
votes
11
answers
124k
views
Choosing a learning rate
I'm currently working on implementing Stochastic Gradient Descent, SGD, for neural nets using back-propagation, and while I understand its purpose I have some ...
- 111
0
votes
1
answer
37
views
Understanding Multi-headed Attention from architecture details
I've a conceptual question
BERT-base has a dimension of 768 for query, key and value and 12 heads (Hidden dimension=768, number of heads=12). The same is conveyed if we see the BERT-base architecture
<...
0
votes
0
answers
8
views
Has someone designed a neural network which can select its own activation functions and/or have multiple activation functions in one model?
I'm wonder if there are any papers or implementations where a neural network has multiple activation functions in a single model (and layer), and preferably also where such activation functions ...
- 101
2
votes
1
answer
118
views
Role of stateful parameter vs shuffle parameter in LSTM keras
I'm trying to make prediction on a multivariate time series using LSTM. I know stateful=True in keras LSTM means state(hidden) of each sequence, in a batch, at index i - is passed to the next batch, ...
- 23.8k
0
votes
1
answer
71
views
Predict the values of variable features over timestamps
HI i am having a dataset which contain timestamps and number of users at that timestamp. Each user has resource values which change per timestamp. How can i make predictions of number of users ...
CommunityBot
- 1
1
vote
1
answer
119
views
What is the preferred approach for this problem?
I have the Data of 10,000 users Time Session in a website/App, The Login time, logout time, the person activity,
The Data is available for 60 days ( per user )
Using this 60 days data for 10k ...
CommunityBot
- 1
0
votes
1
answer
155
views
What does the no of nodes represent in a Convolution Layer?
What does the number 16 (no. of outputs) represent in this Convo layer?
layers.Conv2D(16, 3, padding='same', activation='relu'),
CommunityBot
- 1
0
votes
1
answer
63
views
How can I explain the cause of different performances for two different LSTM models and improve the performance?
I've built two different models for Load Forecasting. Dataset has six features. The performance evaluation metric is the Mean Absolute Percentage Error(MAPE). Both models are based on LSTM. Here is ...
CommunityBot
- 1
2
votes
2
answers
122
views
Why does a filter need to be applied to the output of the input gate before cell state is added to?
In a neural network there are 4 gates: input, output, forget and a gate whose output performs element wise multiplication with the output of the input gate, which is added to the cell state (I don't ...
CommunityBot
- 1
2
votes
1
answer
3k
views
How to generate Anchor boxes for SSD?
I am currently trying to understand the method of generating anchor boxes for object detection. I am looking at a code where the author has done this task in a very flexible way. But I am having ...
CommunityBot
- 1
0
votes
1
answer
65
views
How does TensorFlow handle multiple samples?
Say the mini-batch has $N$ samples $(x, y)$,
how will tensorflow utilize this $N$ samples to train the network.
Will it do $N$ forward loop for each sample independently?
Will it do $N$ backward ...
CommunityBot
- 1
1
vote
1
answer
669
views
CNN model low accuracy
I have 1299 images in 4 classes (374/269/284/372). I want to use the VGG19 model, add a dense layer at the top and fine-tune it with my images. As I only have 1299 images, I also want to use data ...
CommunityBot
- 1
1
vote
2
answers
466
views
Negative examples for a Yes/No image classification neural network
I am trying to retrain a neural network using transfer learning that can classify whether an image has a certain object, say, a car. My positive sample dataset is quite small, only 2500~ images. It ...
CommunityBot
- 1
1
vote
2
answers
80
views
NN converges quickly but is it a problem when performance is good on test set?
I have an LSTM model I'm using for time series predictions. In training it converges already after 3 epochs. The model performs quite well on the test data, but should I still be concerned about the ...
CommunityBot
- 1