Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

Filter by
Sorted by
Tagged with
0 votes
0 answers
11 views

How to generate multiple captions from an image captioning model in Keras/Tensorflow

I am practicing one of the popular image captioning keras model (LINK IS HERE). Basically this model takes Flickr8k dataset where each image has 5 captions. ...
  • 1
2 votes
1 answer
19 views

Mixed language OCR

I'm solving a table data recognition task And the huge problem is the recognition of mixed language pictures. I'm using tesseract for OCR, but it fails to recognize both languages simultaneously. Here ...
  • 21
0 votes
0 answers
10 views

Rapid MSE decrease in one (or less) epoch. Then seeming never ending but glacial decrease in MSE

What does this mean? That my neural network can get to a good MSE very quickly suggests maybe I have too complex a model. (I think?) But that the MSE/R2 is ok, hardly great, and also seems to improve, ...
  • 71
0 votes
0 answers
14 views

How to feed temporal image data to a 3D CNN?

I am using TensorFlow and Keras to build a 3D CNN, where the 3rd dimension is time. I have a dataset which contains evolving images with time (51 frames were taken). For testing, I took 10 simulations....
  • 1
0 votes
0 answers
17 views

How to use batches for training in Sequence-to-Sequence models?

I am working on a seq-to-seq Image Captioning model, with Vision Transformer as the encoder and a LSTM based model as a decoder. The output from the encoder is given as the hidden state and cell state ...
0 votes
0 answers
7 views

Different Kernel Initializers in my prediction layer with Transfer Learning could affect performance?

So I have this model right here and the task is to classify 3 labels.: ...
0 votes
0 answers
2 views

Does ResNet takes less training and testing time compared to squeezeNet?

It is being said that ResNet manages to overcome vanishing gradient problem thereby making the training faster. So, does it means ResNet requires less training and testing time? Also does training and ...
  • 1
0 votes
0 answers
44 views

NLP Text Classification Model with defined context / intent

This is more of a guideline question rather than a technical query. I am looking to create a classification model that classifies documents based on a specific list of strings. However, as it turns ...
  • 101
0 votes
0 answers
9 views

Is GAN better than DBNs?

I understand that they are different models and work differently, but they are both generative models and can both be used to do feature learning and classification. Is GAN most of the time better ...
  • 1
0 votes
1 answer
23 views

Named Entity Recognition using Spacy V3 with imbalance entities

Will the spacy V3 model get affected by imbalanced entities? I have got a dataset annotated in spacy format and if I look into my custom entities the rations are different for different entities. For ...
  • 103
0 votes
0 answers
38 views

Marginal Probability Distribution of Feature space - meaning

I'm reading some literature on Transfer Learning in NLP, and this is one of the definitions that I came across in Pan & Yang (2010) Here is another definition from Sebastian Ruder which is a ...
0 votes
0 answers
18 views

Why are we training Segment Embedding in BERT?

In BERT we have segment embeddings that are used for "Segment Embeddings with shape (1, n, 768) which are vector representations to help BERT distinguish between paired input sequences." Yes,...
  • 85
1 vote
0 answers
23 views

Deep learning model to predict the actual input values

I have some observed parameters to be used as input to the deep learning model. This problem comes from the wireless field where we transmit $x$. The channel $h$ is random in nature. The received ...
  • 11
0 votes
1 answer
28 views

Training deep neural networks with ReLU output layer for verification

Most algorithms for verification of deep neural network require ReLU activation functions in each layer (e.g. Reluplex). I have a binary classification task with classes 0 and 1. The main problem I ...
0 votes
0 answers
20 views

How does CLS token having meaning of the sentence in BERT

As my understanding CLS token is a representation of the whole text (sentence1 and sentence2), which means that model got trained in such a way that the CLS token is having the probability of "if ...
  • 85
1 vote
0 answers
16 views

Can I distill knowledge across different neural network frameworks?

I'm interested in using knowledge distillation to compress a large deep learning model to a smaller size so it will run on an embedded device. I've found a number of open source examples for the ...
  • 254
0 votes
0 answers
16 views

job roles hierarchy formation literature

I have a csv file with job titles, descriptions and skills associated with each job title. These titles and skills span multiple domains (IT, HR, Banking, Healthcare and many more). I am interested in ...
0 votes
0 answers
21 views

What causes the loss to be nan and accuracy to not improve?

I am training a Bi-Directional LSTM model as follows, to predict the probability of a binary class based on a handful of observed time series. The model is as follows: ...
  • 1
1 vote
1 answer
143 views

How special tokens in BERT-Transformers work?

I was trying to understand how tokens work and all I understood is that tokens are the representation of the input in a more meaningful way (data preparation for the "encoder of transformer" ...
  • 85
0 votes
0 answers
8 views

How to build a multi caption image generator model?

I want to build a multi caption image model. This model should be able to produce (5-8) captions from an image. I have searched about it but there are only single caption generator image models. I ...
  • 1
0 votes
1 answer
30 views

DQN not learning anything - Reinforcement Learning

I am trying to train a DQN to play the 8puzzle game. I have implemented a batched gameboards, so I am not using ReplayMemory. Here's training process: ...
  • 1
0 votes
0 answers
15 views

Circular data model for wind direction

Say I have a matrix of 10 X 10 of vectors in 3D, where the dimensions include height of the specific point, speed in m/s, and angle in degrees. The test cases I have are matrices of 10 X 10, where the ...
2 votes
1 answer
58 views

Low validation accuracy when not using shuffled datasets

First I tried creating the training/testing datasets using sklearn train_test_split function like the following, ...
  • 123
0 votes
0 answers
11 views

Which layers are doing image segmentation on AutoEncoders/U-NET?

While I was researching for transfer learning, I saw that people are replacing encoders with VGG-16 weights and only training the decoder part of the network. But in some representations (like This) ...
  • 85
0 votes
0 answers
25 views

Masking pixels for a CNN

I'm trying to implement a CNN with RGB and depth images. But my depth images are a little sparse. So I would like to mask out those neighborhoods in the RGB image where the depth neighbors are empty. ...
1 vote
1 answer
32 views

Which representation of CNN feature maps is correct?

When I extract my features from my CNN, it doesn't look like this: And those pictures are not just representation. From this article it can be seen that these features are actual extracted features ...
  • 85
0 votes
0 answers
8 views

Discrepancy between hyperband best models and identically manually made models

When I do a keras hyperband hyperparameter search, and obtain the best models from a search over e.g. 30 max_epochs and 10 hyperband iterations, and the search is complete, the hyperband tuner https://...
  • 71
0 votes
0 answers
59 views

The observation returned by the `reset()` method is not contained with the observation space

i'm strugling to understand how custom observation_space should be coded and how there isn't so much info about some topics i have to ask here if for example i have to return a observation with an ...
1 vote
2 answers
49 views

Formal conditions on mappings that can NOT be learned from data

I am new to machine learning and would appreciate some help on the following question. I have observed the literature is focused on algorithms, how one learning does better compared to others for a ...
0 votes
2 answers
118 views

How to Predict Probabilty that the Customer will buy specific Product?

We have data consist of previous transaction history consisting of Date,Order-id, Product-id, Product name, ordered or not. We need to predict a specific product probability for all the customers that ...
  • 36
1 vote
2 answers
24 views

In Gradient descent, Why the gradient of cost function do not have to be normalized into unit vector

From my background, I understand that the purpose of having a learning rate (α) is to normalize the magnitude of gradient (▽J), so the step size can properly converge the local minima Since α is ...
  • 11
0 votes
0 answers
21 views

not able to show evaluation metrics for object detection on MSCOCO like dataset using FASTAI

I want to show evaluation metrics for coco object detection,I have saved my learner in .pth file and I have data in variable data(databunch type fastai) , I have the code from the notebook finding ...
1 vote
0 answers
33 views

Why is a neural network not doing better than multivariate linear regressions?

I am making neural networks of multiple targets, all using same training data. For some of these targets, multivariate linear regressions do a very good job, i.e. a strong linear relation exists ...
  • 71
1 vote
1 answer
19 views

Understand the reason of embedding and the size inside it in Pytorch

I'm very new to pytorch - taking a course in udemy. There is something I find hard to understand and would like to get explaination about, in a bit simpler words than what I can find in the ...
  • 349
0 votes
1 answer
21 views

how to handle soft weight constraints in neural network

Let us assume that there is a feedforward neural network with two layers. and weights of each layer are constrained such that sum of the weights is a constant value in each layer and their values are ...
0 votes
0 answers
11 views

Style Transfer - how to choose the cnn layers?

I read this tutorial. In that tutorial they choosed: conv layer #4 for content_layers conv layers: 1, 2, 3, 4, 5 for ...
0 votes
1 answer
15 views

tensorflow beginner demo, is that possible to train a int-num counter?

I'm new to tensorflow and deep-learning, I wish to get a general concept by a beginner's demo, i.e. training a (int-)number counter, to indicate the most repeated number in a set (if the most repeated ...
8 votes
2 answers
862 views

MLOps for beginner

I am 1 year old in ML and have been using jupyter notebook to build static models all these days, do some analysis and present my results to the bosses as it was all POC. Now, we would like to scale ...
  • 2,337
0 votes
0 answers
7 views

I do not understand normalization and standardization

in my ML Course at Uni, a big topic was normalization and standardization, but still - I don't really get why we do that. More specifically, I'm working on a (fairly complex) CNN to predict missing ...
1 vote
1 answer
20 views

In LSTM why h_t output twice?

According the LSTM design: The hidden state (ht) is output twice (1 and 2 in the picture). If they are the same, why we need them twice ? Is there a different use for each one of them ? According to ...
0 votes
0 answers
43 views

nn.NLLLoss() gives negative result - what it's mean?

I saw code which use nn.NLLLoss() (negative log likelihood loss). I looked on the results and some loss results (result of ...
0 votes
0 answers
25 views

Best Neural Networks for Models with If Statements

If I am trying to use a neural network to learn/improve/replace part of an existing computer model, and that has if statements in it which seemingly make it more difficult to predict than other parts ...
  • 71
0 votes
0 answers
15 views

does constant input add information in training a neural network?

I am working on an image denoising task. My noise pattern is generated from a 3rd degree polynomial function of images. I have multiple sets of 4 images (called tables) to generate different noise ...
0 votes
0 answers
15 views

How to see Latency at layer granularity in a CNN

I am finding documents or an example that measure Latency at layer granularity in the AlexNet model. Please could share or tutorial for me.
2 votes
1 answer
32 views

Is batch size of 1 a valid choice for a very deep neural network with high memory requirement?

I am training a very deep neural network (Panoptic-DeepLab) with a ResNet34 backbone on Google Colab on CityScapes dataset for Panoptic Segmentation, and noticed that, with a big crop size, the batch ...
1 vote
1 answer
81 views

Features and LSTM

I have a problem while developing an LStm model. I have 4 feaures that I want to use to make a prediction. When I test my model with a single feaure I get average results but when I test with all 4 ...
  • 11
0 votes
1 answer
36 views

Transpose Convolution feature extraction

Convolution extracts high-level features, but what about Transpose Convolution (or De/Up-Convolution)? Does it behave exactly the opposite? Does it generate lower-level features?
  • 85
2 votes
0 answers
19 views

why layer of dimension 1 is outputting image of size n

I am studying a model where landmarks from an image are calculated. The work comes from Convolutional Experts Constrained Local Model for 3D Facial Landmark Detection. I need to confirm why the ...
  • 21
0 votes
0 answers
68 views

Reinforcement learning using univalent and multivalent heterogeneous features

Problem introduction I have a game in which human players play levels (just like the famous casual game candy crush) where each level has its own properties and its own difficulty. In said game, the ...
  • 101
0 votes
1 answer
49 views

Newly discovered learning rule

Does anyone know how this algorithm performs the learning process for neural networks? I've stumbled over this solution. It works, but I don't know how and why. It's neuron-local and works without ...
  • 101

1 2 3
4
5
93