Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

Filter by
Sorted by
Tagged with
1 vote
1 answer
51 views

High level-Low Level features in U-NET

Why do the first layers of U-Net or CNN generate low-level features? Why not the last layers? What is the logic behind getting low-level features at the beginning of architecture? And yes, high-level ...
user avatar
  • 71
2 votes
1 answer
22 views

Choosing Right Optimiser and Data Scaling

The choice of optimiser and how data is scaled are both very important things in machine learning, yet they are not hyperparameters (as far as I am aware). It is also not necessarily obvious which ...
user avatar
  • 71
0 votes
0 answers
19 views

Why input normalization leads to worse performance?

I'm building quite simple NN with Dense layers followed by ReLU activations and I noticed something unexpected. Generally, I've been confident that normalizing the input to have mean of 0 and standard ...
user avatar
  • 203
1 vote
1 answer
23 views

Different results between hyperparameter optimisation and actual training/val values

If I want to do a hyperparameter optimisation on a dataset using e.g. hyperband or random search, I note that some of the models being randomly chosen seem to have rather good R2 scores, MSE etc. I ...
user avatar
  • 71
0 votes
0 answers
12 views

How to get attention weights from BERT

I am trying to make transfer learning in the encoder and decoder parts of the transformer. The encoder is a whole feature extraction part and there is the feature extraction part in the decoder too(I ...
user avatar
  • 71
0 votes
0 answers
16 views

Label network embedding as input features for multi-label classification

Leveraging correlation between labels is an essential aspect of multi-label classification. I am trying to figure out the best approach for incorporating label correlation information for my task. One ...
user avatar
0 votes
0 answers
33 views

Using BERT embeddings as input for transformer architecture

I will use BERT's embedding weights (as discussed here) for embedding in embedding layers of the transformer model. But my question is: don't embeddings of BERT already go through the whole encoding ...
user avatar
  • 71
1 vote
1 answer
96 views

ValueError: cannot reshape array of size 36276416 into shape (96,227,227,1)

I am running my LeNet code with LFW, but when I run it, I am getting the following error message: Here is the code that it is getting the error ...
user avatar
  • 11
1 vote
1 answer
42 views

Why can't I reproduce my results in keras using random seed?

I was doing a task using RNN to predict a time series movement. I want to make my results reproducible. So I strictly followed this post: https://stackoverflow.com/questions/32419510/how-to-get-...
user avatar
0 votes
0 answers
15 views

Chat message classification

I am trying to build a deep learning model that can classify a chat message of variable length based on the message itself as well as the previous messages (context), all of which may be generated by ...
user avatar
0 votes
1 answer
21 views

What is meant by averaging inhibits it in the paper 'Attention is All You Need'?

Could anyone explain to me about the sentence below? What is meant by averaging inhibits it? Multi-head attention allows the model to jointly attend to information from different representation ...
user avatar
0 votes
0 answers
37 views

Why isn't my backprop matching autograd?

Am attempting to implement backpropagation for a deep learning course but my backprop gradients don't seem to be matching the gradients you get from autograd. Here's the code: Is my math incorrect ...
user avatar
0 votes
0 answers
7 views

How to calculate F1 and EM Match for Question answering système in other languages

I am creating a question answering system. For my English version squad was used, they have an evaluation script in their site to calculate F1 and EM Match, but how can I calculate F1 and EM Match for ...
user avatar
0 votes
0 answers
15 views

Has anyone been able to run sklearn_evaluation.plot.grid_search with the "kind" parameter as "line"?

I am trying to optimize the hyperparameters (one of which is hidden_layer_sizes) of the ML learner: MLPRegressor, appealing to a visual representation to guide me ...
user avatar
0 votes
0 answers
13 views

Fakebert implementaion

I am trying to implement this architecture of fake bert for fake news detection, but I don't know how to feed the word embedding from Bert. Help, please.
user avatar
1 vote
1 answer
18 views

Time series model hardly fitting well

I'm trying to forecast Google's stock prices. I've made two models one with LSTM and another one that's Bidirectional LSTM, but the forecasted values don't converge quite well with the test values. I'...
user avatar
  • 11
0 votes
0 answers
33 views

GAN - discriminator loss remains at a constant value while the generator loss decreases?

I am building my first GAN network, and I noticed that sometimes the discriminator loss remains at a constant value while the generator loss decreases. I couldn't find an explanation - if the ...
user avatar
3 votes
1 answer
126 views

Performing a text classification based on a dictionary

I have been given a kind of dictionary which maps a category with a set of certain strings. A sample of the dictionary is given below: This is all I have, there is no other data. There are around 46 ...
user avatar
0 votes
0 answers
10 views

How to organize data to use a recurrent neural network LSTM?

I am doing an internship in bailiff society. I have to create an IA model which can improve actions to perform, based on existing timeline of actions. I've already tryed some solutions (which did not ...
user avatar
  • 1
1 vote
1 answer
33 views

BertTokenizer Loading Problem

I loaded this BertTokenizer previously, but now it is showing, I have to make sure I don't have a local directory. In my kaggle kernel, I don't have this local directory. How to solve it? ...
user avatar
0 votes
0 answers
18 views

Transfer learning on the same samples for progressively more specific classification

I have a classification task in which classes exist within a directed graph. That is a class may have subclasses which share an is-a relationship with their parent class. Now, I have a relatively ...
user avatar
2 votes
1 answer
23 views

All classification models except neural network giving 100% accuracy

I have a dataset of size (140,000, 10) containing 1 dependent variable. I used MinMax scaler on independent variables. For the target value, there is a class imbalance of 94% 0's and 6% 1's. Used ...
user avatar
  • 21
2 votes
1 answer
306 views

How do researchers actually code novel architectures and layers?

Disclaimer: I am almost a complete novice when it comes to tensorflow, keras, coding in general, and neural networks/data science. While reading papers on novel architectures for neural nets, I see ...
user avatar
  • 23
1 vote
1 answer
26 views

Having weird accuracy graph on deep learning binary classification model

I am doing deep learning binary classification on some data and got very weird results with the accuracy metric. In the first few epochs, it doesn't change at all but then it goes on this weird linear ...
user avatar
  • 125
1 vote
1 answer
26 views

Biometrics identification with embeddings comparison and "unknown"/"other" class/label

This is a general or more conceptual questions about biometric classification models, based on deep learning neural networks. The goal of the system is to take a set of features (e.g. voice recording, ...
user avatar
0 votes
0 answers
16 views

Learning Rate Finder doesn't work with Tversky Loss, any idea?

I'm working with a UNet on a binary segmentation problem. As my dataset is extremely imbalanced (sometimes the objects I'm trying to segment are really, really small) I'd like to use the Tversky Loss ...
user avatar
0 votes
0 answers
18 views

How YOLO detects the object when the object is in multiple grid cells?

I have been reading various articles and watching videos on YouTube, but i cant seem to understand how does YOLO makes a bounding box for an object if it is in multiple grid cells? for example in the ...
user avatar
1 vote
1 answer
16 views

How to deal with spelling errors in NLP classfier (low resource language)

I know there are questions on how to deal with spelling error NLP - but the question and solution are mainly focused on English where there are tons of library for spell-correction. Here I am curious ...
user avatar
0 votes
0 answers
14 views

tensorflow: error with parallel channel model

I'm debugging a model I created to accept a variable number of input channels (each channel is an RGB image). I suspect that not all the channels are properly connected. Model code: ...
user avatar
  • 23
0 votes
1 answer
26 views

What can we learn from visualizing Feature Maps

I have the following classification model (dogs vs cats): ...
user avatar
0 votes
1 answer
26 views

Interpreting a curve val_loss and loss in keras after training a model ; help

I need some help in Interpreting a curve val_loss and loss in keras after training a model These are the learning curves:
user avatar
1 vote
0 answers
13 views

Feature Selection - Comparing Performance of different size datasets

If I have training data X, with N features, and I do feature selection, and discover n of <...
user avatar
  • 71
0 votes
0 answers
21 views

Extracting Features Using TensorFlow(EfficientNet)

I am trying to apply the following model in this paper Owl classification paper on my own dataset. I already created Spectrogram Images and MFCC image from my own dataset, but I don't know. How can I ...
user avatar
  • 27
0 votes
0 answers
10 views

Can I use pre-trained coco weights for medical lesion segmentation? Or should I train the network from scratch?

I am trying to understand in what cases I can benefit from pre-trained weights. Sometimes, pre-trained weights works (can be fine-tuned) for other domains but I cannot understand how to make a ...
user avatar
  • 1
0 votes
0 answers
17 views

How do we arrive at deep neural network architectures with many components?

There are many very deep, complex neural network architectures such as ResNet that have achieved significant success. Something I don't understand is how developers are arriving at these architectures....
user avatar
0 votes
0 answers
69 views

Loading saved model fails

I've trained a model and saved it in .h5 format. when I try loading it I received this error ...
user avatar
  • 1
0 votes
0 answers
12 views

What enables transformers or very deep models "plan" ahead for sequential decision making?

I was watching this amazing lecture by Oriol Vinyals. On one slide, there is a question asking if the very deep models plan. Transformer models or models employed in applications like Dialogue ...
user avatar
2 votes
1 answer
105 views

Neural network / machine learning approach to model specific sequencing-classification problem in industry

I am working on a project which involves developing a machine learning/deep learning for an application in a roll-to-roll industry. For a long time, I have been looking for similar problems as a way ...
user avatar
  • 73
0 votes
0 answers
19 views

Intuitively, why do Non-monotonic Activations Work?

The swish/SiLU activation is very popular, and many would argue it has dethroned ReLU. However, it is non-monotonic, which seems to go against popular intuition (at least on this site: example 1, ...
user avatar
  • 23
0 votes
0 answers
13 views

Computing probabilities in Plackett-Luce model

I am trying to implement a Plackett-Luce model for learning to rank from click data. Specifically, I am following the paper: Doubly-Robust Estimation for Correcting Position-Bias in Click Feedback for ...
user avatar
0 votes
0 answers
10 views

Facebook picture labeling

I want to train a neural network and use open CV for facial recognition. Nicholas Martin, whose a user here on SE told me that this is a supervised learning class (clearly). So I need pictures and ...
user avatar
  • 133
0 votes
0 answers
8 views

Why convolutional layer learns only biases?

I`m training a siamese CNN to distinguish between pairs of images and though my train/val binary cross-entropy loss values show negative trend, implying some of the model parameters are being updated, ...
user avatar
0 votes
0 answers
48 views

How does the margin constant (alpha) in the triplet loss affect the training process when it is a constant?

How does the margin constant in the triplet loss formula affect the gradient calculation when its derivative will be zero?
user avatar
  • 101
1 vote
0 answers
16 views

Val Loss and manually calculated loss produce different values

I have a CNN classification model that uses loss: binary cross entropy: ...
user avatar
  • 201
1 vote
0 answers
11 views

Input-depending model on TensorFlow

I'm working on a TensorFlow model, and I would like to have different logic in the model depending on the input. My question is if it's possible to have a "dispatch" layer in TensorFlow that ...
user avatar
  • 514
1 vote
0 answers
26 views

How to determine the number of Neurons in each hidden layer and number of hidden layers for face recognition [duplicate]

I plan to build a CNN for face recognition using this Kaggle dataset. I tried building a model with a single hidden layer with 256 fully connected neurons, and it gave an accuracy of 45% after 55 ...
user avatar
0 votes
0 answers
21 views

What do you think is the best method (image registration) for the attached image?

I have a set of images for multimodal medical images, and after doing the segmentation, I need to register the green lines with red lines in the attached figure. Which method do you think is best for ...
user avatar
  • 1
2 votes
0 answers
25 views

Are more target labels in a multi-label classification always better?

Context We work on medical image segmentation. There are a lot of potential labels for one and the same region we segment. There can be different medically defined labels like anatomical regions, more ...
user avatar
  • 121
0 votes
0 answers
14 views

How to overcome importerror for tensorflow

I'm working on ubuntu, and i'm trying to train my python model but i receive the following error ...
user avatar
  • 1
0 votes
0 answers
16 views

Image Classification problem for minute defect detection

I am tasked with the problem of finding defects in a compressor wheel.Here is how a good wheel looks like: Here is how a defective wheel looks like ( I have drawn a box around the defective area): I ...
user avatar

1 2
3
4 5
92