Questions tagged [deep-learning]
a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.
4,837
questions
0
votes
0
answers
21
views
Different size of deep learning models but similar inference-time
I have three different semantic segmentation models with large differences in size. The first one includes 30,000,000 trainable parameters, the second one about 20,000,000 and the third one about 200,...
- 111
0
votes
1
answer
22
views
Shape of Flattened Layer in CNN
If I have a convolutional layer with dimension (5,5,4), (i.e, 4 no. of 5x5x1 feature maps), what will be the dimension of the flattened layer, if I apply flattening ...
0
votes
0
answers
21
views
Is it a good idea to use attention in VAEs for image generation?
There are research papers and codebases on GitHub that deal with VAEs for image generation on popular datasets like CelebA, etc. While surfing through Google Scholar I found self-attention and other ...
- 101
0
votes
1
answer
30
views
Label_fields are not valid error when using Albumentations
I'm not sure if you can have duplicates cross-forums, but my previous question on Stack Overflow was never answered. I'll paste it here just in case.
I'm using albumentations with the following code:
<...
0
votes
0
answers
12
views
How to analyze social media data to see its impact on a game's sales
I work for a console gaming giant. We forecasted the sales for a RPG game that was to be released few months back. But the actual sales was twice the forecast. This compelled the developers to ...
1
vote
1
answer
127
views
Input dimensions for the EfficientNetV2 family of models
I have a question regarding the EfficientNetV2 family of models. If my understanding is correct there are 6 models under this family - B0 to B1 & S are the comparatively smaller models while M &...
- 13
0
votes
0
answers
11
views
Decoder model selection for sequence-to-sequence ASR with XLS-R
I am new to building models for Speech ASR. I want to build a model for Speech Transcription in Urdu. I used XLS-R (following the example by Patric von Platen on Huggingface Colab notebook. My ...
- 23
0
votes
0
answers
21
views
What Can Prevent Time-Series Prediction Model From Learning Trend?
I am building an encoder-decoder prediction model based on this paper:
https://www.sciencedirect.com/science/article/pii/S0952197623001483
It is made of a transformer encoder and a 1D CNN Decoder. The ...
- 1
0
votes
0
answers
14
views
How to increase model's validation accuracy?
I'm trying to build a model for text similarity problem using CNN, Bi-directional GRU and Bi-directional LSTM. I've tried changing several parameters but I'm getting the maximum validation accuracy as ...
0
votes
1
answer
48
views
Are there any R packages that support Deep RNNs?
I recently found an interesting paper on what it really means for a recurrent neural network (RNN) to be deep here. Depth can be added in several different ways (state to state, input to state, etc.) ...
- 21
1
vote
1
answer
49
views
How to Justify Anomalies Detected by Unsupervised Anomaly Detection Models? [closed]
I'm working on an unsupervised anomaly detection project involving a large sensor dataset, where I aim to identify anomalies without the aid of labeled data. While I've implemented several ...
0
votes
0
answers
10
views
Enumerable column in simple dataset
I have created my first dataset, which is divided into 3 smaller datasets. Main data is 1000 rows about cars, columns: model,year,mileage,price. Column "model" is enumerable (FiestaSedan,...
- 101
0
votes
1
answer
40
views
What neural network architecture would help me model a spectrogram?
I'm really a novice working with these technologies and I'm struggling to design a neural network that is powerful enough to model a spectrogram. For a personal project, I'm working on a spectrogram ...
- 3
0
votes
1
answer
25
views
Set a numerical range restriction on the output of a model
I am trying to train two models together. Both are regression models.
The first model would output a prediction, which is fed into the second model.
The second model is pre-trained to make good ...
0
votes
0
answers
37
views
Am I using a network that is too simple for the dataset/task?
I am training an RNN to classify some high-frequency financial data. A very good performance on this data would be an accuracy of >52% or so.
I have around 650K training examples and 150K dev set ...
- 103
0
votes
0
answers
12
views
Using neural networks to learn total distributions from linearly transformed samples
I am trying to explore the use of neural network based models to replace a cumbersome challenge with a faster more approximate method. The problem involves learning a distribution termed the "...
- 121
0
votes
1
answer
46
views
Use computer vision to detect door blockage
I want to detect door blockage on a camera. Basically if the exit door is blocked by an object, it detects it as an anomaly. How can we do it? Is it possible to do it using OpenCV?
Remember, it doesn’...
- 217
0
votes
0
answers
24
views
Why do I get such a low accuracy despite having good metrics?
I'm working on an image segmentation problem. I'm fine-tuning the network, and when displaying the metrics I obtain, I have some doubts.
I am going to provide some details about the network:
Solver: ...
- 304
0
votes
1
answer
29
views
I have created a CNN model and now i want to draw its architecture diagram can anyone help me with that
following is my architecture:
...
0
votes
2
answers
33
views
Is this an unusual distribution for a sigmoid output from a neural network?
Shown here is the histogram of around 130K predictions of my deep neural network that is classifying some financial data. This is on the dev set but a similar distribution is also seen on the train ...
- 103
0
votes
0
answers
79
views
Fine-Tune Llama on main and auxiliary task
I am trying to fine-tune Llama model on two task at the same time, using hugging face library:
Main task: Causal language model like the model was initially trained for
A classification task based on ...
- 11
0
votes
0
answers
25
views
Weight decay used by Adam optimizer for neural network caused NaN validation loss
I've built a model with BCE loss for CTR prediction in which the major part is a transformer encoder. I've used 0.1 for dropout probability. When using 0 weight decay for Adam the training and ...
- 392
0
votes
0
answers
13
views
Retraining a TFlite Model for Fall Detection on Smartphone Accelerometer Data
I have developed a CNN model for Fall Detection using Keras and converted it to a TFlite(TensorFlow Lite) model for integration into an Android app. The app allows users to collect samples, which can ...
0
votes
0
answers
14
views
How does the use of 1x1 convolutional layers represent permutation in the GLOW model?
I am currently reading the GLOW paper (found here) and I can not understand how the authors claim that the use of 1x1 convolutional layers is equivalent to permutation holds true.
I understand how a ...
- 1
0
votes
2
answers
32
views
Dropout and BatchNorm decrease speed of learning
Experimenting with the cifar10 dataset and faced with strange behavior when Dropout and BatchNorm don't help at all.
As I get:
Dropout - freezing some of the weights which helps us to prevent ...
- 103
0
votes
1
answer
21
views
Problem for a math formula in Weight Uncertainty in Neural Network
I am studying the paper https://arxiv.org/pdf/1505.05424.pdf and there is a formula I don't get page 4:
I don't understand how they obtain this formula. Moreover, with chain rule, I get $\frac{\...
- 1
0
votes
0
answers
23
views
BCE loss stuck at 0.693 in the beginnng of training and then started to decrease, why?
I'm using a Transformer encoder with a binary cross entropy loss for CTR prediction. The training batch loss is at around 0.693 constantly for the beginning several thousand steps (batches). I'm using ...
- 392
1
vote
0
answers
64
views
Annotating and Structuring a Dataset for Duplicate Detection
I'm currently working on a project that requires the detection of duplicate bands in Western blot images. The task involves two types of duplicates:
...
- 111
0
votes
0
answers
75
views
How to use pretrained encoder for customized Unet
if you have a standard Unet encoder such as resnet50, then it's easy to add pertaining to it. for example:
...
- 105
0
votes
0
answers
23
views
Tensorflow loss: 0.0000e+00 - accuracy: 0.0000e+00
I was making changes to improve myself in a chatbot code using LSTM. But Loss and truth values are getting ridiculous values.
Code:
...
- 1
0
votes
1
answer
16
views
Seeking guidance regarding course selection
I have completed Prof. Andrew Ng's Machine Learning Specialization a week ago and now exploring different competitions (Titanic, House Price Prediction, and from the Playground Series). I want to ...
2
votes
1
answer
31
views
Could resnet handles "one image" to "multiple ouputs" task?
I am not doing image classification. I need a model which take one image as input and ouput multiple values (all values could be larger than 1). Could resnet50 be used for this task?
I checked the ...
- 21
0
votes
0
answers
18
views
Overfitting on implemented Dense-Net architecture
I have been playing with different architectures and see how they would perform on the quick draw dataset. Even though the accuracy is significantly higher, I can't reduce overfitting no matter what I ...
- 1
1
vote
0
answers
34
views
How does HDBSCAN generate cluster hierarchy?
HDBSCAN generates the minimum spanning tree where each vertex represents the data point whereas the edges represent the mutual reachability distance.
But how does it generate the cluster hierarchy?
I ...
0
votes
0
answers
13
views
Using images with 5 or more channel in a tf.data pipeline
I have a data generator used to train FCNs (eg Unets) that uses tf.data. The image parser followed by the creation of the tf.data dataset for the training images looks like:
...
2
votes
1
answer
32
views
How to evaluate multiple methods in paper?
In papers, a problem have multiple ways of solving it. E.g. For an imbalance dataset, there are method A, B, C fixes during modelling. However, how can I objectively evaluate which one to use, given ...
- 103
0
votes
0
answers
13
views
multi label multi class classification problem
I am trying to solve a aspect based sentiment analysis problem.
I am considering to devise a NN but am not sure if it is doable the way I am doing.
here is how I structure it.
I have a training set of ...
- 1
0
votes
1
answer
30
views
Training and validation loss sometimes not decreasing in Keras dense layer with the same data and random seed
I want to use a Multi-Layer-Perceptron in Keras (Dense layer) to map 6 inputs to 1 output. For doing this I use the following code:
...
- 71
0
votes
0
answers
25
views
How do I test with custom images for a model that was trained on quick draw npy dataset
I have been trying to test my CNN model on a custom doodle of an apple that I drew. But even when I preprocess the image to have the same shape with the training data, the model gives wrong prediction ...
- 1
0
votes
1
answer
34
views
Flattening before Fully-connected Layer (DENSE)
Can anyone explain why we need to flatten the data before inputting it into a fully-connected layer? What will happen if we input a matrix of size (m,n) into a fully-connected layer that has k ...
-1
votes
1
answer
43
views
how to select number of number of layers and neurons in neural network(RNN) in standard way?
For example, i have a 4000 samples/data points and we have to categorize them into 4 classes.
while building MLP RNN multi-class text classification model, which has 4 classes.
For building model,
1....
- 37
0
votes
1
answer
67
views
Implementation of Graph Neural Network for Image Classification
I'm currently working on a project where I want to utilize Graph Neural Networks (GNNs) for image classification tasks. However, I'm facing difficulties in understanding how to implement GNNs ...
- 172
1
vote
1
answer
22
views
Doubt in gradient , vanishing gradient problem in Back propagation
As per my knowledge, in back propagation- loss function or gradient is used to update the weights.
in back propagation, weights became small w.r.t gradients, this leads to vanishing gradient problem.
...
- 37
0
votes
1
answer
64
views
Effect of hyperparameters: the hidden size, layers, MLP size number of heads on Transformer
Is there any paper that explains the effect of hyperparameters:
hidden size
Number of layers
MLP size
number of heads
on Transformer performance. I found some explanation on the web but I need ...
- 105
0
votes
1
answer
64
views
Correlation between multiple time series
For research, we put some test samples through a physical process for a certain period of time and make measurements. The general structure of the data we collect is as follows:
...
0
votes
2
answers
37
views
Tensorflow: How to pass multiple images to VGG16 layers
This is a toy problem I am working on. I have an extruded N-sided polygon that I have rendered from 5 different randomly selected viewpoints. The classification task is to determine the number of ...
- 111
0
votes
0
answers
15
views
CGAN - Odd Distribution gap, failure of convergence?
I am trying to train on some 1 dimensional data (675 samples, its very expensive to get more) and trying to match the distribution seen here:
There are labels from 1-3 as to associated with the noise....
- 1
0
votes
0
answers
23
views
Please give me suggestions to identify scammers from communication data
I have some communication behavior data of carrier subscribers, containing information on calls made, duration and base stations. What do I need to do with this kind of data or what kind of machine ...
- 9
0
votes
0
answers
98
views
Can I convert a TensorFlow tensor to a CSV file?
I'm working on a semantic segmentation problem. I have saved my images in a tensor of shape (4767, 192, 192, 3) [It contains 4767 images of size 192192 with RGB channels]. On the other hand, I have ...
- 304
0
votes
0
answers
10
views
Approaches to improve the performance in defect prediction of source code?
I have the task to do defect detection on C source code (on function level) starting from this repo (using RoBERTa): https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/Defect-detection
It does ...
