Questions tagged [deep-learning]
a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.
4,837
questions
0
votes
0
answers
3
views
Help understanding working of KeyBERT for keyphrase extraction
I'm fairly new to reading and understanding research papers, so I wanted to get a second opinion on whether my understanding of KeyBERT was correct. Here is a high level overview of my understanding ...
- 23
0
votes
0
answers
7
views
Can we show only results of some epochs in tqdm?
I am training a NN and use tqdm for showing the results. However, the bad thing is that it shows the results for every epoch. This is too many as I want to train NN for atleast 500 epochs.
Is there ...
- 1
1
vote
0
answers
7
views
Why is T5 often used in text-to-data for text prompt encoders?
In the text-to-data(music, image, audio, etc.) generative AI field, one method of encoding text prompts is to use pre-trained language models. Such an approach was used in research on Moûsai [1] and ...
- 11
0
votes
0
answers
6
views
Text and Checkmarks Extraction from an Image
I am working on a project where in I have a filled form which is a safety inspection checklist and I am processing it through AWS Textract.
I am able to fetch text, layout, tables, signatures but ...
- 1
0
votes
0
answers
10
views
Neural regression predictions all around the mean of target
I have a transformer regression model and some data about last users transactions (categorical and numerical). My target has exponential distribution with mean aroud 10e4 and also zero-inflated, so I ...
0
votes
0
answers
16
views
Keras (Hidden) Layer Setup Causing Issues
EDIT: This appears to have solved itself. I’ll spend some more time on it then delete this post if it still works well. Thanks.
I have a network that works fine if I use only 1 hidden layer, and gives ...
- 1
0
votes
0
answers
16
views
Convert specific domain knowledge text to a knowledge graph
As part of this semester assignment , I'm working on a project that aims to to represent the knowledge in "PMBOK 6th edition, section 11: Project Risk Management (page 395 -> 458)" and the knowledge ...
0
votes
1
answer
17
views
How to add multiple embeddings (layers) to LSTM layer
The similar question was asked before here https://stackoverflow.com/questions/52627739/how-to-merge-numerical-and-embedding-sequential-models-to-treat-categories-in-rn/52629902#...
0
votes
0
answers
6
views
RecSys model performance stalling at 47% AUC and F1-Score. Is the problem due to ratio of users to items in my dataset?
I'm having trouble with making my validation metrics go down for the binary_crossentropy and go up for the F1-score and AUC. I've tried tuning my hyper parameters such as the number of latent features ...
0
votes
0
answers
7
views
How to Use Multiple Adapters with a Pretrained Model in Hugging Face Transformers for Inference?
I have a pretrained Llama-2 model in the models_hf directory and two fine-tuned adapters: a summarization adapter in ...
- 23
0
votes
0
answers
9
views
Generating synthetic labeled data (sampling from p(x,y))
I'm working on a toy problem. Consider a dataset that consists of 1-D vectors (waveforms) that contain noise, except for one prominent spike. Denote the waveform by $\vec{x}$, and let the coordinate ...
- 1
0
votes
0
answers
6
views
Can we use the pretrained WavLM on Portuguese?
I wan to try WavLM for 2 tasks:
Speaker Verification
Speech embeddings
If I understand it correctly, the WavLM pre-trained model was trained on English.
so if I want to use it for my missions, do I ...
- 363
0
votes
0
answers
18
views
Different generated patches from original image using vision transformer (ViT)
I am using ViT for image classification, I scaled images in range of [-1,1], and I also padded images. Then, I used the following code to see the original image and generated patches, but the output ...
- 1
0
votes
0
answers
16
views
Why I am getting error in dataloader in defining a NN?
I am trying to write a NN. However I am getting error.
Here is my Code:
...
- 1
0
votes
0
answers
21
views
Parsing response from llama2
I want to extract phone numbers from a given text and i am prompting a llama2 model for that ..I want the output in form of a list but i am getting unnecessary output like sure here are the phone ...
0
votes
0
answers
25
views
LLM GPU Scalability for multiple inferences
New to LLMs and have a question on scalability. Supposing I take a pre-trained open-source LLM and only wish to perform inference (eg. a simple chatbot on a local machine).
If it takes me 2 GPUs to ...
- 1
0
votes
0
answers
20
views
ROC curve for multiclassification - results sound not correct
I'm working on a multiclassification task using LSTM algorithm, i generated my roc curve plots but they give scores like 1 , 0.99, 0.97 however i have an accuracy of 0.97, Precision 0.65, Sensitivity/...
- 11
1
vote
0
answers
14
views
Avoid overfitting to noise by a noise penalty approach instead of early stopping?
I came across this article on deep learning for computational MRI and found an interesting sentence "However, early stopping has to be performed to not overfit to the noisy measurements." ...
- 11
0
votes
0
answers
31
views
3D CNN accuracy is too low, how to improve it?
I have just started learning image processing and this is my first time working on video classification. I am trying to develop a model that recognizes hand gestures using the EgoGesture dataset(more ...
- 1
2
votes
0
answers
25
views
CLIP Visual Transformer image encoder
I was doing some experiments with the CLIP's visual transformer encoder output (clip-ViT-B-32). So basically given the same scene or image, it should output almost ...
- 217
0
votes
0
answers
14
views
Why use sliding window input features in deep learning?
I was reading through the DNABERT paper and found that their input features were k-mers. This is equivalent to using rolling/sliding window features in the other common family of sequential problem, ...
- 131
0
votes
0
answers
10
views
Retraining with the same test data returns different accuracies
I am using Pytorch for training my vit-3d model on my data. I have designed my cross-validation function and trained the model. While running the code file, I am getting different test accuracies each ...
0
votes
1
answer
28
views
What Deep Learning model to use in this spectroscopy task?
I have a task to be solved. There are energy measurements over the square area 40x40. One measurement consists of values : x, y and the energy. The all area is almost whole covered with data (a few ...
- 101
0
votes
1
answer
17
views
Appropriate input size for nn.Embedding
I’m quite new to using Pytorch and deep learning. What size of unique categories of a categorical variable is appropriate for applying the nn.Embedding ideally (best practices)? for example, if a ...
1
vote
1
answer
43
views
Dealing with varying predictive horizon
I know that the predictive horizon is the time window that runs from the observation of the data to the manifestation of the target variable.
But how can I deal with prediction if the time horizon ...
0
votes
0
answers
11
views
Why does the first call to a TensorFlow function execute much slower than the second call?
I was doing an Image Classification problem using TensorFlow. I was generating the mean images for two image datasets having the same size. The dataset was generated using the tf.data API. Thereafter ...
- 77
0
votes
0
answers
16
views
0
votes
0
answers
25
views
Math Behind Additive Bahdanau Attention
I am new to NLP field and wanted to apply attention model in one of my projects. I have LSTM model to train, and concatenate some external data sources though attention mechanism.
The hidden state ...
0
votes
0
answers
17
views
Training with optuna-tuned hyperparameters leads to different results
I'm training an image classifier in Pytorch Lightning and tuning hyperparameters with Optuna. When I use the best hyperparameters to train a separate model, the accuracies differ from those obtained ...
- 33
1
vote
1
answer
47
views
Why my validation loss and accuracy decays over epochs?
Im trying to build 2 simple networks with cleaned dataset for tweets sentiment classification(0/1):
one with all dense layers(binary bag of words)
another with RNN layer(embedding layer).
But it both ...
- 35
0
votes
0
answers
15
views
Image classification of centered objects with convolutional neural networks
Given that I have a set of images that contain multiple objects for which labels exist and the object the image label refers to is always in the center. The objects vary in size. I want to train a ...
- 1
0
votes
0
answers
17
views
Training a two-layer neural network for multi-label data (binary bit array of dim 50)
This is my problem setup.
Train Input size (6300x300) These are standard BERT embeddings, so floating point numbers, mostly negatives.
Train Output size (6300x50) These are binary bit arrays like [0, ...
- 101
-1
votes
1
answer
15
views
Build a topic model without data?
I need to come up with a topic model, without any labelled dataset, the model should also be multilingual, thinking of using LLM's as they are accurate and awesome but if Im to build one on my own how ...
- 35
0
votes
0
answers
10
views
Using auxiliary softmaxes to measure impact of each submodule on the final softmax classifier
I am attempting to assess the impact of various submodules (CNN 1D, CNN 2D, CNN 3D, FFNN) on the final classifier of the neural network that i am currently building. The neural network itself is ...
- 101
0
votes
0
answers
12
views
Riemannian metric in Layer Normalization
I'm reading a paper about Layer normalization, and I couldn't find any clear explanation for this part:
Q1. Can anyone describe the derivation of the first equation in (8)?
Q2. I cannot understand ...
0
votes
0
answers
18
views
How to count number of circular pipes in an image using Deep Learning or Machine Learning?
I'm trying to build a program to count the number of pipes from a given image, here are some example test images.
[![enter image description here][2]][2]
[![enter image description here][3]][3]
I want ...
- 159
0
votes
1
answer
29
views
How did Andrej Karpathy make the LSTM output byte values for sampling Shakespeare?
I'm wondering how continuous output values of deep learning networks are converted to byte values or other discrete values for that sake.
For example here: In his famous article The Unreasonable ...
- 103
1
vote
0
answers
15
views
Can CNNs complete lines and contours?
Are there deep convolutional networks capable of recognizing two overlapping triangles in this image - or is this beyond the capabilities of CNNs?
And are there CNNs that can recogize two boxes ...
0
votes
1
answer
22
views
different range of target values in neural network
I am working on a neural network regression code. The dataset includes 14 features in the range value between -1 and 1. while the target variable is changing among (0.000759) to (1100).
The target ...
- 1
0
votes
0
answers
16
views
How to implement research paper network architecture algorithms
I'm struggling to implement this CNN architecture for this research paper: https://github.com/lindawangg/COVID-Net/blob/master/assets/COVIDNet_CXR.pdf. In Figure 2, the paper has a diagram that ...
- 85
0
votes
0
answers
6
views
Is a csv file to store image path and class neccessary for image classification?
I just get my hand-on a basic deep-learning project. I am working on multi-class image classification project with e-commerce dataset. I am not sure whether by storing training images in sub-folder ...
- 1
0
votes
0
answers
9
views
How should I save data in deeplearning with nosql (mongodb)?
I usually use file system to manage data for my deep learning model, but one of my boss told me to make nosql database to manage data.
Datasets I use have m rows, and n columns of count matrix and ...
0
votes
0
answers
11
views
How to generate synthetic data samples of Raman Spectroscopy by using GAN?
I am working on the Raman Spectroscopy dataset. The wavenumber/frequency range used by Raman spectra starts from 151.25 and ends at 1999.17. These values are used on the x-axis. While amplitude/...
0
votes
0
answers
10
views
Detection of noise and signal components with deep learning
There are thousands of datasets from the signal in the bellow image. With these datasets and machine learning, I want to first detect the jumping points (blue in the image) which are noises, and after ...
- 1
0
votes
0
answers
9
views
How to solve MemoryError problem?
I have audio signals which I need to convert them to melspectrogram. I am using Deepfilternet to remove background noise. when I use the output of Deepfilternet for next phase like padding, it showed ...
- 1
0
votes
0
answers
12
views
Python Darts API
I'm using the darts API for some time series prediction in python, while looking at the documentation for the RNN model I saw this parameters called training length. The description is not very clear ...
0
votes
0
answers
9
views
Why does weight decay produce regularisation in Deep Neural Network?
Weight decay penalizes the model to have smaller weights but how does this help in regularisation?
Any intuition on smaller weights => simpler model?
0
votes
0
answers
11
views
When training deep learning model which is better, training with sampled data Vs. training on shorter epoch
I am running multiple hyperparameter optimization trials therefore trying to find a way to reduce time consumption. Two ways that I could think of are
search hyperparameter on subset of data.
search ...
- 385
0
votes
0
answers
13
views
How would you treat imbalanced training data, and you don't know how test data distribution looks like in deep learning?
I posted this question on another place, but I want to get many tips, so I post here too.
I am building deep learning classification model in bioinformatics. I made training dataset by merging 12 ...
0
votes
0
answers
14
views
Sending rolling statistics to RNN
I'm curious if anyone has seen cases where sending rolling statistics such as mean, median, min, max, standard deviation, skewness, kurtosis, etc. have been helpful for model accuracy?
If so please ...
- 21