Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

Filter by
Sorted by
Tagged with
1 vote
1 answer
125 views

What do special tokens used for in Roberta?

When I use this code: ...
user avatar
0 votes
0 answers
65 views

IoU metric for multi class image segmentation task

My input shape is of (168,18). I create batches of size 256 and create my dataset using timeseries_from_Array_dataset. I am visualizing this 2D snapshot of a multivariate timeseries (batch size- 256, ...
Vjs's user avatar
  • 13
0 votes
1 answer
37 views

Why was the learning rate decreased for Roberta compared to LSTM?

I'm reading the codebase of a project that uses Bidirectional-LSTM. The learning rate for it is 0.02. Later, someone improved the project by replacing LSTM with Roberta and decreased the learning rate ...
user avatar
0 votes
0 answers
50 views

How to find adjacent neighbours using Python?

Full problem description at stackoverflow I need to find the adjacent neighbours (not necessarily nearest neighbours) to a given point in a multidimensional space. As shown in the screenshot below, I ...
skm's user avatar
  • 101
0 votes
0 answers
37 views

Why is there a difference in Training Accuracy Output, when the training dataset is the same but the validation dataset is different?

I am looking at the output of a multi-class image segmentation deep learning model. I used U-Net to implement this. I am confused about why the training accuracies are different for a different ...
user10529827's user avatar
2 votes
1 answer
206 views

What do these terms mean in the context of Roberta?

When I read articles about Roberta, I often read the terms "transfer learning" and "fine-tuning". Additionally, they also mention "feature extraction". What are the ...
user avatar
1 vote
0 answers
161 views

Why do the Llama 2 weights have eight different files?

I downloaded the weights for Llama 2 (70B-chat). This process created a folder titled "llama-2-70b-chat," which contained 8 files titled consolidated.00.pth, consolidated.01.pth, and so on ...
jskattt797's user avatar
0 votes
1 answer
71 views

What are the differences between Embedding Layer and Roberta Embedding?

I'm reading an article about the Embedding Layer: The Embedding Layer learns word embeddings from raw text. It is initialized with small random numbers and can be learned simultaneously with a neural ...
user avatar
1 vote
1 answer
22 views

Extraction of name from phonetic transcription

I have a use case where I want to extract the name from the phonetic transcription. For example if the phonetic transcription is - “m a j n e j m ɪ z s ʌ m i ɹ z o w ʃ i”, the output should be the ...
Sameer Joshi's user avatar
0 votes
2 answers
111 views

What are the differences between contextual embeddings of Bidirectional-LSTM and Transformer?

A Transformer, like Roberta, can generate contextual embeddings using the encoder part, similar to a Bidirectional-LSTM that concatenates hidden states. What are the differences between them ? Are ...
user avatar
0 votes
0 answers
11 views

how to get similar traning data for a test sample?

I want to get the most similar samples for test sample on which the model choose it's - output. SHAP isn't useful because it show the contribution of each feature. I want to get the most similar ...
user3668129's user avatar
4 votes
1 answer
120 views

The end-to-end Training Process for Knowledge Distillation

I'm a bit confused on the complete training process for Knowledge Distillation. I was reading the Geoffrey Hinton "Distilling the Knowledge in a Neural Network" 2015 paper and some random ...
Chuu's user avatar
  • 43
0 votes
1 answer
40 views

How to optimize my CNN classification architecture

I have this CNN based model architecture that takes an RGB image. Now I'm trying to change it for a color classification case on an object (10 color classes: white, black, yellow, etc). This current ...
Mary's user avatar
  • 217
0 votes
0 answers
19 views

Custom loss function for collinearity of 3 embeddings

I am trying to implement a loss function that takes as input 3 embeddings and output a value that is proportional to the collinearity of the embeddings. This is to shape the latent space of a ...
vlc's user avatar
  • 1
1 vote
0 answers
59 views

How can I use Time-GPT for pretraining my model

I am mentioning Time-GPT here as a placeholder example. It can be any pretrained model. Suppose I have a dataset that requires some time series prediction. How can I leverage a well-trained model and ...
Mohammad Mosiur's user avatar
0 votes
1 answer
70 views

Questions about hidden states of bidirectional LSTMs

I read this in an article about bidirectional LSTM: In bidirectional LSTM, each word corresponds to two hidden states, one for each direction. Thus, we concatenate these two hidden states to ...
user avatar
0 votes
0 answers
33 views

Why Latent Space in Stable Diffusion has shape 64x64x3?

I am wondering why the dimensionality of Latent Space in Stable Diffusion is 64x64x3. Since ...
Renat Abdrakhmanov's user avatar
0 votes
0 answers
15 views

How to detect abnormal fetal head size with image classification?

I'm writing Python code to predict fetal head circumference 10mm range using classification. The model will train to classify a fetal head image into a range (e.g., 50–60 mm) representing its ...
NiStack's user avatar
1 vote
1 answer
95 views

Train Reward Model using Llama2:

this is my code that use to train reward model: ...
Sandun Tharaka's user avatar
1 vote
1 answer
194 views

Is vision transformer (ViT) always better than CNN?

The paper - AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE proposed vision transformer and outperformed CNN-based models in many cases. When it comes to sequential data, we ...
Chuck Liu's user avatar
2 votes
1 answer
843 views

What are the differences between BPE and byte-level BPE?

In Roberta, I'm not sure if the model use BPE or byte-level BPE tokenization, are these techniques different or the same ? Can someone explain ? Thanks
user avatar
0 votes
0 answers
28 views

How do I shape my output data for a time series classification problem using LSTM

I am wanting to use an LSTM for anomaly detection on a multivariate time series data. Let's say there are n rows each corresponding to a timestamp incrementing by an hour and d input features and d ...
Vjs's user avatar
  • 13
0 votes
0 answers
17 views

Bulding Deep Learning model for multiclassification case

I am soo confused i read a lot of information in forumas and still cna't get what is wrong. my data is around 500.000 rows and 32 columns. my target variables consists of 3 classes (0, 1, 2). Hyperopt ...
Shamkhal Mammadov's user avatar
2 votes
1 answer
122 views

What is the difference between hidden states in RNN and Transformers model?

I'm very terrible at NLP and I have searched for these questions but didn't find any answer, my question is, in RNNs, there are hidden states to remember information for processing the next state, and ...
user avatar
0 votes
1 answer
78 views

Why not Back propagate through time in LSTM , similar to RNN

I'm trying to implement RNN and LSTM , many-to-many architecture. I reasoned myself why BPTT is necessary in RNNs and it makes sense. But what doesn't make sense to me is, most of resources I went ...
Amith Adiraju's user avatar
0 votes
1 answer
64 views

Anomaly Detection in Log Data using LSTM

Problem Overview: I am currently working on a project involving anomaly detection in log data. The anomalies are defined by deviations from historical patterns. The log data has a simple structure: [...
Raj's user avatar
  • 1
0 votes
0 answers
25 views

Tile color, shape, and type detection altogether

I'm trying to apply some detections/classification on the set of tiles we have. Specifically, I need to detect color (15 classes), pattern (25 classes- on the surface of tiles, there can be certain ...
Mary's user avatar
  • 217
0 votes
0 answers
23 views

Clarifying the arguments of "Understanding the difficulty of training deep feedforward neural networks"

I made the decision to try to push through the paper "Understanding the difficulty of training deep feedforward neural networks". (The paper is given as a reference in "Hands-On Machine ...
Chris's user avatar
  • 101
0 votes
0 answers
10 views

Is it fair to say that Hausdorff Distance (HD) focuses on low level details while dice score (DSC) high level

I wonder if its make sense to say that Hausdorff Distance (HD) measures low-level details while dice score (DSC) focuses on high levels. If you could cite a paper, I would appreciate it.
user836026's user avatar
1 vote
2 answers
131 views

Seeking Guidance on Constrained Input Modeling for Soil Moisture Correction Using Rainfall Observations

I find myself immersed in the intricacies of working with 2D modeled fields (images) representing soil moisture in regions where direct observations are unfortunately absent. However, there is a ...
Seyed Omid Nabavi's user avatar
0 votes
0 answers
9 views

How to calibrate IMU for large scale deployments possibly using deep neural network

We were testing our visual SLAM algorithm on robots. We were getting poor performance. Then we calculated wite noise and random walk parameters (using kalibr) for the IMU and used it in our algorithm ...
Mahesha999's user avatar
1 vote
1 answer
55 views

text extraction from bank statements from pdf format

I have bank statement memos containing transaction tables I need to extract. I only need to extract the transactions list. I have tried to use the Amazon text extractor, LayoutLM but since every bank ...
Abduhoshim's user avatar
0 votes
0 answers
23 views

why does my multi-modal model can not learn anything?

I have a multi-modal model. I want to train it using the Pytorch Framework. I have a balanced dataset. I have approximately 150 samples for each client. (I had preprocessed my text data.) when I train ...
arcane_data's user avatar
0 votes
1 answer
60 views

How to automate the restarting of training of deep learning model in TensorFlow

I am trying to automate the (recursively) restart of a finished deep-learning training session in TensorFlow. Currently, to restart I am manually restarting my kernel and re-running the training code. ...
user10529827's user avatar
1 vote
1 answer
26 views

Building a CNN (with Keras for pixelwise classification)

I have a set of 120x120 input images with 3 channels. I want to build a basic CNN to predict the value of each pixel. I have 2 doubts. One is regarding the last layer - should be a Dense layer, or a ...
Filippo Nunes's user avatar
0 votes
1 answer
58 views

batch Normalization and Layer Normalization difference

In Batch Normalization, mean and standard deviation are calculated feature wise and normalization step is done instance wise and in Layer Normalization mean and standard deviation are calculated ...
April's user avatar
  • 1
0 votes
0 answers
19 views

is concatenating embeddings of different dimension as input to two tower model valid?

I'm trying to build a two tower retrieval system for recommender system. Sudden question popped into my head, when concatenating all embeddings then sending it off to dense layer, does embeddings with ...
haneulkim's user avatar
  • 469
0 votes
1 answer
39 views

Object localization and text extraction using VGG

I'm new to Computer Vision and training a TensorFlow neural network using VGG16. The problem is quite simple: I'm training in a custom dataset to detect and localize numbers in a 100x100 image. The ...
zoddin's user avatar
  • 1
0 votes
0 answers
35 views

GAN/DC-GAN isn't converging

I've been trying to train a vanilla GAN(for MNIST) for a few days, and nothing works. I've tried a lot of different layers, hyperparameters, and more, but every time the discriminator's loss decreases(...
Complex's user avatar
0 votes
0 answers
11 views

Why do all the resulting curves from my function combining N-many random ReLUs look like quadratics?

I've written a function that generates a sum of N-many RelU functions, with random slopes and activation points. I was expecting these resulting functions to be arbitrary, random curves, but for some ...
Marco Acea's user avatar
1 vote
1 answer
114 views

Is this the correct way to calculate word embeddings using Roberta?

I'm trying to write a program that using Roberta to calculate word embeddings: ...
user avatar
0 votes
0 answers
49 views

RLHF fine-tune llama2 in vertex ai

I have fine tune RLHF with Vertex AI Pipeline. But deployed model not showing in model registry. Why? code i have used: ...
Sandun Tharaka's user avatar
1 vote
1 answer
79 views

How to choose (mean, std) for normalization in transfer learning?

I'm working on transfer learning based on ResNet50 pretrained model. Basically, I remove the last layer of ResNet50 and add new head layer. Then I train the model with my image dataset. Obviously, my ...
Morgan Cheng's user avatar
0 votes
1 answer
38 views

How to prevent update a pretrained model if a model is optimized with backpropagation in Pytorch?

I use Pytorch exclusively to develop my model, and these are components in my model and how it works: A generator An encoder: a pretrained, and should not updated. A loss function. Input is passed to ...
Jesse's user avatar
  • 101
0 votes
1 answer
27 views

Where can I get 5000+ classified images of zoo animals? [closed]

please help! We are college students doing this for a project. The project is using neural networks and want to build a model that takes in an input of a colored image of an animal and outputs the ...
user90061's user avatar
0 votes
1 answer
68 views

Training Loss for Classification Model Isn't Decreasing

I'm currently building a video classification model for engagement detection but I'm having some trouble training it. The model takes in two tensors as inputs: a 10x48x48x1 tensor which holds a stack ...
snowball's user avatar
1 vote
1 answer
32 views

Handle multiple categorial features in character level RNN

I am working on a fantasy name generator and I have 2 auxiliary categorical features (gender and race). I initially tried concatenating their one hot tensors directly into the input tensor (I think it'...
Shubham Patel's user avatar
1 vote
1 answer
234 views

Understanding the concepts of word embedding in GPT-2

I have a program that calculate the word embedding using GPT-2 specifically the GPT2Model class: ...
user avatar
1 vote
0 answers
26 views

Learning curve dip after plateau when adding more samples

I have a questions regarding my keras machine learning model. Context: I am working on elementary particle physics, specifically LHC related data. I am training a regression model of 4 Dense layers ...
helton_arruda's user avatar
1 vote
0 answers
60 views

Could someone help with fine-tuning dolphin-2.2.1?

Could someone help with fine-tuning dolphin-2.2.1? I have a problem with training: my train\loss - 0 and validation\loss - 0.000... after 800-1000 steps and this is overfitting ...
kabba62's user avatar
  • 11

1 2
3
4 5
98