Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

Filter by
Sorted by
Tagged with
2 votes
2 answers
571 views

Neural network model for sparse multi-class classifier on Tensorflow

The problem I'm trying to solve is the following: the data is Movielens with N_users=6041 and N_movies=3953, ~1 million ratings. For each user, a vector of size N_movies is defined, and the values ...
0 votes
1 answer
167 views

Train and Validation Curve

I'm new in DeepLearning. I'm not good at understanding and commenting on graphics.Can you help me with these graphs
0 votes
0 answers
10 views

how to get similar traning data for a test sample?

I want to get the most similar samples for test sample on which the model choose it's - output. SHAP isn't useful because it show the contribution of each feature. I want to get the most similar ...
0 votes
1 answer
210 views

How many bounding boxes does the YOLOv6 model predict in total before thresholding?

I understand that the YOLOv5 model predicts 25200 bounding boxes between all 3 levels of output. How many does the YOLOv6 model predict, if the input resolution is 640x640?
0 votes
1 answer
483 views

Tensorflow model works for classification but not for regression (all predictions equal the output layer bias)

I'm trying to build a model for FX prediction. It's giving some promising results for classifying each period as buy/sell/neutral. When used as a classifier, actual returns are converted to 0, 1, or ...
0 votes
1 answer
96 views

A way to init sentence embedding for unsupervised text clustering, better than glove wordvec?

For unsupervised text clustering, the key thing is the init embedding for text. If we want to use deepcluster for text, the problem for text is how to get the init embedding from deep model. BERT can ...
0 votes
1 answer
195 views

Image classification with CNN Python

I'm working on image classification using CNN, my dataset contains more than 50 classes (50 folders) which represent the types of car parts, and in each folder we have vehicle brands, each vehicle ...
1 vote
1 answer
110 views

Working Behavior of BERT vs Transformers vs Self-Attention+LSTM vs Attention+LSTM on the scientific STEM data classification task?

So I just used BERT pre-trained with Focal Loss to classify Physics, Chemistry, Biology and Mathematics and got a good f-1 macro of 0.91. It is good given it only had to look for the tokens like ...
1 vote
1 answer
127 views

Train on multi-domains, then fine-tune on specific domain

Would it make sense to first train a model on images from multiple domains, and then do "fine-tuning" on one specific domain to improve its performance on it? For instance, one could train an object ...
0 votes
0 answers
30 views

Why Latent Space in Stable Diffusion has shape 64x64x3?

I am wondering why the dimensionality of Latent Space in Stable Diffusion is 64x64x3. Since ...
0 votes
1 answer
62 views

Questions about hidden states of bidirectional LSTMs

I read this in an article about bidirectional LSTM: In bidirectional LSTM, each word corresponds to two hidden states, one for each direction. Thus, we concatenate these two hidden states to ...
1 vote
0 answers
48 views

How can I use Time-GPT for pretraining my model

I am mentioning Time-GPT here as a placeholder example. It can be any pretrained model. Suppose I have a dataset that requires some time series prediction. How can I leverage a well-trained model and ...
0 votes
0 answers
16 views

Custom loss function for collinearity of 3 embeddings

I am trying to implement a loss function that takes as input 3 embeddings and output a value that is proportional to the collinearity of the embeddings. This is to shape the latent space of a ...
3 votes
1 answer
118 views

Perceptron Learning Rule

I am new to Machine Learning and Data Science. By spending some time online, I was able to understand the perceptron learning rule fairly well. But I am still clueless about how to apply it to a set ...
0 votes
2 answers
135 views

Is the Cross Entropy Loss important at all, because at Backpropagation only the Softmax probability and the one hot vector are relevant?

Is the Cross Entropy Loss (CEL) important at all, because at Backpropagation (BP) only the Softmax (SM) probability and the one hot vector are relevant? When applying BP, the derivative of CEL is the ...
3 votes
2 answers
723 views

LeNet-5 - combining feature maps in C3 layer

Famous LeNet-5 architecture looks like this: The output of layer S2 has dimension: 10x10x6 - so basically an image with 6 convultions applied to it to derive features. If each dimension was again ...
1 vote
1 answer
423 views

How to get vector representations(or embeddings) of time series?

Even if a time series is constructed up of numbers only, finding abstract fixed-dim vector representation would be interesting for classification/clustering purposes. As we can learn & find ...
0 votes
1 answer
90 views

different range of target values in neural network

I am working on a neural network regression code. The dataset includes 14 features in the range value between -1 and 1. while the target variable is changing among (0.000759) to (1100). The target ...
1 vote
2 answers
184 views

Dynamically remove data from training dataset

I was wondering today if it would be a good approach to remove data dynamically from the training dataset when learning a neural network. Assuming a classification task, the approach would be ...
2 votes
1 answer
243 views

What is the best way to use another classification model as input to my model?

Let’s say I have a classification model. And my job is to predict the correct class out of 30 different classes. The current accuracy is 60%. The thing is: I have to consume another team’s ...
1 vote
1 answer
139 views

Audio Classification with Counter

I'm trying to create a model that can identify one particular sound, and every time it hears that sound, it increases a counter by 1. So for example, if it hears a specific bird chirping ten times, ...
1 vote
1 answer
1k views

Classification of scanned documents in pdf files using deep learning or NLP

I know classifying images using cnn but I have a problem where I have multiple types of scanned documents in a pdf file on different pages. Some types of scanned documents present in multiple pages ...
1 vote
1 answer
59 views

Train Reward Model using Llama2:

this is my code that use to train reward model: ...
0 votes
1 answer
988 views

How to reduce RMS error value in regression analysis & predictions - feature engineering, model selection

There's this dataset containing the metadata of Twitch's top 1,000 streamers of 2020. You can have the details here. I am currently participating in a challenge to predict the values for Followers ...
3 votes
2 answers
2k views

Cross validation for convolutional neural network

I am using Keras to create a CNN model, and I would to use K-fold cross-validation to train the dataset. The dataset contains images and I am using ...
4 votes
1 answer
291 views

Importance/intuition behind stacking RNNs

Nowadays there's a trend towards using architectures of "deep" RNNs i.e. vertically stacked RNNs. RNN chapter from Bengio's bookThese networks seem to work well in practice. What's the intuition ...
0 votes
1 answer
88 views

Does adding of many FC layers during re-training increase the model size ? Are there any ways to optimize the size of model?

I am re-training a pretrained model VGG16. In the last layers, im using two FC layers of size 2048 each, with dropout=0.5. When I saved the model, the size of the ...
1 vote
1 answer
131 views

CNN 3D line angles prediction regression - results of training of phi depend on theta

I am a beginner in "deep learning". What I am trying to do, is to predict two angles of a 3D line projected on a 2D image. The toy model is that I create a line going out from the centre of 48x48 ...
2 votes
1 answer
191 views

Why yolo4 pytorch re-training loss seems high as like first time training?

I had a setup a yolo4 pytorch framework in google colab by cloning git clone https://github.com/roboflow-ai/pytorch-YOLOv4.git. I generated checkpoints by giving ...
1 vote
2 answers
1k views

Vanishing gradient problem even after existence of ReLu function?

Let's say I have a deep neural network with 50 hidden layers and at each neuron of hidden layer the ReLu activation function is used. My question is Is it possible for vanishing gradient problem to ...
1 vote
1 answer
461 views

CNN can't predict images outside the dataset

I am using celeba dataset to train my CNN face landmark detection model. Here is my model ...
3 votes
2 answers
738 views

The difference between data science and algorithm development

I see a lot of job opportunities in the field of data science but I'm not sure the difference between a data scientist and deep learning algorithm developer. Can someone explain that to me?
0 votes
1 answer
208 views

Extract phrases/keywords that are SIMILAR to a python list of keyword/phrases, from a document

EDIT : If I had to match single worded phrases, I could first tokenize the text from the document and then calculate the cosine similarity of all the tokens with all the keywords from the ...
3 votes
1 answer
753 views

Discriminator of a Conditional GAN with continuous labels

OK, let's say we have well-labeled images with non-discrete labels such as brightness or size or something and we want to generate images based on it. If it were done with a discrete label it could ...
0 votes
0 answers
15 views

How to detect abnormal fetal head size with image classification?

I'm writing Python code to predict fetal head circumference 10mm range using classification. The model will train to classify a fetal head image into a range (e.g., 50–60 mm) representing its ...
0 votes
2 answers
138 views

Fast AI Lesson 4 - MNIST. Confused about multiplying weights by pixels?

I’m on lesson 4 of the Fast AI "Deep Learning for Coders" course, and have been back through the same lesson a few times now but I don’t think I’m quite getting a few things. I want to have ...
2 votes
1 answer
149 views

Dimensionality of the target for DQN agent training

From what I understand, a DQN agent has as many outputs as there are actions (for each state). If we consider a scalar state with 4 actions, that would mean that the DQN would have a 4 dimensional ...
0 votes
1 answer
40 views

Anomaly Detection in Log Data using LSTM

Problem Overview: I am currently working on a project involving anomaly detection in log data. The anomalies are defined by deviations from historical patterns. The log data has a simple structure: [...
5 votes
1 answer
1k views

Difference Between Attention and Fully Connected Layers in Deep Learning

There have been several papers in the last few years on the so-called "Attention" mechanism in deep learning (e.g. 1 2). The concept seems to be that we want the neural network to focus on ...
2 votes
1 answer
467 views

What are the differences between BPE and byte-level BPE?

In Roberta, I'm not sure if the model use BPE or byte-level BPE tokenization, are these techniques different or the same ? Can someone explain ? Thanks
3 votes
1 answer
520 views

How to train millions of doc2vec embeddings using GPU?

I am trying to train a doc2vec based on user browsing history (urls tagged to user_id). I use chainer deep learning framework. There are more than 20 millions (user_id and urls) of embeddings to ...
0 votes
1 answer
603 views

Keras CNN model is throwing as error message as 'ValueError: Layer 'conv1d_12' expected 2 variables, but received 0 variables during loading'

Hope you're in good health and doing great. I am trying to implement a CNN model to help predict kidney stones. Now, this model is running as expected on my local machine, but when I try deploying the ...
1 vote
1 answer
147 views

why CNN model can't learn well the peak from data

here I have two different datasets. dataset1 is force plate data and dataset2 is plantar pressure data. dataset1 has shape (2050,2) and dataset2 has shape(2050,89). before doing the training I have ...
0 votes
1 answer
156 views

Should deep layers ever have more units than the input layer?

i.e. if a model, with 10 inputs, say,: ...
1 vote
1 answer
90 views

interpret cnn structure

I am trying to interpret the CNN model from the below settings. AS I am new to deep learning and I am not able to fully comprehend the layer structure . Could someone please tell me is these two ...
0 votes
1 answer
2k views

Epoch 1/5 won't stop

When i run my code with 5 epochs, code gets stuck at first epoch and run continuesly. I tried applying various parameters but couldn't make it. here is my code... ...
2 votes
1 answer
120 views

How to get intuitive understanding which deep learning architecture suits for my problem

I'm working on a research problem where I need to perform classification for coarse prediction in a feature space and then fine grained regression for getting more precise values. I know that this way ...
0 votes
0 answers
28 views

How do I shape my output data for a time series classification problem using LSTM

I am wanting to use an LSTM for anomaly detection on a multivariate time series data. Let's say there are n rows each corresponding to a timestamp incrementing by an hour and d input features and d ...
4 votes
2 answers
2k views

How is attention different from linear MLPs?

Each output for both the attention layer (as in transformers) and MLPs or feedforward layer(linear-activation) are weighted sums of previous layer. So how they are different?
2 votes
1 answer
346 views

Problem when cherry picking actions - Proximal Policy Optimization

I am using the implementation of PPO2 in stable-baselines (a fork of OpenAI's baselines) for a Reinforcement Learning problem. My observation space is $9x9x191$ and my action space is $144$. Given a ...

1
3 4
5
6 7
98