Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

Filter by
Sorted by
Tagged with
0 votes
0 answers
20 views

Feedforward Deep neural network

Is there anyone capable of converting this diagram? Please see the image description provided. Similar to this
Veertud Tv's user avatar
0 votes
0 answers
35 views

Feedforward Deep neural networks

Hello everyone can you help me to create a diagram for these F-DNN ...
Veertud Tv's user avatar
1 vote
1 answer
83 views

activation=tf.keras.activations.relu vs activation='relu'

Both models are for binary classification problems Model 1 ...
Justin Jonany's user avatar
0 votes
1 answer
37 views

Confusion with tensorflow's Sequential Dense Layers

I'm working on a regression probem using Tensorflow, and have created two models with slight differences in their first Dense layer. The Models ...
Justin Jonany's user avatar
6 votes
1 answer
628 views

How high of a correlation coefficient of a feature with a target variable is considered too high?

Currently my classification model is doing too well on all of the train, validation, and test datasets. I'm assuming there is a data leakage in the features, and therefore I've computed the ...
haneulkim's user avatar
  • 469
0 votes
0 answers
11 views

Measuring Product Search effectiveness

I want to measure the effectiveness of my search engine, one of the ways i can do that is by measuring the rate at which a customer reformulates the previous query. Hence, I need to quantify inter-...
ricardo's user avatar
  • 23
0 votes
0 answers
43 views

How was the word2vec model trained?

Let's take the CBOW (continuous bag of words) model as the example. Suppose that, there are $c$ context words, each of which is a one-hot encoding vector. So the total number of elements of input ...
J. Doe's user avatar
  • 56
0 votes
0 answers
97 views

Can't overfit Transformer Encoder

In the below code I am trying to train a very simple Transformer Encoder model to basically do nothing with its input. Giving some arbitrary input vector x, the aim of the model is then to output that ...
SeñorDavid's user avatar
0 votes
1 answer
90 views

Neural Network for binary classification not working

I have made a neural network that was working correctly as a multi-class classifier, but after changing the loss and the activation function, plus the output layer to just 1 neuron, it is not working ...
alex martinez's user avatar
0 votes
0 answers
15 views

Do ML model measurements and validation standards (e.g. NIST, ISO) exists for the finance, healthcare, and technology industries? Provide citations

Normally, for example, we talk about splitting datasets into training and test datasets. But. The splitting % per train and test sets happens in a subjective manner. Sometimes. The train is 60% or 70%,...
Full Array's user avatar
0 votes
0 answers
45 views

Competition test set performance much lower than validation set

We are a team of 3 participating in a university competition for a deep learning course. The competition involves a binary image classification task where we have to predict leaf diseases on a (5200, ...
Fiorenzo Fiorenzi's user avatar
0 votes
0 answers
33 views

Should non-trainable functions be part of a nn model?

Some explanation for the somewhat obscure title: I want to train a model which can produce images given some input data. However, actually I want the model to learn some abstract representation which ...
Roland Deschain's user avatar
5 votes
1 answer
506 views

What do we mean by optimizer.zero_grad()

This should be a simple question. But it is vague to me. What do we mean by optimizer.zero_grad(). Consider SGD as an example: $W^{t+1}= W^{t}- \lambda g_t$. Which one becomes zero for each batch. It ...
Ali.A's user avatar
  • 71
0 votes
0 answers
29 views

Test accuracy is very low, compare to Trian and validation accuracy for image classification for 400 class

I am working on image classification with 400 class , during training , I am getting good training and validation accuracy , but test accuracy is approximate 0-1% .My input image is 1 scale , with ...
NeelPatwa's user avatar
  • 101
0 votes
1 answer
148 views

Adding multi-image context to a CNN

I'm looking for an approach to classify a similar dataset to the exposed next. Let's say we have an image with some elements inside it (imagine a large building footprint with several structures). ...
Alejandro Graciano's user avatar
0 votes
1 answer
44 views

How can I approach this transactions data problem?

I am trying to approach the following problem: Imagine that I am a bank and I have a dataframe of transactions that customers make, the columns that this dataframe has are transaction date, customer ...
Sebastian Nin's user avatar
0 votes
2 answers
22 views

changing to gray scale

I want to transfer X-ray data images to grayscale in this code ...
user155950's user avatar
0 votes
0 answers
23 views

Why are low probabilities problematic for knowledge destilation?

Recently, I have been reading the Knowledge Distillation paper (Distilling the Knowledge in a Neural Network) and I have two main questions: Neural networks typically produce class probabilities by ...
Amir Jalilifard's user avatar
0 votes
1 answer
52 views

PyTorch ResNet implementation's Training Loss increasing with every Epochs

I'm implementing a ResNet network from scratch using PyTorch. This network is unique to my requirements, since I need to perform Image Classification for Satellite Imagery with 14 different channels ...
Gamma-ray-burst's user avatar
0 votes
0 answers
23 views

Can a multivariate MIMO LSTM forecaster learn the relationships between the multiple feature outputs?

Question: Can a multivariate MIMO LSTM learn the relationships between the multiple feature outputs? This question arose when I decided to modify a multivariate (Multiple Input - Single Output, MISO) ...
Zezimabig's user avatar
0 votes
0 answers
22 views

How to choose the correct NN model if the metrics are different in training and test time?

I am trying to build an LSTM model which has a lot of Dropout and Batch Norm Layers. When I run model.fit, the accuracy comes out to around 0.7 on the training data....
Jeffrey Davidson's user avatar
0 votes
0 answers
10 views

Align Vectors are Easy to Learn?

I have three vectors $x,y_1,y_2\in\mathbb{R}^{n\times 1}$, where $x=y_1$, $x\perp y_2$. If I use $x$ as input of a 2-layer perceptron, will regressing $y_1$ be easier than $y_2$ (i.e., when fully ...
Duber's user avatar
  • 1
0 votes
0 answers
49 views

Doubts on a custom loss function for regression problems

From what I read, I know we don't use log loss or cross entropy for regression problems. However, the entire logic behind binary cross entropy(say) is to firstly squeeze the y_hat between 0 and 1 (...
the_he_man's user avatar
0 votes
0 answers
13 views

【NLP】Is there a model or task that determines contextual similarity?

I am trying to work on an engagement detection task in which I have to determine if a student is engaged in class. I am looking for an NLP approach where I can calculate the similarity score of a ...
Leo's user avatar
  • 1
0 votes
0 answers
24 views

decision tree limitation VS deep learning

I wonder if decision trees (and their derivatives like Random Forest and Gradient Boosting) have interpolation power as deep learning based model. Most of my experience is with deep learning model. ...
user3197748's user avatar
1 vote
1 answer
44 views

How to use additional features in image captioning?

I have the following question - is it possible to train a model based on Transformer architecture to use additional attributes to generate a caption for an image? For example, I have a dataset with ...
Jeremy Cuberian's user avatar
0 votes
1 answer
62 views

Deep Q-Learning: How are network parameters updated, and why consider episodes in the first place?

I'm trying to wrap my head around the implementation of deep $Q$-learning, and why we even consider episodes in the first place. The usual set-up is that we initialize some starting state $s_0$, then ...
infinitylord's user avatar
0 votes
0 answers
17 views

Which image classification methods/models could suit my (product) image classification problem?

Say you are a potato chips company. The goal is to have consumers upload images of the product they are having issues with and be able to identify the product by brand/variant using machine learning. ...
dataengineer22's user avatar
1 vote
0 answers
31 views

ML paper reproducibility

How can I reproduce results in an ML paper if I don't have the identical resources to train the models as in the paper ? (in my case I only have a laptop spec NVidia gpu and in most of the papers I ...
okm02's user avatar
  • 11
0 votes
0 answers
14 views

Meaning of mean squared error in multistep prediction

In multistep prediction with LSTM(keras), say we had this kind of result: target = [[1,2,3] ,[4,5,6] ] predictions = [[1.1,2.2,3.3] , [4.4,5.5,6.6]] When we choose mean_squared_error as the loss ...
the_he_man's user avatar
0 votes
0 answers
25 views

3D Design file labelling and classification for manufacturing

I have ~1 million 3D design (.STP and/or .OBJ) files of various parts for medical devices, aerospace, automotive or defense systems. I'd like to label them based on appropriate manufacturing methods ...
rootcage's user avatar
0 votes
0 answers
8 views

Has someone designed a neural network which can select its own activation functions and/or have multiple activation functions in one model?

I'm wonder if there are any papers or implementations where a neural network has multiple activation functions in a single model (and layer), and preferably also where such activation functions ...
BigMistake's user avatar
0 votes
1 answer
49 views

Understanding Multi-headed Attention from architecture details

I've a conceptual question BERT-base has a dimension of 768 for query, key and value and 12 heads (Hidden dimension=768, number of heads=12). The same is conveyed if we see the BERT-base architecture <...
Namburi Srinath's user avatar
2 votes
1 answer
221 views

Role of stateful parameter vs shuffle parameter in LSTM keras

I'm trying to make prediction on a multivariate time series using LSTM. I know stateful=True in keras LSTM means state(hidden) of each sequence, in a batch, at index i - is passed to the next batch, ...
the_he_man's user avatar
0 votes
0 answers
14 views

How is it called when instead of creating predective models finding patterns in observed data (ML) you tried to guess the model theorically...?

I'm a college student appasionated of machine learning and I've decided to my bachelor thesis about it. I thought that as an interesting introduction to machine learning, I could introduce it by ...
ADayWithoutRain's user avatar
0 votes
1 answer
95 views

how to define a linear function WX+b in pytorch?

I am practicing pytorch. I want to define a linear function Y=WX+B for inputs shape as (3,32,32) and output, the same shape i.e. (3, 32, 32). I defined m network as: ...
Ali.A's user avatar
  • 71
0 votes
0 answers
17 views

Cost function looks like the real math which is responsible for actually working on out problem statement If I talk on a whole and on a surface level?

Looking at the cost function for say linear regression, apart from changing the weight or the parameters, the cost function does the real job, right? If it is correct, what does cost function do in ...
Gurjot Singh's user avatar
1 vote
1 answer
808 views

issue loading the ckpt file PytorchStreamReader failed reading zip archive: failed finding central directory

I am trying to load the ckpt file and getting error PytorchStreamReader failed reading zip archive: failed finding central directory Here is the code ...
Shruti's user avatar
  • 35
0 votes
0 answers
22 views

Model Performance not improving

I am currently working with a GNN (a Graph attention Model) based model and the main task is to do Graph prediction. My model doesnot improve its performance when I change the number of heads or the ...
Susan's user avatar
  • 1
0 votes
0 answers
34 views

RetNet Paper Multi Scale Retention dimemsion question

From the paper: https://arxiv.org/pdf/2307.08621.pdf But since X is of size n by $d_{model}$. How can we compute $XW_Q$? Since the row length of X which is $d_{model}$ is not the same as the column ...
KaizerBox's user avatar
0 votes
0 answers
28 views

Using Embedding For Regularization

Is using embeddings for regularization a valid practice? My reasoning for that is that encoding training/tests datasets into smaller vectors would allow a smaller network with fewer parameters and ...
Adenilson Arcanjo's user avatar
0 votes
0 answers
85 views

Alternative to ELU and Leaky ReLU?

I was talking with a friend about different activation functions (we are still new to ML). One thing that I didn't like about ELU was the vanishing gradient, and about Leaky ReLU that it's not ...
Nasa's user avatar
  • 1
0 votes
0 answers
82 views

how to implement federated transfer learning?

I'm exploring the concept of Federated Learning and Transfer Learning and am interested in combining both to implement Federated Transfer Learning. I understand that Federated Learning allows model ...
Rezuana Haque's user avatar
0 votes
1 answer
797 views

why I got TypeError: linear(): argument 'input' (position 1) must be Tensor, not int in NN?

I am writing a NN in pytorch. I have a list of tensors as input i.e. I created a list Y by appending 1000 tensor vectors(linear tensors) of size 3072. So, each Y[i] is a linear tensor of size 3072. ...
Ali.A's user avatar
  • 71
0 votes
0 answers
18 views

How to define a DataLoader or Loss for a e.g. multivariable functions?

I am trying to write a NN for estimating a f:R^n --> R^m. My problem is how to train network. I mean if I want to define a dataloader, how to attach X \in R^n to its related Y \in R^m ? Because ...
Ali.A's user avatar
  • 71
0 votes
0 answers
30 views

Is it the right approach to select the model when it gives highest accuracy on validation dataset?

I am training the Densenet121 Model on an image dataset. I divided the dataset into 80% for training and 20% for testing. Then I further divided the training data into 85% for training and 15% for ...
Dawood Ahmad's user avatar
0 votes
0 answers
30 views

How to handle different sequence sizes during training and production use?

My training data consists of sequences that are mode of tokens. The objective of my model is to predict score for each token in the input sequence. This score depends on the particular token itself, ...
Druudik's user avatar
  • 101
0 votes
0 answers
42 views

How to work with multiple feature types on autoencoder?

This is my first post here. I am working on an adversarial autoencoder that receives different features, encodes them, and decodes them. For instance, suppose you have a dataset from a large survey ...
Umberto Mignozzetti's user avatar
0 votes
0 answers
22 views

What are the most important hyperparameters to tune to optimize the training of a 3D U-NET used for pixel classification?

Leaving aside the training loss, the optimizer (ADAM), the number of U-NET blocks (tuned to a meaningful target receptive field for the problem) and the number of filters per layer (set to the maximum ...
Sebastien's user avatar
1 vote
1 answer
55 views

Deep learning model produces very different results when classifying the same samples

I'm trying to design a simple deep learning application for biometric system verification, but every time I run the application I get very different results and I can't figure out why. I don't use ...
uuR's user avatar
  • 11

1 2 3
4
5
98