Questions tagged [deep-learning]

a new area of Machine Learning research concerned with the technologies used for learning hierarchical representations of data, mainly done with deep neural networks (i.e. networks with two or more hidden layers), but also with some sort of Probabilistic Graphical Models.

Filter by
Sorted by
Tagged with
2
votes
1answer
41 views

Why do Transformers need positional encodings?

At least in the first self-attention layer in the encoder, inputs have a correspondence with outputs, I have the following questions. Isn't ordering already implicitly captured by the query vectors, ...
0
votes
0answers
7 views

Reduce Training steps for SSD-300

I am new to deep learning and I am trying to train my SSD-300 (single shot detector) model which is taking too long. For example even though I ran 50 epochs, it is training for 108370+ global steps. I ...
0
votes
0answers
15 views

How does state information get transferred when predicting with LSTM

I have a typical mutivariate time series forecasting problem that I want to solve using an LSTM, with mutliple features in the input sequnce and one feature in the output sequence. If I train my LSTM ...
0
votes
0answers
12 views

Convert a simple NN model to LSTM model (made using Tensorflow.compat.v1)

I have a simple neural net which I would like to convert to an LSTM model. Can someone please help me with the code? The following code contains the neural net part: ...
0
votes
0answers
12 views

Architecture for ConvLSTM

I have an input data with 2000 samples each having shape of (5, 3, 178, 178) where 5 is time dimension, 3 is a color channel, and the rest are x and y-axis. Now I want to use ConvLSTM layer to predict ...
0
votes
0answers
12 views

Is an output layer with 2 units and softmax better than one with 1 unit and sigmoid for binary classification using LSTM?

I am using an LSTM for binary classification and initially tried a model with 1 unit in the output(Dense) layer with sigmoid as the activation function. However, it didn't perform well and I saw a few ...
0
votes
0answers
28 views

How to train a policy and a value network, implementing alphazero at chess

So, I'm trying to implement alphazero's logic on the game of chess. What I understand so far of the algorithm is: Load 2 models, one of which is the best model you have so far. Both these models have ...
0
votes
1answer
20 views

Working of Dense Layer

What kind of operation does Dense Layer perform to reduce dimemsion. So basically I have used Dense layer to compress the dimension all the time like from 10000 neurons to direct 2000 neurons or even ...
0
votes
0answers
14 views

LSTM - unable to get a 3D output

I have an array with shape (55834, 250, 30) and I'd like to get an output of the same shape from this LSTM model. ...
0
votes
1answer
47 views

Error during the compilation of a neural network in Vitis AI, "Not found op in super_const_dict: name: Decoder_Section_1_UpConv_1/kernel"

I'm following a Xilinx Tutorial about the implementation of a Neural Network in a System on Chip (ARM Processor + Xilinx FPGA) and I have come up with an error during the compilation step. I've ...
0
votes
1answer
20 views

How do I feed my keras model in batches?

I am trying to feed a Sequential model in batches. To be reproducible my example, suppose my data is: X=np.random.rand(24,432) Y=np.random.rand(24,432) My goal is ...
0
votes
0answers
22 views

How to decide number of hidden layers and number of neurons for Autoencoder for dimensionality reduction function?

I have been looking into deep learning and what caught my attention is the implementation of Autoencoder as a dimensionality reduction function for anomaly detection. I found out about it through the ...
1
vote
2answers
37 views

What effect repetitive data will have on the performance of the model

I understand that my question is very broad and that the correct answer may depend on various things. I want to get an idea in general what we may expect if we have repetitive data in our dataset. ...
0
votes
0answers
86 views

Huggingface - TypeError: 'TensorSliceDataset' object is not subscriptable

I'm trying to make my own model for translate a language to another with T5ForConditionalGeneration and Huggingface using no pretrained model (I need to use my own dataset and tokenizer because no ...
0
votes
0answers
8 views

Soft-NMS during Inference

I have read the paper about the soft-NMS and the problem that tries to solve. Does it make sense to use this algorithm during inference apart from training? It smooths the predictions on the bounding ...
0
votes
0answers
21 views

Predict value from array (Assisted Sequence Prediction)

TL; DR How to develop a machine learning implementation (in python) that given an array of values is able to predict its maximum percentage of increase? [11.082, 11.052, 11.052, ... 11.202, 11.202, 11....
0
votes
0answers
9 views

Custom vocabulary with Transformer model in Trax

I'm currently working on a programming language translation problem (NMT) using a transformer model with Trax. So I basically need to convert a language A to a language B. In order to tokenize the ...
2
votes
1answer
31 views

Data Art/ Data Visualization Art/ Information Art

Few days ago, I learned about data art/ data visualization art/ information art. I think I have interest in it. I want to see how I can use my data science skills in this area. However, I don't know ...
1
vote
0answers
16 views

Can a reformer model really handle long-range dependency?

I read this article about new attention model called Reformer. Here is the main strength of this model: The Reformer pushes the limit of longe sequence modeling by its ability to process up to half a ...
2
votes
1answer
50 views

Is the explained variance a good metric for autoencoders?

I want to evaluate how an autoencoder will perform on my data. Now, I can do this with the mean squared error of the decoded data compared to the original data, and this is fine when comparing this ...
0
votes
0answers
27 views

How to use Softmax and CrossEntropyLoss in a classification problem?

I am new in the deep learning field and I would like to go in to understand some concepts. I started learn with pytorch and I have the next question: I constructed my own dataset that contains images ...
1
vote
1answer
32 views

Robustness vs Generalization

I don't quite understand the difference between robustness and generalisability in relation to image processing (CNN). If my model generalises well, it is also robust to changes in the image material. ...
1
vote
0answers
19 views

Should I reshuffle the training set when benchmarking neural networks?

I'm trying to set up a fair benchmarking between various RNN models, where each of them is trained until convergence with a fixed random seed. Because the task is very costly, I am only able to run ...
0
votes
0answers
11 views

How to represent source code in NLP for tasks like code retrieval?

How to represent code features in NLP? Can code be treated like a language and pour into the neural network? Are there some work, paper or material around this topic?
0
votes
0answers
21 views

how to in enhance A3C entropy?

I'm trying to implement this A3C code in my custom environment, and I have a basic understanding of the algorithm. The algorithm worked, but it did not give me a good performance. I looked into ...
0
votes
0answers
12 views

How many layers/ convolutions does DensNet-169 have?

I am a little confused by the numbers. From the Digram in the paper I count 312 convolutions without the transition layers. Can anybody help understand how the 169 in the name is calculated?
0
votes
0answers
17 views

How can I create an XOR Gate using two hidden layers and one output layer (besides the or gate)?

Here is my nueron class with activation function (sigmoid), fn for printing truth table and inputs Mathematically and_gate + nor_gate + nor_gate are giving me the correct output. But the function is ...
0
votes
2answers
61 views

High accuracy in mode.fit but low precision and recall. Overfit? Unbalanced? Error?

Hello ive been training a CNN with keras. A binnary clasificator where it says if a depth image has a manhole or not. Ive labeled manually the datasets with 0 (no manhole) and 1(it has a manhole). I ...
1
vote
0answers
11 views
7
votes
2answers
131 views

Why deep learning models still use RELU instead of SELU, as their activation function?

I am a trying to understand the SELU activation function and I was wondering why deep learning practitioners keep using RELU, with all its issues, instead of SELU, which enables a neural network to ...
1
vote
0answers
21 views

How do dilated convolutions used for upsampling the inputs in FCNs?

I am reading the paper (Long et al., 2015) on fully convolutional networks (FCNs), and I came across the section where the authors describe dilated convolution as the trick to compensate for the cost ...
0
votes
1answer
19 views

predict a binary vector of size 40

I have a dataset of shape (2600, 95) with first 55 columns are features and 40 columns are label. Label is a binary matrix of size 10x4 that flattened, and features are real valued numbers ranging (0....
1
vote
1answer
46 views

Splitting the dataset manually for k-Fold Cross-Validation

I manually divided the dataset into three sets: train, test, and validation. Each set includes several folders, one for each patient. Each patient has many images from a different point of view. As a ...
0
votes
0answers
15 views

How to improve accuracy? BERT

Dataframe: ...
1
vote
0answers
11 views

Confused about Generalized Zero shot learning Validation dataset creation

Sorry for the naive question but I am really struggling to understand how to create the Validation dataset in the case of GZSL. For Zero Shot learning, I can imagine we pick certain classes from the ...
0
votes
0answers
10 views

Why does my Yolov1 output tensor contains negative value?

I have been working on my own imlplementation of the Yolov1 model. The first thing I want to mention is that It seems to be learning. Here is the train/val curves : (<0.1 train and 0.26 val loss) ...
0
votes
0answers
79 views

Input Features of a Hierarchical Structure

I have input features of a hierarchical structure. Each feature consists of a header element and 0 to n subfeatures of the same structure. Also, there is no upper limit for n and n can be different ...
0
votes
0answers
15 views

Differentiation of Learning Capabilities of Different Networks

I have a conceptual problem regarding the overall learning capability of a neural network differentiated by the different types of input that we can give to the network. Suppose that we have a ...
1
vote
0answers
29 views

Why are the results from the keras validation split different from sklearn metrics?

I am training a Keras model, and running: model.fit(X, y, epochs=10, validation_data=(X_test, y_val)) I'm using AUC, precision, and recall as the metrics (also ...
1
vote
1answer
135 views

Difference between ReLU, ELU and Leaky ReLU. Their pros and cons majorly

I am unable to understand when to use ReLU, ELU and Leaky ReLU. How do they compare to other activation functions(like the sigmoid and the tanh) and their pros and cons.
0
votes
1answer
24 views

How to reduce RMS error value in regression analysis & predictions - feature engineering, model selection

There's this dataset containing the metadata of Twitch's top 1,000 streamers of 2020. You can have the details here. I am currently participating in a challenge to predict the values for Followers ...
0
votes
0answers
38 views

Is there any mathematical basis for distributed training of a neural network on multiple machines?

Let us say that I have 50 people download a program that engages in supervised training of a neural network I designed. The data samples being provided are pretty random and the correct outputs are ...
0
votes
0answers
15 views

when depthwise separable convolution should be preferred over normal convolution?

As a novice in the realm of deep learning, I recently learned about Depthwise Separable Convolution. I have seen some tutorials and articles about it on internet, and in all of them the author ...
1
vote
1answer
32 views

Trained model performs worse on the whole dataset

I used pytorch as the training framework and the official pytorch imagenet example to train the image classification model with my custom dataset. My custom dataset has 2 different label (good and bad)...
0
votes
0answers
36 views

Can Adagrad or Adam be used in loss function with l1-norm regularization?

there is one question for me. I want to know that how Adam or Adagrad treat l1-norm regularization in loss-function? (e.g. Lasso) I know that l1-norm is not differentiable function at zero but we can ...
0
votes
0answers
9 views

How does 1 x 1 convolution operation help in finding cross channel patterns?

1 x 1 convolutions are very popular and I see that they are mostly used as bottlenecks. They help in dimensionality reduction which is why architectures like GoogLeNet use it. Xception net uses it for ...
1
vote
0answers
15 views

Precision and Accuracy of a custom Object Detection Models usind networks from TensorFlow Model Zoo

I am trying to develop a model with three classes. To do so, I tried to develop a model with different combinations of the data samples in each class. For example: the $1^{st}$ model has 500 images ...
2
votes
0answers
28 views

Multi-arm bandits

I would like to model a problem as a multi-armed bandit problem. In the data we have contextual information (user demographics, preferences, etc.) but this contextual information of each user is not ...
0
votes
1answer
34 views

after overcoming the overfitting, how to increase training accuracy?

I am building a CNN using keras for a classification task. I started with a simple model as a starting point and as almost all ML problems go, especially if the dataset is not very big, I faced an ...
2
votes
2answers
143 views

Convolutional Neural Network for Signal Modulation Classification

I recently posted another question and this question is the evolution of that one. By the way I will resume all the problem below, like if the previous question didn't ever exist. Problem description ...

1 2
3
4 5
84