Questions tagged [neural-network]

Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

Model that predicts probability of correctness of another model

Problem: Given a neural network for image classification with $1000$ classes, the objective is to create another model which will output the probability of the neural network giving the correct ...
0
votes
0answers
3 views

Do we calibrate prediction-threshold for a neural network based on prior distribution of each class?

I have a dataset with 4 classes, say their distribution in the training-set is $P_{prior}(C1) = 60\% $ $P_{prior}(C2) = 25\% $ $P_{prior}(C3) = 10\% $ $P_{prior}(C4) = 5\% $ I have trained a CNN over ...
1
vote
1answer
11 views

What happens when the vocab size of an embedded layer is larger than the text corpus used in training?

Full disclosure this question is based on following this tutorial: https://tinyurl.com/vmyj8rf8 I am trying to fully understand embedded layers in Keras. Imagine having a network to try and understand ...
0
votes
0answers
8 views

Siamese Network with Triplet Loss function using the Matlab's Deep Network Designer

How can I design a Siamese Network with Triplet Loss function using the Matlab's Deep Network Designer? I was not able to achieve this.
0
votes
1answer
10 views

Training the network with some batch size - code

There is my "training" code below, I wrote it based on one youtube tutorial. I don't understand actually one part: batch_X = train_X[i:i+BATCH_SIZE], batch_y = train_y[i:i+BATCH_SIZE]. How ...
0
votes
1answer
21 views

How to choose feedforward architecture for few number of features but very large instance?

Assume I have 1 million of data instance and each instance contains 100 feature. For each instance, I also have a lable. The ...
0
votes
0answers
3 views

Using bias rather than weights to train RBM (Contrastive Divergence)

When training an RBM, we used the above to figure out how to update the weights (contrastive divergence). We are now being asked to update the bias and not update the weights to train. The hint was ...
0
votes
0answers
17 views

Learning sequences consisting of single non-zero entry and remaining zeros

Pattern such as [1 0 0 0], [0 1 0 0], [0 0 1 0], [0 0 0 1] can easily be learned by using LSTM. We have created patterns where above mentioned vectors serve as the basis but we reveal one index ...
0
votes
0answers
10 views

Keras Custom Layer calling fit method before compile

Below is the SS of the custom function I am trying to apply on every image of the batch and the custom Layer ...
0
votes
0answers
10 views

How to train a NN with mutliple variables? [closed]

I have a csv file with following content (the full list has more entries): ...
0
votes
0answers
8 views

Keras Custom Layer call Method

I have implemented a Keras Custom Layer in which I call a custom function inside the call method, the issue is when Keras is constructing the model, it is calling my custom layers call function with ...
0
votes
0answers
19 views

Why don't I find a decision making problem and dataset [closed]

I am a master student trying to work on decision support systems and improving their accuracy with machine learning techniques, namely neural nets for now. For 2 months, I have been trying to start to ...
1
vote
1answer
14 views

Slight confusion on the learning process

Hi guys I have a slight confusion on the learning process of neural networks. When the input layer receives inputs, goes through the hidden layers and then into the output layer. How does the neural ...
0
votes
0answers
4 views

Inverse Autoregressive Spline Flow Implementation

I have to implement an algorithm for a university project, however I can not seem to wrap my head around it. The algorithm should be an inverse autoregressive normalizing flow using splines. It should ...
0
votes
0answers
7 views

Using a multi-headed neural network, how should I approach the regression head loss

I have a multi-headed NN where one head performs multi-label classification and the other a regression task on a set of images. The classification head outputs a one-hot vector where each value in the ...
0
votes
0answers
27 views

How does shuffling make data identically distributed?

It makes sense that gradient descent algorithms (like stochastic GD, mini batch GD) work better when we shuffle data. (It makes instances in a dataset independent) (...
0
votes
1answer
14 views

Neural network activation function [duplicate]

im fairly new to neural networks. I want to ask what exactly does the activation output, is it the probability the combined summation of inputs and weights lead to a match for the next neuron? Thanks
0
votes
1answer
19 views

when will the mlp give constant prediction?

I have a regression task.(to predict price for finanical market) I build a mlp to do the regression. I found mlp will stop at giving a constance prediction. which i think it's useless. Does this mean, ...
0
votes
0answers
5 views

Multi Input Network MNIST-CIFAR10

I have the following task of meta learning: We want that our neural network learns to sum weights. 1)Do the training on MNIST, and on CIFAR10 (as support dataset). We want that performance (accuracy) ...
2
votes
1answer
23 views

what is the meaning of independent and identically distribution of samples in a dataset for neuralnetworks

random variables like heads or tail that generated by flipping a coin is independent because each time we tossing the result isn't depend on previous toss( in other ...
1
vote
0answers
20 views

How to use keras custom data generator to build a model

I have the following data generator: ...
0
votes
0answers
9 views

Getting gradient for gradCam in pytorch

I am using forward and backward hook in my pytorch densenet121 model. I set requires_grad to False at the time of training. ...
-1
votes
0answers
16 views

Deep q learning from scratch weights diverge to NaN

I'm trying to make a deep q learning algorithm with neural network from scratch, minbatch gradient descent, replay memory, and target network. But weights diverge to NaN after a around 40 episodes ...
0
votes
0answers
6 views

Matrix dimensions issue on calculating Softmax Derivatives

I am trying to get the matrix dimensions right for computing derivative of a two layers network where the last layer is softmax function. For simplicity I am only interested to get derivatives of W2 w....
0
votes
1answer
11 views

Normalization of possibly not fully representative data

I am trying to train a classification RNN model on a sequence of table medical data, but I stuck with the normalization problem. I realized that I cannot simply use MinMaxScaler, because of 3 problems:...
2
votes
1answer
117 views

What is Inductive bias

Bias in a neural network is an additional neuron to be fired i.e let $y=a+bx$ where a is bias term Do we have any difference between bias and inductive bias. How Inductive bias is helpful in ...
1
vote
0answers
56 views

feature Extraction with mobileNet visualization

I am trying to create a logical visualization regarding how feature extraction with mobile net works in ml5.js. There's a good explanation here: https://youtu.be/kRpZ5OqUY6Y With ml5, you use a part ...
0
votes
0answers
24 views

Is this a task of meta-learning or transfer learning?

I have a task that I am not able to identify if it is of transfer or meta learning. I want to know this, in order to ask help in solving it, because there are some parts that I have not understood. ...
0
votes
0answers
6 views

Choosing the size of the network for Neural Collaborative Filtering (NCF)?

I've been working on Neural Collaborative Filtering (NCF) recently to build a recommender system. After doing some hyperparameter tuning with various sizes for embedding and dense layers sizes, from ...
0
votes
0answers
23 views

Generalization between 2 datasets of same domain

Suppose that I'm have 2 different datasets of same domain and trained a model with d1 dataset. Can we generalize the model to predict d2 dataset ? Is it possible or not. For example consider Dataset ...
0
votes
0answers
10 views

Unused bottleneck neurons in autoencoder

I'm using an autoencoder to compress categorical data, by a factor of about 20x. For this part of my data set, I have roughly 3500 variables, so my final bottleneck size is 180. I'm getting pretty ...
0
votes
1answer
24 views

Applying activation on part of the layer in Keras

Context I am trying to implement the YOLO algorithm in Keras. What I have so far is the following network: ...
0
votes
0answers
18 views

What type of ANN architecture to choose?

I have N number of teachers each of which has an input feature vector (25 dimensional) consisting of positive numerical values for different quality of aspects (for example, lecturing ability, ...
1
vote
0answers
22 views

One Year Ahead Forecasting with Unevenly Spaced Time Series

I have many products in my warehouses which can be "demanded" any day by my different clients. I want to forecast how many of each item will be demanded for the whole next year. Naturally, ...
1
vote
1answer
24 views

Why is the kernel of a Convolutional layer a 4D-tensor and not a 3D one?

I am doing my final degree project on Convolutional Networks and trying to understand the explanation shown in Deep Learning book by Ian Goodfellow et al. When defining convolution for 2D images, the ...
2
votes
0answers
21 views

How does attention for feature fusion works

I am struggling to understand how would a self-attention layer be used for features of different modalities fusion. What I understand until now is that : Every unique modality is fed into a self-...
0
votes
0answers
19 views

Backprop with relu and softmax in matlab

I am trying to implement a network for a classification task and I am kinda struggling with backpropagation. The network should classify MNIST dataset (0 - 9). As a training set, I have 4500 images of ...
0
votes
1answer
13 views

Deep fully connected NN with vanishing gradients

I am making NN for choosing best bets possible for football matches. And I tried to make network quite deep (12 hidden layers with BN between them and ReLu as activation function) but it resulted in ...
0
votes
0answers
9 views

Output shapes of Keras AdditiveAttention Layer

Trying to use the AdditiveAttention layer in Keras. On manual implementation of the layer from tensorflow tutorial https://www....
1
vote
1answer
33 views

Which ANN structure to use?

Let $\mathcal{S}$ be the training input data set where each input $u^i \in \mathcal{S}$ has $d$ features. I want to design a ANN so that the cost function below is minimized (the sum of square of ...
1
vote
1answer
12 views

Why do we need to concatenate in a U-Net?

You might be familiar with the U-Net, a machine learning network deceived for image segmentation. It's basically an encoder/decoder network with some direct links between encoder and decoder segments: ...
0
votes
0answers
14 views

Verifying the implementation of Multihead Attention in Transformer

I have implemented the MultiAttention head in Transformers. There are so many implementations around so its confusing. Can ...
1
vote
0answers
15 views

DQN CartPole-v1 neural network doesn't optimize

I'm doing my first dnq algorithm, I'm trying to build a dnq agent, and neural network from scratch, but it seems that neural network doesn't optimize, I did 2 hidden layers, with ReLU, and the output ...
0
votes
0answers
13 views

How to predict multiple time points in future?

Let's say I have a database of customer's purchase history. So, my data has below info a) customer demographics such as age, gender, country etc. b) customer order history such as order_id, order_date,...
0
votes
1answer
17 views

Dimensions of Transformer - dmodel and depth

Trying to understand the dimensions of the Multihead Attention component in Transformer referring the following tutorial https://...
0
votes
0answers
12 views

Neural Networks with a list of undetermined length as input

I have a list of strings and for each input (each list), there is a target that is again a string. The idea is to let the network learn to generalize from the inputs. Here is one example: ...
0
votes
0answers
12 views

Deep Belief Networks

I understand that there is a stack of RBM's and then a supervised step where the feed forward neural network is initialized with the weights from the RBM. However, what I don't understand is- does it ...
0
votes
0answers
5 views

what would happen in max_pool layer if backprop would add gradient to all inputs for particular neuron but only if it's positive

in max_pool layer ANN performs this operation max([in1, in2, ... inN]), now if gradient that comes back to this layer is ...
0
votes
1answer
12 views

Keras: Provide One-Hot-Encoded input values to neural network

I have a dataframe which has two columns of interest: A and B with string values. I am trying to build a prediction model which takes in a set of values in A as input and predicts the corresponding B ...
1
vote
1answer
19 views

Should your learning rate get smaller as your neural net gets larger?

Is it correct to say that as you add more layers and more neurons, your learning rate should then decrease? So, generally speaking, the larger the net, the smaller the learning rate?

1
2 3 4 5
75