Questions tagged [neural-network]

Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.

1,330 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
8
votes
2answers
229 views

How to implement hierarchical labeling classification?

I am currently working on the task of eCommerce product name classification, so I have categories and subcategories in product data. I noticed that using subcategories as labels delivers worse results ...
8
votes
3answers
2k views

Loss Function for Probability Regression

I am trying to predict a probability with a neural network, but having trouble figuring out which loss function is best. Cross entropy was my first thought, but other resources always talk about it in ...
6
votes
0answers
186 views

How does one derive the modified tanh activation proposed by LeCun?

In "Efficient Backprop" (http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf), LeCun and others propose a modified tanh activation function of the form: $$ f(x) = 1.7159 * tanh(\frac{2}{3}*x) $$ ...
6
votes
0answers
86 views

Training value neural network AlphaGo style

I have been trying to replicate the results obtained by AlphaGo following their supervise learning protocol. The papers specify that they use a network that has two heads: a value head that predicts ...
6
votes
1answer
254 views

Training stateful LSTM with different number of sequences

I'm using a stateful LSTM for stock market analysis, and I have varying amounts of data for each stock, ranging from 20 years to just a few weeks (i.e. for newly listed stocks). I use 3 years of data ...
6
votes
1answer
138 views

How to model segmentation of a sequence to similar parts?

I guess LSTM is good for sequence modeling but how would you model "clustering" with it? Meaning, the input is a sequence and the output is labels with similar properties (I have labeled data). For ...
5
votes
1answer
393 views

How to use multiple text features for NLP classifier?

I am trying to build text classifier, Usually, we have one text column and ground truth. But I am working on a problem where dataset contains many text features. I am exploring different ways how to ...
5
votes
2answers
69 views

Reasoning behind using Deep Learning on non-local data

I understand the using of deep learning for data that have "local" structure, for example, images/videos/texts, as the convolutional layers reduce the amount of dimensions. However, I saw ...
5
votes
1answer
135 views

LSTM Long Term Dependencies Keras

I am familiar with the LSTM unit (memory cell, forget gate, output gate etc) however I am struggling to see how this links to the LSTM implementation in Keras. In Keras the input data structure for X ...
5
votes
0answers
140 views

How to predict advantage value in deep reinforcement learning

I'm currently working on a collection of reinforcement algorithms: https://github.com/lhk/rl_gym For deep q-learning, you need to calculate the q-values that should be predicted by your network. There ...
5
votes
0answers
1k views

Fine tuning accuracy lower than Raw Transfer Learning Accuracy

I've used transfer learning on Inception V3 with ImageNet weights on Keras with Tensorflow backend on python 2.7 to create an image classifier. I first extracted and saved the bottleneck features from ...
4
votes
1answer
74 views

What ML architecture fits fixed length signal regression?

My problem is of regression type - How to estimate a fish weight using a fixed-length signal (80 data points) of the change in resistance when the fish swim through a gate with electrodes (basically 4 ...
4
votes
1answer
2k views

SGD versus Adam Optimization Clarification

Reading the Adam paper, I need some clarificaiton. It states that SGD optimization updates the parameters with the same learning rate (i.e. it does not change throughout training). They state Adam is ...
4
votes
2answers
454 views

How propagate the error delta in backpropagation in convolutional neural networks (CNN)?

My CNN has the following structure: Output neurons: 10 Input matrix (I): 28x28 Convolutional layer (C): 3 feature maps with a 5x5 kernel (output dimension is 3x24x24) Max pooling layer (MP): size 2x2 ...
4
votes
1answer
1k views

Is it wrong to use Glorot Initialization with ReLu Activation?

I'm reading that keras' default initialization is glorot_uniform. However, all of the tutorials I see are using relu ...
4
votes
0answers
777 views

Tensorflow, Optimizer.apply_gradient: 'NoneType' object has no attribute 'merge_call'

My program gives the following error message: ...
4
votes
1answer
70 views

Training Machine Learning Model - Neural Network - Islands Problem

I was working on the following leetcode problem: Given a 2d grid map of '1's (land) and '0's (water), count the number of islands. An island is surrounded by water and is formed by connecting ...
4
votes
4answers
244 views

Backpropagation: Relevance of the error signal of a neuron

During my quest to understand back propagation in a more rigorous approach I have come across with the definition of error signal of a neuron which is defined as follows for the $j^{\text{th}}$ neuron ...
4
votes
1answer
525 views

What is Bit Per Character?

What is Bits per Character (bpc) metric which has been used to measure the model accuracy with reference to text8 and ...
4
votes
2answers
5k views

Saving and loading keras.callbacks.History object with np.save and np.load

I have been saving my training history in keras as follows: ...
4
votes
2answers
99 views

What are the reasons for drawing initial neural network weights from the Gaussian distribution?

Are there theoretical or empirical reasons for drawing initial weights of a multilayer perceptron from a Gaussian rather than from, say, a Cauchy distribution?
4
votes
1answer
84 views

EM-ELM Cross validation

I know that cross validation is used to find the best hyperparameters that minimize the average error. For example, the number of neurons that minimize the average error of cross-validation is ...
4
votes
0answers
307 views

Maths of Xavier initialization

The paper I read is Glorot et al (2010). And the math part is in Section 4.2.1. Formula (5) and (10) make sense to me but I cannot derive formula (6) and (7) myself from (2) and (3). I found many ...
4
votes
1answer
312 views

Why is my U-matrix visually not separating the classes?

I am visualizing the U-matrix generated using a Self Organizing Map codebook to (visually) identify regions of similarity in the data. Although I would like to use SOM to identify clusters in ...
4
votes
0answers
80 views

Learning a logical function with a 2 layer BDN network - manual weight setting rule question?

So I am trying to construct a 2-layer network of binary decision neurons as proposed by McCullough and Pitts (1943) to learn a logical function (a composition of AND's and OR's) such as: $((\neg x_1\...
4
votes
1answer
114 views

Neural Network - Sparsity of collaborative based filtering and modelling the prediction problem

I'm fairly new to machine learning and for that matter, neural networks, but for the past couple of days I decided to take a stab at a fairly classical and practical problem of neural networks/machine ...
4
votes
0answers
272 views

how to propagate error from convolutional layer to previous layer?

I've been trying to implement a simple convolutional neural network. But I've been stuck at this problem for over a week. To be specific, assume there are 3 layers in a convolutional pass, marked as ...
4
votes
1answer
223 views

How to automatically verify official documents?

I am new to machine learning and data science. I apologise if the question seems very basic. I have a requirement where I need to verify information submitted via a form with the corresponding ...
3
votes
0answers
27 views

How to train a neural network where computing the loss requires multiple object values?

I want to train a function that given metadata about an image produces hyper-parameters for an algorithm which operates on the image. My understanding is (please forgive me I'm a novice here) a neural ...
3
votes
0answers
72 views

Why are neural networks equivalent to kernel methods?

I read a recent paper by Pedro Domingos, claiming that Every Model Learned by Gradient Descent Is Approximately a Kernel Machine. I wanted to understand the key idea a little better. Why are neural ...
3
votes
1answer
176 views

Why the sigmoid activation function results in sub-optimal gradient descent?

I need some help understanding the second shortcoming of the sigmoid activation function as described in this video from Stanford. She says that because the output of sigmoid is always positive, that ...
3
votes
0answers
23 views

Force Matching in Coarse Grained Molecular Dynamics with Jax - Forces do not match when neglecting energy loss

I am currently exploring force matching approaches for molecular dynamic simulations. As I am still in an exploration state, I'd tried investigated Force Matching Neural Network Colab Notebook ...
3
votes
0answers
63 views

How is the input gate in the LSTM learn?

How is the input gate neural network trained what to remember by propagating the error rate from predicting the next word in the language model? How does it help it to learn if it remembered the right ...
3
votes
0answers
65 views

Train NN for balanced accuracy

I'm trying to get a NN (a CNN, to be precise) to predict labels on hugely imbalanced data, i.e. some classes are much bigger than others. I want to optimize the balanced accuracy. So I give the class ...
3
votes
4answers
23k views

ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type int) in Python

I have written the following code for a neural network to perform regression on a dataset, but I am getting a ValueError. I have looked up to different answers and ...
3
votes
0answers
121 views

Best way to remove useless features (non-english words) when there are more than 100,000 features?

I am in a situation where i have more than 100,000 features, and i need to select the top features to give them to my final neural network model. So far i have been using RandomForestClassifier in ...
3
votes
1answer
107 views

Neural Network regression negative performance

I have a problem with the performance of a multi layer perceptron regressor (neural network) and I cannot figure out why. Task: I am trying to improve a time series prediction. I have predictions of a ...
3
votes
2answers
196 views

Can CNNs detect features of different images?

In lecture, we talked about “parameter sharing” as a benefit of using convolutional networks. Which of the following statements about parameter sharing in ConvNets are true? (Check all that apply.) ...
3
votes
0answers
41 views

NN training with repetitive features

I posted the question also on ai.stackexchange but it didn't get any answers so I though I could try here. Here is a copy paste: Let's say you are training a NN in a RL setting where the state (i.e. ...
3
votes
2answers
44 views

Backpropagation with a different sized training set?

I'm trying to create a NN whose input is a (length m) array of 3d vectors $$\vec{x}_i = [x_{i,1},x_{i,2},x_{i,3}], \hspace{5mm}i=1:m $$ and whose output is a similarly sized array: $$\vec{h}_{\theta,...
3
votes
0answers
425 views

Understanding depthwise convolution vs convolution with group parameters in pytorch

So in the mobilenet-v1 network, depthwise conv layers are used. And I understand that as follows. For a input feature map of (C_in, F_in, F_in), we take only 1 ...
3
votes
0answers
25 views

Information Extraction from image / text - approach?

I need assistance with a ML project I am currently trying to create. I receive a lot of invoices from a lot of different suppliers - all in their own unique layout. I need to extract 3 key elements ...
3
votes
2answers
266 views

How to perform node classification using Graph Neural Networks

I'm am trying to perform node classification using graph neural network methods. My initial plan was to convert my graphs to adjacency matrices and train my network on that, with the node features ...
3
votes
2answers
1k views

Keypoint detection from an image using a neural network

I am trying to design and train a neural network, which would be able to give me coordinates of certain key points in the image. Dataset I've got a dataset containing 1800 images similar to these: ...
3
votes
1answer
93 views

Physical modelling with neural networks - single output + stack ensemble vs multi-output

We are trying to replace an existing physical model (8 inputs/7 outputs) with artificial neural networks. The physics behind the existing model is mainly thermodynamics of humid air for air ...
3
votes
0answers
105 views

How to create anchor-positive and anchor-negative pair for feature X in a signature data set for training Siamese network

How to create anchor-positive and anchor-negative pair for feature X in a signature data set for training Siamese network? Im have a cedar signature data set with 55 peoples signatures(classes) with ...
3
votes
0answers
104 views

Unable to transform (greatly performing) Autoencoder into Variational Autoencoder

Following the procedure described in this SO question, I am trying to transform my (greatly performing) convolutional Autoencoder into a Variational version of the same Autoencoder. As explained in ...
3
votes
0answers
246 views

Training an LSTM with different time steps and number of features

I want to use an LSTM using Keras to make course grade predictions. My dataset includes student transcripts, which consist of courses taken and their respective grades of students. For each course, I ...
3
votes
1answer
836 views

how is correct usage of the validation split in neural networks?

I have a dataset separated in train, test and validation splits. After each epoch, I evaluate the loss and accuracy in the validation split. When the loss in validation split is not better, I stop ...
3
votes
0answers
72 views

Neural network cost is constant never changing during training

I am trying to build a binary classifier to predict a pulsar star with Single Hidden layer Neural Network. But the cost on training dataset after almost 100 iterations has no change, following is the ...

1
2 3 4 5
27