Questions tagged [neural-network]

Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.

1,288 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
8
votes
3answers
2k views

Loss Function for Probability Regression

I am trying to predict a probability with a neural network, but having trouble figuring out which loss function is best. Cross entropy was my first thought, but other resources always talk about it in ...
7
votes
2answers
202 views

How to implement hierarchical labeling classification?

I am currently working on the task of eCommerce product name classification, so I have categories and subcategories in product data. I noticed that using subcategories as labels delivers worse results ...
6
votes
1answer
232 views

Training stateful LSTM with different number of sequences

I'm using a stateful LSTM for stock market analysis, and I have varying amounts of data for each stock, ranging from 20 years to just a few weeks (i.e. for newly listed stocks). I use 3 years of data ...
6
votes
1answer
137 views

How to model segmentation of a sequence to similar parts?

I guess LSTM is good for sequence modeling but how would you model "clustering" with it? Meaning, the input is a sequence and the output is labels with similar properties (I have labeled data). For ...
5
votes
2answers
66 views

Reasoning behind using Deep Learning on non-local data

I understand the using of deep learning for data that have "local" structure, for example, images/videos/texts, as the convolutional layers reduce the amount of dimensions. However, I saw ...
5
votes
1answer
85 views

Reinforcement learning with sparse acting agent

I'm working on a problem where the optimal policy involves the agent "doing nothing" most of the time, and "doing something" during rare critical moments. Is there any literature or best practices ...
5
votes
1answer
818 views

Comparison between addition and multiplication function in deep neural network?

I designed a specific Convolution Neural Network to study in the area of image processing. The network has a part that there are two tensors that have to be transformed into a tensor in order to be ...
5
votes
1answer
129 views

LSTM Long Term Dependencies Keras

I am familiar with the LSTM unit (memory cell, forget gate, output gate etc) however I am struggling to see how this links to the LSTM implementation in Keras. In Keras the input data structure for X ...
5
votes
0answers
68 views

Training value neural network AlphaGo style

I have been trying to replicate the results obtained by AlphaGo following their supervise learning protocol. The papers specify that they use a network that has two heads: a value head that predicts ...
5
votes
1answer
2k views

Loss function for optimising precision & recall / sensitivity & specificity?

I've been using precision and recall as my metrics, as per keras-team/keras/pull/9393/files Sensitivity & specificity is what I want to optimise for. Every epoch I output it: ...
5
votes
0answers
133 views

Deep advantage learning: how to predict the value

I'm currently working on a collection of reinforcement algorithms: https://github.com/lhk/rl_gym For deep q-learning, you need to calculate the q-values that should be predicted by your network. There ...
5
votes
0answers
1k views

Fine tuning accuracy lower than Raw Transfer Learning Accuracy

I've used transfer learning on Inception V3 with ImageNet weights on Keras with Tensorflow backend on python 2.7 to create an image classifier. I first extracted and saved the bottleneck features from ...
4
votes
1answer
66 views

What ML architecture fits fixed length signal regression?

My problem is of regression type - How to estimate a fish weight using a fixed-length signal (80 data points) of the change in resistance when the fish swim through a gate with electrodes (basically 4 ...
4
votes
1answer
1k views

SGD versus Adam Optimization Clarification

Reading the Adam paper, I need some clarificaiton. It states that SGD optimization updates the parameters with the same learning rate (i.e. it does not change throughout training). They state Adam is ...
4
votes
2answers
374 views

How propagate the error delta in backpropagation in convolutional neural networks (CNN)?

My CNN has the following structure: Output neurons: 10 Input matrix (I): 28x28 Convolutional layer (C): 3 feature maps with a 5x5 kernel (output dimension is 3x24x24) Max pooling layer (MP): size 2x2 ...
4
votes
0answers
306 views

How to use multiple text features for NLP classifier?

I am trying to build text classifier, Usually, we have one text column and ground truth. But I am working on a problem where dataset contains many text features. I am exploring different ways how to ...
4
votes
1answer
1k views

Is it wrong to use Glorot Initialization with ReLu Activation?

I'm reading that keras' default initialization is glorot_uniform. However, all of the tutorials I see are using relu ...
4
votes
0answers
729 views

Tensorflow, Optimizer.apply_gradient: 'NoneType' object has no attribute 'merge_call'

My program gives the following error message: ...
4
votes
1answer
67 views

Training Machine Learning Model - Neural Network - Islands Problem

I was working on the following leetcode problem: Given a 2d grid map of '1's (land) and '0's (water), count the number of islands. An island is surrounded by water and is formed by connecting ...
4
votes
4answers
221 views

Backpropagation: Relevance of the error signal of a neuron

During my quest to understand back propagation in a more rigorous approach I have come across with the definition of error signal of a neuron which is defined as follows for the $j^{\text{th}}$ neuron ...
4
votes
2answers
5k views

Saving and loading keras.callbacks.History object with np.save and np.load

I have been saving my training history in keras as follows: ...
4
votes
2answers
82 views

What are the reasons for drawing initial neural network weights from the Gaussian distribution?

Are there theoretical or empirical reasons for drawing initial weights of a multilayer perceptron from a Gaussian rather than from, say, a Cauchy distribution?
4
votes
1answer
82 views

EM-ELM Cross validation

I know that cross validation is used to find the best hyperparameters that minimize the average error. For example, the number of neurons that minimize the average error of cross-validation is ...
4
votes
0answers
301 views

Maths of Xavier initialization

The paper I read is Glorot et al (2010). And the math part is in Section 4.2.1. Formula (5) and (10) make sense to me but I cannot derive formula (6) and (7) myself from (2) and (3). I found many ...
4
votes
1answer
306 views

Why is my U-matrix visually not separating the classes?

I am visualizing the U-matrix generated using a Self Organizing Map codebook to (visually) identify regions of similarity in the data. Although I would like to use SOM to identify clusters in ...
4
votes
0answers
80 views

Learning a logical function with a 2 layer BDN network - manual weight setting rule question?

So I am trying to construct a 2-layer network of binary decision neurons as proposed by McCullough and Pitts (1943) to learn a logical function (a composition of AND's and OR's) such as: $((\neg x_1\...
4
votes
1answer
113 views

Neural Network - Sparsity of collaborative based filtering and modelling the prediction problem

I'm fairly new to machine learning and for that matter, neural networks, but for the past couple of days I decided to take a stab at a fairly classical and practical problem of neural networks/machine ...
4
votes
0answers
270 views

how to propagate error from convolutional layer to previous layer?

I've been trying to implement a simple convolutional neural network. But I've been stuck at this problem for over a week. To be specific, assume there are 3 layers in a convolutional pass, marked as ...
4
votes
1answer
214 views

How to automatically verify official documents?

I am new to machine learning and data science. I apologise if the question seems very basic. I have a requirement where I need to verify information submitted via a form with the corresponding ...
3
votes
0answers
25 views

How to train a neural network where computing the loss requires multiple object values?

I want to train a function that given metadata about an image produces hyper-parameters for an algorithm which operates on the image. My understanding is (please forgive me I'm a novice here) a neural ...
3
votes
2answers
401 views

Why is the "dying ReLU" problem not present in most modern deep learning architectures?

The $ReLU(x) = max(0,x)$ function is an often used activation function in neural networks. However it has been shown that it can suffer from the dying Relu problem (see also What is the "dying ...
3
votes
0answers
117 views

Why the sigmoid activation function results in sub-optimal gradient descent?

I need some help understanding the second shortcoming of the sigmoid activation function as described in this video from Stanford. She says that because the output of sigmoid is always positive, that ...
3
votes
0answers
23 views

Force Matching in Coarse Grained Molecular Dynamics with Jax - Forces do not match when neglecting energy loss

I am currently exploring force matching approaches for molecular dynamic simulations. As I am still in an exploration state, I'd tried investigated Force Matching Neural Network Colab Notebook ...
3
votes
0answers
60 views

How is the input gate in the LSTM learn?

How is the input gate neural network trained what to remember by propagating the error rate from predicting the next word in the language model? How does it help it to learn if it remembered the right ...
3
votes
0answers
37 views

Train NN for balanced accuracy

I'm trying to get a NN (a CNN, to be precise) to predict labels on hugely imbalanced data, i.e. some classes are much bigger than others. I want to optimize the balanced accuracy. So I give the class ...
3
votes
3answers
15k views

ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type int) in Python

I have written the following code for a neural network to perform regression on a dataset, but I am getting a ValueError. I have looked up to different answers and ...
3
votes
0answers
106 views

Best way to remove useless features (non-english words) when there are more than 100,000 features?

I am in a situation where i have more than 100,000 features, and i need to select the top features to give them to my final neural network model. So far i have been using RandomForestClassifier in ...
3
votes
1answer
90 views

Neural Network regression negative performance

I have a problem with the performance of a multi layer perceptron regressor (neural network) and I cannot figure out why. Task: I am trying to improve a time series prediction. I have predictions of a ...
3
votes
2answers
161 views

Can CNNs detect features of different images?

In lecture, we talked about “parameter sharing” as a benefit of using convolutional networks. Which of the following statements about parameter sharing in ConvNets are true? (Check all that apply.) ...
3
votes
1answer
89 views

Semantic network using word2vec

I have thousands of headlines and I would like to build a semantic network using word2vec, specifically google news files. My sentences look like ...
3
votes
0answers
40 views

NN training with repetitive features

I posted the question also on ai.stackexchange but it didn't get any answers so I though I could try here. Here is a copy paste: Let's say you are training a NN in a RL setting where the state (i.e. ...
3
votes
1answer
179 views

Custom connections between layers Keras

I would like to manually define connections in neural network between layers using TensorFlow and keras with Python. By default connections are beween all pairs of neurons. I need to make connections ...
3
votes
2answers
40 views

Backpropagation with a different sized training set?

I'm trying to create a NN whose input is a (length m) array of 3d vectors $$\vec{x}_i = [x_{i,1},x_{i,2},x_{i,3}], \hspace{5mm}i=1:m $$ and whose output is a similarly sized array: $$\vec{h}_{\theta,...
3
votes
0answers
24 views

Information Extraction from image / text - approach?

I need assistance with a ML project I am currently trying to create. I receive a lot of invoices from a lot of different suppliers - all in their own unique layout. I need to extract 3 key elements ...
3
votes
2answers
248 views

How to perform node classification using Graph Neural Networks

I'm am trying to perform node classification using graph neural network methods. My initial plan was to convert my graphs to adjacency matrices and train my network on that, with the node features ...
3
votes
0answers
147 views

How does one derive the modified tanh activation proposed by LeCun?

In "Efficient Backprop" (http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf), LeCun and others propose a modified tanh activation function of the form: $$ f(x) = 1.7159 * tanh(\frac{2}{3}*x) $$ ...
3
votes
2answers
1k views

Keypoint detection from an image using a neural network

I am trying to design and train a neural network, which would be able to give me coordinates of certain key points in the image. Dataset I've got a dataset containing 1800 images similar to these: ...
3
votes
1answer
72 views

Physical modelling with neural networks - single output + stack ensemble vs multi-output

We are trying to replace an existing physical model (8 inputs/7 outputs) with artificial neural networks. The physics behind the existing model is mainly thermodynamics of humid air for air ...
3
votes
0answers
88 views

How to create anchor-positive and anchor-negative pair for feature X in a signature data set for training Siamese network

How to create anchor-positive and anchor-negative pair for feature X in a signature data set for training Siamese network? Im have a cedar signature data set with 55 peoples signatures(classes) with ...
3
votes
0answers
98 views

Unable to transform (greatly performing) Autoencoder into Variational Autoencoder

Following the procedure described in this SO question, I am trying to transform my (greatly performing) convolutional Autoencoder into a Variational version of the same Autoencoder. As explained in ...

1
2 3 4 5
26