Questions tagged [neural-network]

Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.

Filter by
Sorted by
Tagged with
1 vote
0 answers
11 views

If two functions are close apart can I proof the difference of their empirical loss is also small?

I am trying to understand the proof of Theorem 3 in the paper "A Universal Law of Robustness via isoperimetry" by Bubeck and Sellke. Basically there exist atleast one $w_{L,e}$ in $\...
user avatar
0 votes
0 answers
15 views

What is the difference between one hot encoding and 1-of-c encoding?

I am tasked with using 1-of-c encoding for a NN problem but I cannot find an explanation of what it is. Everything I have read sounds like it is the same as one hot encoding... Thanks
user avatar
  • 111
0 votes
0 answers
5 views

Backtracking filter coefficients of Convolutional Neural Networks

I'm starting to learn how convolutional neural networks work, and I have a question regarding the filters. Apparently, these are randomly generated when the model is generated, and then as the data is ...
user avatar
0 votes
0 answers
15 views

Best gradient free methos to converge a 2 layers Neural Network on MNIST?

I've developed a C++ Neural Network that work on the MNIST dataset. I don't want to use backpropagation. Are there optimal methods to avoid it and that make the network converge to high accuracies?
user avatar
0 votes
0 answers
6 views

WGAN-GP: how to understand whether my networks are working as they are supposed to?

I am training a WGAN-GP. Is there any way to verify whether my networks are working as they are supposed to during training? I have no feeling about the outputs of my networks. I do not want to wait ...
user avatar
  • 1
0 votes
2 answers
16 views

NNs for fitting highly oscillatory functions

in a scientific computing application of neural networks, I have to maximize several neural networks with scalar output with respect to a target/loss function (coming from a weak form of a PDE). It is ...
user avatar
  • 1
1 vote
0 answers
13 views

Contextual word embeddings from pretrained word2vec vectors

I would like to create word embeddings that take context into account, so the vector of the word Jaguar [animal] would be different from the word Jaguar [car brand]. As you know, word2vec only gives ...
user avatar
0 votes
0 answers
8 views

CNN Eliminate Wrong Results

I extracted images of human faces from the videos, but the model also recorded images without faces. I wrote CNN for emotion classification. In the obvious pictures, the probability is closer to a ...
user avatar
2 votes
1 answer
59 views

Why Deep Learning / Neural Networs don't achieve state of the art results in tabular data problems?

Apparently, deep learning methods don't achieve state-of-the-art results on tabular data problems [1,2]. This claim appears to be known also by Kagglers. The SOTA method looks like it is the gradient ...
user avatar
0 votes
0 answers
22 views

How to detect anomalies?

I have timeseries data with one value per day for a year. (there is one column with temperature data). I am using autoencoders to train a reconstruction model with mse loss. Firstly, I normalized the ...
user avatar
0 votes
1 answer
12 views

Coefficients values in filter in Convolutional Neural Networks

I'm starting to learn how convolutional neural networks work, and I have a question regarding the filters. Are these chosen manually or are they generated by the network in training? If it's the ...
user avatar
0 votes
0 answers
11 views

Inference speed of ReLU networks

I'm fairly new in the topic, and I was wondering whether some of you can point to existing works in which the inference of deep neural networks with ReLU activation functions is tested on GPUs as a ...
user avatar
1 vote
0 answers
30 views

Random Forest significantly outperforms neural net for my regression task. What insight can I extract from this?

I'm working on a regression problem for a personal project, and I've noticed that random forest performs significantly better than the current neural net architectures I'm using. (about 0.7 R2 for a ...
user avatar
  • 11
1 vote
1 answer
20 views

RNN/LSTM timeseries, with fixed attributes per run

I have a multivariate time series of weather date: temperature, humidity and wind strength ($x_{c,t},y_{c,t},z_{c,t}$ respectively). I have this data for a dozen different cities ($c\in {c_1,c_2,...,...
user avatar
  • 13
0 votes
0 answers
8 views

DeepSets Keras Architecture

I am interested in the Deepsets Model but I am not quite sure how to implement it in Keras/Tensorflow for a normal regression task. More specifically the DeepSet model should take an arbitrary-sized ...
user avatar
0 votes
0 answers
10 views

Neural Network One-hot Feature concatenation

I'm trying to add features to a model with two one hot encoded features. The features are defined like this. ...
user avatar
0 votes
0 answers
9 views

Proper iteration over time series data for LSTM neural network

I’m using the supervised learning method with an LSTM network to predict forex prices. To achieve this I’m using deeplearning4j library but I doubt several points of my implementation. I turned off ...
user avatar
0 votes
0 answers
8 views

What should be the loss and accuracy value while training the input and output data in deep learning using jupyter notebook?

I am working on fault detection and fault classification in power system using deep learning, when I am training the input data (fault coefficients m, n, p, q) and output data (fault type A-G, B-G, C-...
user avatar
0 votes
0 answers
7 views

How to choose max layers and units to search over in hyper parameter tuning

When performing any hyper parameter tuning, let's say random search for simplicity, and I want to search over a minimum to max units/nodes in a layer, and a minimum to max number of layers, are there ...
user avatar
  • 21
3 votes
2 answers
53 views

Approaches to fit a theoretical model on a physical device

Happy to join this community. Thank you in advance for your kind help! :) Intro I have a physical device characterized by its internal parameters, of which I know the nominal values. I also have the ...
user avatar
0 votes
1 answer
19 views

How to deal with one output for multiple inputs?

Hei! I want to train a model, that predicts the sentiment of news headlines. I've got multiple unordered news headlines per day, but one sentiment score. What is a convenient solution to overcome the ...
user avatar
1 vote
0 answers
19 views

How to predict a mathematical progression with keras

I try the following model for a many-to-many recurrent network: ...
user avatar
0 votes
0 answers
17 views

How should I process the output from this neural network?

I have a neural network that takes an np.array of a mel spectrogram of a 3 second audio clip from a song as input, and outputs vector of individual predictions that it is from 494 given (individual) ...
user avatar
0 votes
1 answer
24 views

What kind of neural network am I using? How can I build a specific kind of network?

I'm going through a tutorial for tensorflow with keras and at one stage you build the neural network model using the code below. ...
user avatar
1 vote
1 answer
34 views

Why is Word2vec regarded as a neural embedding?

In the skip-gram model, the probability that a word $w$ is part of the set of context words $\{w_o^{(i)}\}$ $(i= 1:m)$ where $m$ is the context window around the central word, is given by: $$p(w_o | ...
user avatar
  • 43
0 votes
0 answers
8 views

Are the weights of a trained neural network repeatable in their convergence?

The question came up whether a neural network will always converge to the same weights if it is retrained repeatedly from the same starting values. Of course this would assume that each repeat ...
user avatar
  • 101
0 votes
0 answers
8 views

Custom loss for classification of geolocation in Keras

I have a model that predicts geolocation coordinates based on some data. The way I have it set up at the moment is that I clustered my points (2D coordinates) into 100 classes that my model predicts ...
user avatar
0 votes
0 answers
14 views

Which LSTM Training Strategy Performs better?

I would like to use LSTM for predicting multiple time series (Time series about sales per day in multiple countries. In parts, contradicting regional trends are present within the data. Sales is the '...
user avatar
  • 1
0 votes
0 answers
65 views

Poor neural net regressive fit to data that exhibit clear structure

I've been trying to use a simple NN to model data I've generated. The data lack a closed form expression, but exhibit clear structure. The MWE below emulates similar data. I find that any NN I create, ...
user avatar
1 vote
0 answers
5 views

How to convert ECG Data to Graphical Data so that it can be used in GNNs?

I am trying to predict arrythmia using GCNN but the problem i am facing is that the data is in tabular format screenshot attached below. Upon reading i found out that there needs to nodes and edges ...
user avatar
0 votes
0 answers
14 views

Neural Network for solving these linear algebra problems

Intro There are several questions on this site about whether or not machine learning can solve specific problems. The answer (in my words) seems to be: "Yes, trivially, if you choose a model to ...
user avatar
0 votes
0 answers
19 views

Multioutput Neural Network for function approximation

I am trying to extend the example here to be capable of handling multiple outputs for function approximations ...
user avatar
0 votes
0 answers
29 views

Performance metrics for LSTM Autoencoder

I am building an LSTM Autoencoder (unsupervised model) to detect anomalies in a time series dataset. The input is telemetry data from routers and I want to detect anomalies in the throughout of router....
user avatar
0 votes
0 answers
16 views

Backpropagation in NN

During backward pass, which gradients are kept and which gradients are discarded? Why are some gradients discarded? I know that forward pass is computing the output of the network given the inputs and ...
user avatar
0 votes
0 answers
17 views

Too optimistic results with efficientnet v2 with cifar10

I'm trying to run efficient net with cifar-10 which should get high results, but not 100%. Here is my code: ...
user avatar
0 votes
0 answers
9 views

Convolutional Neural network learning curve results

Working on a convolutional neural network with 6 classes and about 1500 image per class. The model that works best for me has given the results below, in previous models I have worked on has given ...
user avatar
  • 1
0 votes
0 answers
10 views

Stride in time series classification/regression using neural networks

When dealing with time series in neural networks, we use windows with a size and a stride as input. Is it advantageous to train such a neural network with a stride that is smaller than the stride used ...
user avatar
  • 101
0 votes
0 answers
8 views

Automatic scaling and resizing

I have a CAD-like system: users create Canvases and put different Objects on it. Sometimes users need to scale the Canvas and move all included objects to different positions and probably change their ...
user avatar
  • 101
0 votes
0 answers
6 views

How to detect that sequence of points belong to some model of first order theory?

Assume that every neural network can be recast to the sequence of layers (https://arxiv.org/abs/2106.14587 has chapter how to do this). Assume that layer U has N neurons. The set of possible ...
user avatar
  • 121
0 votes
0 answers
20 views

ResNet: Derive the gradient matrices w.r.t. W1 and W2 and backprop equation in a Residual Network

How would I go about step by step deriving stochastic gradient matrices w.r.t. W1 and W2 and backpropagation equation in a residual block that is a part of a larger ResNet network with forward ...
user avatar
  • 1
0 votes
0 answers
7 views

Can preprocessing in time-series data (e.g. deseasonaliation or detrending) helps create better forecasting model?

I am reading a paper that mentions the following. ...
user avatar
  • 131
0 votes
1 answer
14 views

Activation Functions in Haykins Neural Networks a comprehensive foundation

In Haykins Neural Network a comprehensive foundation, the piecwise-linear funtion is one of the described activation functions. It is described with: The corresponding shown plot is I don't really ...
user avatar
0 votes
0 answers
15 views

Is Loss value (e.g., MSE loss) used in the calculation for parameter update when doing gradient descent?

My question is really simple. I know the theory behind gradient descent and parameter updates, what I really haven't found clarity on is that is the loss value (e.g., MSE value) used, i.e., multiplied ...
user avatar
  • 1
1 vote
0 answers
16 views

Early vs Late Fusion in Multimodal Convolutional Neural Networks

As for Early, Middle, and Late Fusion in Multimodal Convolutional Neural Networks What is difference among them? Are there nice document or arcile which are describing these. Best regards.
user avatar
  • 11
0 votes
1 answer
31 views

Overfitting CNN model - any relation to input image size?

If my CNN model is over-fitting despite trying all possible hyper parameter tuning, does it mean I must decrease/increase my input image size in the Imagadatagenarator?
user avatar
  • 11
1 vote
0 answers
112 views

How to plot the computational graph and derive the update procedure of parameters using the backpropagation algorithm?

Please help me to solve this problem without a code (ps: this is a written problem): Given the following loss function, please plot the computational graph, and derive the update procedure of ...
user avatar
  • 21
1 vote
1 answer
81 views

Validation loss not decreasing using dense layers altough training and validation data have the same distribution

I have a problem that I have great difficulties understanding the concept that leads to these results. I use a keras dense layer to map 13 input features to 3 output labels. During the training, the ...
user avatar
  • 47
0 votes
0 answers
8 views

Efficient Searching for a basis of information as a hyperparameter in a large possible hyperparameter space

I have a set of inputs, let's call them 'I', that can be fed through a complicated group of functions to produce/calculate a wide variety of outputs (let's call them 'O'). I want to find a subset of ...
user avatar
  • 13
0 votes
0 answers
9 views

How much an experienced machine learning developer will recommend Google Machine Learning Crash course?

I am an android developer and I've four years of development experience. I've worked on Java, Kotlin, and SQL. Also, I know python. Now I want to switch my field to Machine Learning and deep learning. ...
user avatar
0 votes
0 answers
28 views

Using LSTM for text generation keeps generating same word

I work on a simple text generation problem using a portion of the Shakespeare dataset that I decided to use LSTM for. I primarily used this tutorial for reference. However, as I ran the below code, I ...
user avatar
  • 257

1
2 3 4 5
82