Questions tagged [neural-network]
Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.
4,333 questions
0
votes
0
answers
7
views
How does innovation deal with the problem of different connections in NEAT?
How does innovation deal with the problem of different connections in NEAT?
An example makes it clearer: the neural network has two outputs 3 and 4, they are connected according to the scheme 10 -> ...
1
vote
0
answers
13
views
Neural Network Overfitting on Linearly Separable Dataset
Please let me know if this question is not proper to ask here
For context, I have a dataset regarding to tiktok user engagement. The predicted variable is binary, either 'claim' or 'opinion'. From ...
- 11
1
vote
0
answers
22
views
How to choose a neural network architecture?
How to choose a neural network architecture?
Examples:
«What if I need to translate words?»
«Generate text, images?»
«Play a regular game?»
«Play a game that changes depending on the player's actions, ...
2
votes
0
answers
20
views
LSTM produces a straight line for predictionsout of range data
I have this problem:
I am trying to predict daily temperatures. I have data of 30 years, and I am using this neural network:
...
- 21
1
vote
0
answers
29
views
ML. How to make a neural network remember the context and data?
I want the neural network to be able to remember, but a perceptron can only remember something during training, but I want the neural network to adapt to new conditions without retraining, for example,...
2
votes
0
answers
41
views
Image-to-Image with U-Net no Skip-connections. Is it real?
In general, U-Net is needed to create another style image, but preserve the structure. For example, a full-fledged drawing from a sketch. Right?
I want to preserve the style, but change the structure. ...
- 21
1
vote
1
answer
53
views
Why LLM's train for only one epoch?
Why it is so crucial to NOT let the model see same texts for multiple times as we do for example in computer vision? Or should it actually be the other way around and due to the fact that we have ...
- 73
2
votes
0
answers
16
views
Multiclassification McNemar
I have a multiclassification problem and I want ro compare two classifiers using McNemar's test to find if there is any statistical significance. Do I have to do it for every class (one vs all ...
0
votes
0
answers
26
views
What is this strange behaviour of loss during training?
Picture from training gpt2 model from here
- 73
2
votes
1
answer
58
views
Why are the second-order derivatives of a loss function nonzero when linear combinations are involved?
I'm working on implementing Newton's method to perform second-order gradient descent in a neural network and having trouble computing the second order derivatives. I understand that in practice, ...
- 21
1
vote
0
answers
28
views
A small free dataset for 3D reconstruction from 2D images
I am trying to start coding on 3D image reconstruction from 2D images (so to map images to a 3D point cloud). Can any one please introduce a small free dataset to start with? I know BlendedMVS. But ...
- 103
0
votes
1
answer
23
views
Why does tanh activation work better with Pytorch than with Keras?
I'm doing a neural network to recognize written Cyrillic letters, and I found out that, when I use tanh activation function, it works WAY better with PyTorch than with Keras.
Keras code:
...
- 1
0
votes
0
answers
7
views
What happens to the operators (matmul, etc) when a model is quantized?
Model parameter quantization talks about how to reduce the precision of model weights such as in Quantization or Introduction to Quantization cooked in 🤗 with 💗🧑🍳.
But, what about the operator ...
- 791
0
votes
0
answers
23
views
Difference between Double Descent phenomenon and Benign overfitting
I am trying to wrap my head around understanding the difference between Double descent phenomenon and Benign overfitting.
Double descent occurs in a model when the test error rises as the model ...
- 101
0
votes
0
answers
8
views
Keep Error Gradient From Being to High Torwards Input Levels
During Gradient Descent, after the error goes from each neuron down to the input layer, it gets really high. How do I fix this?
0
votes
0
answers
6
views
Permutation Multiplier for Marginal Likelihood
In the famous paper Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning by Immer et al., Immer uses an approximate Laplace approximation for estimating the marginal log ...
0
votes
0
answers
30
views
Efficient Net V2 M ONNX model infers significantly slower on small input
When I convert an Efficient net v2 m model from Pytorch to Onnx on differently sized inputs, I notice a strange and unexplained behavior. I was hoping to find an explanation to my observations from ...
0
votes
0
answers
8
views
Should I interleave sin and cosine in sinusoidal positional encoding?
I'm trying to implement a sinusoidal positional encoding. I found two solutions that give different encodings. I am wondering if one of them is wrong or both are correct. I showcase visual figures of ...
- 101
0
votes
0
answers
15
views
My network to classify dialects is not working
I have written the following code to classify dialects based on the timit dataset using .wav files. For some reason my model is not learning and classifies everything into the same class. Is it ...
0
votes
1
answer
32
views
Understanding the output of Google's wide and deep model?
I'm trying to implement Google's wide and deep model and I have a question about its output.
According to the equation (3) in the paper:
$$ P(Y=1|X) = \sigma(w_{wide}^T[x,\phi(x)] + w_{deep}^T a^{(l_f)...
- 103
0
votes
0
answers
8
views
I can't overfit Resnet50 to map a single image to its corresponding set of 1000 (x,y) points. Why?
I have a 360p image: https://imgur.com/a/u3uDFj9
I have a corresponding human mask and a (1000,2) output, i.e, 1000 two dimensional points: https://file.io/Vc4hA1LYd5NO
My dataset actually has a huge ...
- 141
0
votes
0
answers
42
views
Why can't my neural network model learn abs(x1-x2) function?
I am trying to train a simple neural network model for multiclass classification.
I have x1,x2,x3,x4 columns with 4 classes to predict.
If just train on x1,x2,x3,x4 then I get accuracy of 88%
With ...
0
votes
0
answers
36
views
Train a neural network on tabular data - loss on validation data stagnates
I am training a neural network on tabular data. I have a training set and a validation set with 55000 and 20000 samples respectively. I have trained it with 200 epochs. I observe a steady decrease in ...
- 11
0
votes
0
answers
13
views
Data augmentation strategies when there is a difference in image resolution between train and test set
I am doing transfer learning Resnet34 for an object detection problem. My dataset has around 9500 observations, 20 classes. I tried the basic augmentation such as: horizontal flip, gaussian noise, ...
0
votes
0
answers
41
views
Dimensionality problem with LSTM input and output with MATLAB ```TrainNetwork```
I was trying to load my input array to LSTM for data training and validation. I encountered an error stating that the expected input are different from my input dimension:
...
- 1
0
votes
0
answers
34
views
Neural network for complex-valued data
I have some complex data, and I want to teach a neural network to perform certain operations on it.
Now, since Tensorflow does not allow complex inputs, I separated my data into real and imaginary ...
- 103
0
votes
1
answer
51
views
Neural network with just one Dense layer
I have a model which works well with just one Dense layer. My model has an input layer, a Dense layer, and then one Reshape layer, which reshapes the output into the desired form. Normally, neural ...
- 103
0
votes
0
answers
18
views
Artificial Neural Network - interpreting model.save output
I have implemented an ANN with 2 hidden layers, activation PRelU. I have saved my model in an hdf5 file and am trying to interpret the results from model.save (kernel, bias, and alpha)... I can see ...
- 1
0
votes
1
answer
27
views
Where should I learn pytorch from?
I'm a undergraduate student. I've coded a three-node neural network (that works) based on my professor's guidance. However, I'd like to pursue a career in AI and Data Science, and I'd like to teach ...
0
votes
0
answers
14
views
Can RNNs Learn Time-Bounded Distributions of Stochastic Processes and Time-Dependent Transition Densities?
I'm currently reading the book Applied Stochastic Differential Equations by Särkkä and Solin (link - page 234), where it is mentioned that one can calculate the maximum likelihood estimate (MLE) for a ...
- 101
0
votes
0
answers
18
views
Detecting text in binary blobs using ML
Context - I'm an experienced SWE with little expereince in ML, I would like to perform the following:
Given a binary blob (bytearray) of size 256, consisting of random bytes with meaningful text ...
- 101
0
votes
0
answers
10
views
Examples of Confidence Intervals in Input Data
I am researching possible ways to efficiently train models that have to deal with confidence intervals in input data. In the example below, the feature has expected values of 15 and 21, respectively, ...
1
vote
1
answer
112
views
Where has RSquare from tensorflow_addons.metrics gone and what's the appropriate replacement in model.compile(…) of keras?
tensorflow_addons is dead:
...
- 121
0
votes
0
answers
37
views
Does Increasing Dimensionality Before Compression Make Sense for Anomaly Detection with Autoencoders?
Given a dataset $X$ of shape $(n, p)$ such that $n \gg 1$ and $p \approx 10$, I would like to train an autoencoder to solve an anomaly detection problem.
I did some experiments considering a classical ...
- 61
0
votes
0
answers
38
views
hacky backprop outperforms clean backprop - Why?
I implemented a basic NN for MNIST in Numpy and started with a hacky implementation of backprop (just randomly multiplying gradients together), but somehow that one works better than my cleaned up ...
-1
votes
1
answer
42
views
How can I select subsets of features using neural network?
This listing selects the best features from the 1000 available columns in a given dataset.
The first three columns are dropped because they are useless data.
The dataset is huge. So, they were read in ...
- 99
0
votes
0
answers
64
views
How weight vector behave when we initialize the weight to 0 in case of perceptron
While reading in book i encountered this statement
Now, the reason we don't initialize the weights to zero is that the learning rate (eta) only has an effect on the classification outcome if the ...
- 101
1
vote
1
answer
49
views
Everything is classified as background by segmentation model
I am training a U-NET model for medical image segmentation. Problem is that the binary masks that im using to train the model mostly consist of background pixels and a very small region of the whole ...
- 61
0
votes
0
answers
22
views
Does it common for LM (hundreds million parameters) beat LLM (billion parameters) for binary classification task?
Preface
I am trying to fine-tune the transformer-based model (LM and LLM). The LM that I used is DEBERTA, and the LLM is LLaMA 3. The task is to classify whether a text contains condescending language ...
0
votes
0
answers
15
views
How to increase the optimial cutoff point(youden index) after training a model?
So I trained a model based on a medical dataset and and I got an AUROC for detecting cancer in brain images as about 0.96 and i noticed that the youden index is 0.1 but i want to increase it to 0.5 , ...
-1
votes
1
answer
13
views
WGAN generating images from the training data
Is it possible for gan to remember somehow training data distribution?
Or maybe somеthing leaks out when I calculate gradients?
...
- 73
0
votes
0
answers
38
views
Is it legit to normalize time series with respect to the x-axis?
I have a data set consisting of multivariate time series, e.g. a batch of my data has the shape (batch_size, timesteps, number_input_features) and I want to train a neural network on it to predict ...
- 13
1
vote
1
answer
40
views
How does seeing training batches only once influence the generalization of a neural network?
I am referring to this question/scenario Train neural network with unlimited training data but unfortunately I can not comment.
As I am not seeing any training batch multiple times I would guess that ...
- 13
0
votes
0
answers
11
views
How to handle sequences with crossEntropyLoss
fist of all i am ne wto the whole thing, so sorry if this is superdumb.
I'm currently training a Transformer model for a sequence classification task using CrossEntropyLoss. My input tensor has the ...
- 101
0
votes
0
answers
30
views
What is the most accurate way of computing the evaluation time of a neural network model?
I am training some neural networks in pytorch to use as an embedded surrogate model. Since I am testing various architectures, I want to compare the accuracy of each one, but I am also interested in ...
- 1
0
votes
0
answers
29
views
Mobilenet vs resnet
Q1-Why dont we remove relu after addition of skip connection in resnet50 like we do in mobile-net v2 for better performance?
Q2-And why dont we have Convolution layer in skip connection for dimention ...
2
votes
1
answer
27
views
Benchmark Neural Networks on High-Dimensional Functions
For a personal project, I am interested in benchmarking certain neural network architectures in the context of high-dimensional function approximation. Specifically, I am interested in continuous, ...
- 121
0
votes
1
answer
26
views
What is the "fast version" of ZFNet referenced in SPPNet and Faster R-CNN papers?
I'm reading old papers:
SPPNet: Link
Faster R-CNN: Link
In both cases, the authors refer to a "fast version of Zeiler and Fergus (ZF) Net"; specifically:
In SPPNet:
ZF-5: this ...
1
vote
0
answers
47
views
Why can't I replicate the results from this paper?
I'm trying to train a model to evaluate chess positions, following the methodology from this paper (note that the author presents several different architectures, but I'm only looking at the ANN with ...
1
vote
1
answer
59
views
wierd neural network approache
I'm working on a problem where I need to create a neural network to optimize the seating arrangement for 24 unique individuals in a 6x4 grid, minimizing conflicts between adjacent (up,down,left,right) ...
- 101