Questions tagged [neural-network]
Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.
4,333 questions
0
votes
0
answers
17
views
Training the neural network does not give the expected result
I'm trying to create a pytorch neural network capable of recognizing peaks in 2D graphs. Previously, I was able to get a result close to what I wanted, but it was not ideal and did not give a ...
- 1
0
votes
0
answers
8
views
Finding invariant feature areas within representation vector for each meta-class/group?
I have pairs of images which are not the same class, but are from the same meta-class/group. I have a standard CNN which produces a representation for each sample.
If I have several pairs of images ...
- 43
0
votes
0
answers
14
views
How to derive the formula 13 in the Xavier Initialization paper
How to derive the formula 13 in the Xavier Initialization paper Understanding the difficulty of training deep feedforward neural networks from the formula 6?
- 791
0
votes
0
answers
19
views
Temporal mismatch
I am building a predictive model to determine risk for a disease over the course of a hospital stay. I am using medical records from a hospital electronic medical record database. The predictions are ...
- 141
4
votes
2
answers
885
views
What is the best way to train a neural network with a variable number of inputs?
Suppose I have a neural network with 5 inputs: [A,B,C,D,E]
There is only 1 output. The expected accuracy of the model should increase when all 5 inputs are ...
- 41
0
votes
0
answers
29
views
ML Methods For Modelling Latent Variables
I have some time series predictor variables, $\{\mathbf{X}_t\} = \{\mathbf{X}_0, \ldots, \mathbf{X}_n\}$, and some other time series data $\{\mathbf{Z}_t\} = \{\mathbf{Z}_0, \ldots, \mathbf{Z}_n\}$.
...
- 101
1
vote
0
answers
21
views
Improving Detection Model - Adding image clarification
I trained an object detection model with 5K images, it works most of the time, but I am facing an issue, for few times, the object is not getting detected.
So, I planned to retrain the model, for that ...
- 21
2
votes
0
answers
31
views
What are the analogies between decision trees and neural trees?
How can I draw analogies between decision trees and neural trees? For example, how are splitting thresholds analogous between these models, and how can paths in a neural tree be represented in a ...
- 71
0
votes
0
answers
13
views
0
votes
0
answers
10
views
Using a neural network to predict disease outcomes in individual cases
I'm working on a research project with the goal of using a neural network to predict disease outcomes for patients. I've built a neural network using Tensorflow and Keras and I've trained and tested ...
0
votes
0
answers
39
views
What is the advantage of positional encoding over using additional features?
Popular models such as the transformer model use positional encoding on existing feature dimensions. Why is this preferred over adding more features to the feature dimension of the tensor which can ...
- 11
0
votes
1
answer
83
views
Connecting Flatten layer to Dense layer
I'm struggling with my neural network.
In short, I need to recreate a model from anywhere on the internet, I've found a model that combines BiLSTM, LSTM and GRU. However, based on the error I got when ...
0
votes
1
answer
67
views
How do transformer-based architectures generate contextual embeddings?
How do transformer-based architectures, such as Roberta, etc., generate contextual embeddings? The issue is, I haven't found any articles that explain this process.
user159173
0
votes
0
answers
11
views
In which cases would we not like to go to the global minimum?
I would like to know in which cases we do not want to reach the global minimum. As I understand it, this can lead to overfitting. But why is this happening? And how can I avoid this in a real task?
- 3
0
votes
1
answer
73
views
Fine tuning or just feature extraction or both using Roberta?
I'm reading a program that use the pre-trained Roberta model (roberta-base). The code first extracts word embeddings from each caption in the batch, using the last hidden state of the Roberta model. ...
user159173
2
votes
1
answer
128
views
Creating a custom loss function for an image classification model where the label matters
I have the following dataset of images, where we can see the image distribution of labels below.
I want to construct a loss function that, on the one hand, outputs probabilities for a specific class ...
- 21
0
votes
0
answers
7
views
Executing error_handler() during NN prediction
I am taking the course: Sequences, Time Series and Prediction. In this notebook we train for the first time a single layer neural network, and I see at the bottom line: executing ...
- 163
0
votes
0
answers
74
views
Weighting training instances by time in machine learning models
I am training a neural network based on data whose relevance I think diminishes based on how far each instance is in the past. I've had a look and one way to do this it seems is to 'weight' training ...
- 11
0
votes
0
answers
21
views
How to handle time series data in ANN?
I want to use ANN to forecast the next #games played in my mobile game. There are 39 features: 9 features that describe the player's state (level, amount of in game-currencies, etc.) and the last 30 ...
- 163
0
votes
0
answers
22
views
Can an OCR model consistently recognize every digit of a long number correctly?
I'm working on OCR on scanned documents and we need to recognize the exact sequence of some printed numbers on it. Imagine you're reading a bank cheque serial number (16 digits) so the system needs to ...
2
votes
1
answer
52
views
Impose minimal value on ANN prediction?
I want to predict a feature using a NN, but some business logic require that the prediction will be no less than min_value.
I imposed it after the training by:
<...
- 163
0
votes
0
answers
65
views
How can I use two different datasets to train an ML model?
I am trying to create a machine learning model that takes in two different pandas data-frames from a basketball stats website and given multiple variables, will output a prediction of how many points ...
- 1
0
votes
1
answer
47
views
My custom neural network is converging but keras model not
in most cases it is probably the other way round but...
I have implemented a basic MLP neural network structure with backpropagation. My data is just a shifted quadratic function with 100 samples. I ...
1
vote
1
answer
341
views
1
vote
1
answer
28
views
Neural Network Weights - How do they know their position?
I am a copyright scholar so please forgive my ignorance.
When weights are stored external to a model what is the mechanism by which the weight knows which neuron or node in a decision tree it is ...
0
votes
1
answer
92
views
Why was the learning rate decreased for Roberta compared to LSTM?
I'm reading the codebase of a project that uses Bidirectional-LSTM. The learning rate for it is 0.02. Later, someone improved the project by replacing LSTM with Roberta and decreased the learning rate ...
user159173
0
votes
0
answers
14
views
Gradually increasing CPU load on using sentence embeddings model with kmeans
I am having a ML based production application, using flask, deployed on GCP server using gunicorn workers. In each incoming request, a text sentence is received.
It is using sentence transformers (All-...
- 1
2
votes
1
answer
225
views
What do these terms mean in the context of Roberta?
When I read articles about Roberta, I often read the terms "transfer learning" and "fine-tuning". Additionally, they also mention "feature extraction". What are the ...
user159173
1
vote
0
answers
385
views
Why do the Llama 2 weights have eight different files?
I downloaded the weights for Llama 2 (70B-chat). This process created a folder titled "llama-2-70b-chat," which contained 8 files titled consolidated.00.pth, consolidated.01.pth, and so on ...
- 155
0
votes
1
answer
207
views
What are the differences between Embedding Layer and Roberta Embedding?
I'm reading an article about the Embedding Layer:
The Embedding Layer learns word embeddings from raw text. It is
initialized with small random numbers and can be learned
simultaneously with a neural ...
user159173
3
votes
1
answer
152
views
will ANN identify a feature that has no influence?
I am doing my first steps in training ANN, one of the features in my data X is user_id.
Assuming ...
- 163
0
votes
1
answer
73
views
How to create a multi label text classification model for small dataset in production [closed]
I have a multi-label text classification dataset which is very small around 80Kb, I am only going to receive a small amount of data for training from my client. But it is expected to build a high ...
0
votes
1
answer
23
views
Does ANN returns the same prediction for the same input?
Does ANN predictable? By this I mean that if I re-run the same script over and over, does it make sense that the error (MAE / MSE / R^2) is different on every run?
if true, then a follow-up question: ...
- 163
0
votes
2
answers
285
views
What are the differences between contextual embeddings of Bidirectional-LSTM and Transformer?
A Transformer, like Roberta, can generate contextual embeddings using the encoder part, similar to a Bidirectional-LSTM that concatenates hidden states. What are the differences between them ? Are ...
user159173
-1
votes
1
answer
56
views
0
votes
1
answer
37
views
What exactly is saved when I save a NN?
After we trained a Neural Network, we can save it in order to be able to predict without re-training. So when we use model.save('my_model.keras') what exactly is ...
- 163
0
votes
1
answer
126
views
Questions about hidden states of bidirectional LSTMs
I read this in an article about bidirectional LSTM:
In bidirectional LSTM, each word corresponds to two hidden states, one
for each direction. Thus, we concatenate these two hidden states to
...
user159173
0
votes
1
answer
67
views
Which ML algorithm is suitable for a dataset that has seasonality and trend?
Hello everyone I have a small datasets from 2006 to 2023, I would like to predict monthly sales for the next year. This is my data:
I already tried Prophet and NeuralProphet, but unfortunately they ...
- 1
1
vote
1
answer
57
views
CS 224N Back Propagation and Margin Loss in Neural Networks
I was going through Stanford CS 224 lecture notes on Back propagation.
Page 5 states:
We can see from the max-margin loss that:
∂J /∂s = − ∂J/∂s(c) = −1
I'm not sure I understand why this is the ...
- 113
1
vote
1
answer
38
views
Time series predictions with multiple series
I have a dataframe shaped something like this
patient_id
admission_id
admit_date
diagnoses
1
1
2125-10-18
[1,2]
1
2
2125-10-26
[1,2,3]
1
3
2125-11-30
[1,2,3,4,5]
...
4
7
2130-06-23
[2,3,4,7,9.......
- 11
2
votes
1
answer
414
views
Is vision transformer (ViT) always better than CNN?
The paper - AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE proposed vision transformer and outperformed CNN-based models in many cases.
When it comes to sequential data, we ...
- 23
2
votes
1
answer
2k
views
What are the differences between BPE and byte-level BPE?
In Roberta, I'm not sure if the model use BPE or byte-level BPE tokenization, are these techniques different or the same ? Can someone explain ? Thanks
user159173
0
votes
0
answers
46
views
Why do we use the RELU activation function?
I reading about activation functions in feedforward neural networks.
ad a really old paper https://web.njit.edu/~usman/courses/cs677_spring21/hornik-nn-1991.pdf.
They prove that by using arbitrary ...
- 76
0
votes
1
answer
157
views
Anomaly Detection in Log Data using LSTM
Problem Overview:
I am currently working on a project involving anomaly detection in log data. The anomalies are defined by deviations from historical patterns. The log data has a simple structure: [...
- 1
0
votes
0
answers
37
views
Why is it bad for neural networks to output indices?
Let's say you wanted to train a neural network to output N indices (maybe it's sorting an array). There are at least 2 possible ways one might sample from this ...
- 125
0
votes
0
answers
14
views
Dataset with replicates: aggregation or not?
I am currently developing a neural network tailored to a regression problem using a synthetic dataset derived from an experimental simulation campaign. Given the stochastic nature of both the ...
1
vote
1
answer
75
views
In a Computational Graph, how to calculate the total upstream gradient of a node with multiple upstreams?
Given a Computation Graph with a node (like the one below), I understand that I can use the upstream gradient dL/dz to calculate all of my downstream gradients.
But what if there are multiple ...
- 111
0
votes
0
answers
22
views
Neural network does not overfit my data. (Primarily linear function)
I am using TensorFlow and Keras. My goal is to approximate a primarily linear function that is partially nonlinear, such that a linear regression yields a Mean Absolute Error (MAE) of 0.13. All ...
0
votes
0
answers
27
views
Keras DNN outputs the same value over and over
I'm creating an ensemble of NNs with the same architecture, but each NN only outputs one value when given X_test data. The data (continuous values transformed to be [-1,1]) yields results as expected ...
0
votes
2
answers
529
views
Batch Normalization vs Layer Normalization
In Batch Normalization, mean and standard deviation are calculated feature wise and normalization step is done instance wise and in Layer Normalization mean and standard deviation are calculated ...
- 1