Questions tagged [neural-network]

Artificial neural networks (ANN), are composed of 'neurons' - programming constructs that mimic the properties of biological neurons. A set of weighted connections between the neurons allows information to propagate through the network to solve artificial intelligence problems without the network designer having had a model of a real system.

Filter by
Sorted by
Tagged with
2
votes
0answers
14 views

Variance of Jacobian in backpropagation

Glorot and Bengio introduced the Glorot uniform initialisation in their 2010 publication. My question concerns some details in the derivation, specifically how equation (2) leads to equation (6). I ...
0
votes
1answer
27 views

AutoML for categorical feature encoding

I have an input dataset with more than 100 variables where around 80% of the variables are categorical in nature. While some variables like gender, country etc can be one-hot encoded but I also have ...
2
votes
1answer
34 views

exclude variables with no variation during prediction?

I am working on a binary classification problem. I do have certain input categorical variables such as gender, ethnicity etc. ...
1
vote
0answers
19 views

Why we call Mix-up method is a data augmentation technique?

I am bit confused in the Mixup data augmentation technique, let me explain the problem briefly: What is Mixup For further detail you may refer to original paper . We double or quadruple the data ...
-1
votes
0answers
8 views

Training and Validation loss are same but not decreasing for LSTM model

I have a timeseries data and I am doing univariate forecasting using stacked LSTM without any activation function, Like following. ...
0
votes
0answers
12 views

Saturation of exponential linear units (ELU)

The ELU activation function saturates to $-a$ for large negative inputs. \begin{align} f(x) = \left\{ \begin{array}{rcl} a(\text{exp}(x) - 1) & \mbox{for } x \leq 0 \\ x & \...
0
votes
1answer
15 views

How to use rule-based labelling intelligently?

I have a dataset like below The outcome column is labelled as positive if the % difference between target final Qty and ...
0
votes
0answers
5 views

Update tensorflow model with new data

I have trained the time series prediction model using old data, and as time goes by, I get more data and want to update my model with it. But it seems like my model perform worse as I update the model....
0
votes
0answers
20 views

Why is my extremely simple neural network code performing so badly?

first time poster here. I am trying to build a NN using sklearn MLPRegressor on a file which has the shape (1024,3). The first two columns are two dimensional input data, the third is the target. ...
0
votes
0answers
9 views

TensorFlow Speech Emotion Recognition Model gives same prediction for all inputs

Dataset used: RAVDESS (I've only used the audio only files) Here's a sample after I've processed the data: And the code for the label encoding: ...
1
vote
1answer
37 views

Advice / Good practises | CNN poor image diversity

I am currenty working on a project that involves multiple cameras fixed on the ceiling. Each time I take a picture, I check whether there is a "cart" right under the camera. I would like to ...
0
votes
0answers
7 views

Can I fine-tune pre-trained unconditional StyleGAN2/3 to be conditional?

All papers and articles cover how to fine-tune unconditional StyleGAN on more unconditional data (meaning belonging to one class). I can't find sources of knowledge covering conditional StyleGANs and ...
0
votes
2answers
24 views

How to show prototype of output before building model

Currently in my work, we are working on a POC for a AI project. We intend to do a binary classification using traditional classification algorithms. However, my boss wants me to show a feel of the ...
0
votes
1answer
15 views

Estimate timeline for a ML Project

I am a novice data scientist and have been asked to provide an estimate for a data science project in our organization. From the problem stmt description, i am able to understand that it is a ...
1
vote
0answers
15 views

Constructing circular towers to show that single hidden layer feedforward neural networks can approximate any continuous function

In this intuitive explanation of why wide-enough shallow feedforward neural networks can satisfy the universal approximation theorem from any continuous function on a compact domain, the author uses, ...
1
vote
1answer
18 views

Keras: How to restore initial weights when using EarlyStopping

Using Keras, I setup EarlyStoping like this: EarlyStopping(monitor='val_loss', min_delta=0, patience=100, verbose=0, mode='min', restore_best_weights=True) When I ...
0
votes
0answers
8 views

Need some idea for modifying the Self-Organizing Map (ranking-awareness)

I am working on a project which aims to modify a neural network called the Self-Organizing Map SOM - which is essentially a clustering/dimensionality reduction algorithm that preserves the topology of ...
0
votes
0answers
9 views

How to handle the case of multiple ground truth boxes having high IOU with the same predicted box?

In single shot detector the matching strategy between ground truth and predicted box starts with the following step: For each ground truth box we are selecting from default boxes that vary over ...
1
vote
0answers
26 views

Sigmoid activation functions do not seem to cause vanishing gradients

I have fitted a feedforward NN on the Pima diabetes data. The data consists of 8 features and 1 binary response. My model consists of 3 hidden layers parameterised by logistic sigmoid activation ...
0
votes
1answer
12 views

Number of input and output channels of MAX POOL layer

This is what Andrew Ng draws in his pooling layers video in the Coursera Deep Learning Specialization: and this is what he draws in Inception network video: Notice in first slide, number of input ...
0
votes
2answers
18 views

How does Keras Tokenizer choose tokens given a sentence?

I tried to find the answer to this question but I can't find anything, so I ask here: How does Keras Tokenizer choose tokens given a sentence of words ? To be more precise with what I want to know, ...
0
votes
0answers
17 views

What is leaking rate?

I am implementing an echo state network using TensorFlow and am studying the parameters, one of which is called "leaky". The documented definition is as follows: Float between 0 and 1. ...
0
votes
0answers
4 views

Studies on where to apply L2? [migrated]

Are there any studies on where (and maybe how much) L2 to apply per parameter? E.g. in a more complex neural network, e.g. some encoder-decoder, with different components, from my own experience, just ...
1
vote
0answers
15 views

Is my calculation of the partial derivative of the cost function with respect to a single weight in the first layer correct?

I'm trying to understand the chain rule of backpropagation. This is what I understood. Is it correct? $$ \frac{\partial E }{ \partial w} = \sum_{i} \frac{\partial E }{ \partial a_i^{(l)} } (\sum_{j} \...
0
votes
0answers
11 views

can i implement a control flow? [closed]

https://github.com/swap-253/Twitter-US-Airline-Sentiment-Analysis/blob/main/Airline_Sentiment_Analysis_Using_Bidirectional_LSTMs.ipynb how can i put the all per-processing of text data in this code in ...
0
votes
0answers
24 views

Is it ok if i post process my ML model's output used to predict stock movement?

So I made a machine learning model which predicts stock movements which returns 1 for the price going up or 0 for the price going down. without the post processing the training accuracy and testing ...
0
votes
0answers
17 views

NN regression model predictions incomprehensible

I'm trying to build a deep learning regression model for price prediction of AirBnB listings. As a baseline, I started with a simple 3-layer NN as follows: ...
0
votes
0answers
18 views

Multiple time series forecasting method

Your help will be very useful in the below exercise as I have issues in my try to identify the correct ML approach to solve it. My target here is to predict the value of the of 23:00 hour for the last ...
0
votes
0answers
12 views

Custom Tensorflow loss function that disincentivizes all black pixels

I'm training a Tensorflow model that receives an image and segments the image into foreground and background. That is, if the input image is w x h x 3, then the ...
0
votes
0answers
24 views

Model weights not updating on the higher layers

I would like to use a feedforward NN with 3 sigmoid hidden layers to demonstrate the vanishing gradient problem. I used the Pima dataset containing 8 features and it is binary classification task. I ...
0
votes
0answers
14 views

Aggregation vs binning continuous values

For any generic DNN, When should one aggregate a continuous variable and when should one put them into intervals instead? For example, the number of requests a user has sent in the last N days could ...
0
votes
0answers
9 views

Feature importance by removing all other features?

For neural network feature importance, can I zero-out all features except one in order to gauge that feature's importance? I know shuffling a feature is one approach. For example, leaving in the 4th ...
0
votes
0answers
5 views

Is video memory DRAM or SRAM?

Oftentimes even moderate size models, such as DeepMind AlphaFold2 (which requires 20GB RAM) can't fit into the video RAM (such as TPUv3 with 16GB RAM) and have to re-calculate activations during the ...
0
votes
1answer
14 views

Can a single label be a vector/matrix in a neural network and not a scalar?

My training data consists of individual sentences and each sentence has a few labels (say 10) and each of these labels has a discrete score from 1-10 -- so in essence, a single training example has a ...
0
votes
0answers
10 views

Lime explainer - Numpy broadcast error

I am working on a ML tutorial project with my own dataset. I built a ML model using training dataset and generated predictions using test dataset. Shape of test dataset is (418,10) My code for model ...
1
vote
0answers
6 views

Audio Classification with Counter

I'm trying to create a model that can identify one particular sound, and every time it hears that sound, it increases a counter by 1. So for example, if it hears a specific bird chirping ten times, ...
1
vote
0answers
26 views

Can an Imbalanced Datset be an oportunity for Transfer Learning with Neural Networks?

While solving classification tasks on imbalanced datasets with Neural Networks(NN) there are two general ways of handling imbalanced data: A. Resample the data, either with over or undersampling ...
0
votes
3answers
42 views

Interpretation of learning curve - neural network

When I run my three different neural networks I obtain the following learning curves using MSE. I believe that my model base is okay and is not overfitting or underfitting. Furthermore, I believe ...
0
votes
0answers
14 views

Back propagation with help of Taylor series in calculating derivative of loss function

I am wondering how to train a multi-layer neural network using back propagation algorithm with help of Taylor series when calculating partial derivative of loss function with respect to weights. I ...
1
vote
1answer
27 views

How to analyze repeated measure data for prediction?

In my work, we collect sales data of our products. We have a set of 1st level customers (lets call that group as jacks) with whom we do we business. These jacks then sell our products to end customers ...
0
votes
0answers
8 views

Question from a paper: How to transfer features of only negative samples from another device in a siamese network?

I am trying to understand a paper that uses a siamese network for authentication. In this paper (see Figure 4), they have two parts: mobile device and cloud. Negative samples are stored in the cloud ...
0
votes
0answers
8 views

Train a model when input can contain a smaller options output with the correct output

I have service order lines to charge customers, each line needs to be set to an actual product. If the customer had only one product, so all lines are set to that product. But, if the there are many ...
1
vote
0answers
11 views

Modeling events with an intermediate stage

For a lot of prediction problems, there's an intermediate stage which must occur for the target event to occur. For example, to graduate from college, one must first be accepted. For an internet ad to ...
0
votes
0answers
8 views

Tune SIRD Model Parameters using a Neural Network

I want to use a neural network to predict the number of new cases of COVID-19. For the same, I have decided to use an SIRD (Susceptible-Infected-Recovered-Deceased) Model, which is parameterized by ...
0
votes
0answers
7 views

Neural Network Architecture for Mixed Frequency Data

I have accelerometer and gyroscope data generated at 119 Hz. I have magnetometer data generated at 20 Hz. I would like to build 2D-CNN based on this data. One solution is that decreasing the freq. of ...
1
vote
0answers
17 views

How should I think when I want to compare mu and sigma for different images in VAE?

I'm searching for a way to compare mu and sigma values of the encoder network's output of variational autoencoders. In detail, imagine I trained my VAE on the MNIST digits dataset using the official ...
1
vote
0answers
23 views

What kind of ML approach is more suitable for detecting event related signal changes?

First of all please if there is a better way to phrase my question let me know. It will help with search. ( This part : "detecting event related signal changes" ) Here you can see 4 black ...
1
vote
0answers
21 views

How to predict data from sequence of sequences of variable size?

input data ...

1
2 3 4 5
80