Skip to main content
11 votes
Accepted

How high of a correlation coefficient of a feature with a target variable is considered too high?

You should not remove features just because their correlation to the target is high. Such high correlation is a sign of potential target leakage. You should understand why their correlation is high. ...
noe's user avatar
  • 26.7k
6 votes
Accepted

How does variational autoencoders actually work in comparison to GAN?

VAEs were a hot topic some years ago. They were known to generate somewhat blurry images and sometimes suffered from posterior collapse (the decoder part ignores the bottleneck). These problems ...
noe's user avatar
  • 26.7k
6 votes
Accepted

What do we mean by optimizer.zero_grad()

You are right, $g_t$ is what becomes zero when we invoke optimizer.zero_grad(). After invoking zero_grad(), we compute the ...
noe's user avatar
  • 26.7k
5 votes
Accepted

The end-to-end Training Process for Knowledge Distillation

Correct. Nevertheless, you may see some variations, e.g. when the number of classes is very large, the size of the transfer dataset can be huge (this is the case of text generation, where the ...
noe's user avatar
  • 26.7k
5 votes
Accepted

LLMs for text generation

Yes, there are open multimodal LLMs that you can fine-tune yourself, like LlaVa, NextGPT, IDEFICS or SPHINX. Closed multimodal LLMs like GPT-4v don't offer a way to fine-tune them yet.
noe's user avatar
  • 26.7k
4 votes

Modeling uncertainty from known physics

Yes. I was searching for the same thing a while back and I came across the concept of PINNs. Physics-informed neural networks (PINNs) are neural networks that encode model equations, such as partial ...
spectre's user avatar
  • 2,105
4 votes

How to Justify Anomalies Detected by Unsupervised Anomaly Detection Models?

Your domain experts and other stakeholders are primary sources of information. Work them and work with them in order to find out what is an appropriate definition for "normal" and "...
Jon Nordby's user avatar
  • 1,482
3 votes

how to select number of number of layers and neurons in neural network(RNN) in standard way?

To answer your question: First of all, there is no as such rule of thumb for selecting no of layers or no of neurons. But you can follow some tips: Selecting the number of layers and neurons in a ...
Harshad Patil's user avatar
3 votes
Accepted

Why my validation loss and accuracy decays over epochs?

You are experiencing a lot of overfitting on the training set here. I would go back and see if there are any inherent issues with the data (scaling? class imbalance? etc.) before diving into modeling. ...
NLP from scratch's user avatar
3 votes
Accepted

Why the test accuracy showing some odd behaviour in comparison to train accuracy?

Why the test accuracy showing odd patterns ? As @mohottnad mentioned in the comment, it appears your model overfits. It means that it doesn't generalise well and works badly on testing data. I don't ...
Tomasz Witkowski's user avatar
3 votes

Car Make and Model detection

For detecting the make and model of cars from images with high accuracy across a large number of classes, I would recommend a convolutional neural network (CNN) architecture tailored for fine-grained ...
Multivac's user avatar
  • 2,969
3 votes
Accepted

What are the differences between BPE and byte-level BPE?

Byte-level BPE is a subtype of BPE that uses bytes instead of characters as basic token component. In the character-level BPE, the vocabulary is composed of sequences of characters that appear ...
noe's user avatar
  • 26.7k
3 votes

What is the best way to train a neural network with a variable number of inputs?

The expected accuracy of the model should increase when all 5 inputs are available Not necessarily true, NNs do the learning, and may decide that the most import features are A,C,D, and the others ...
fam-woodpecker's user avatar
2 votes
Accepted

Why Deep Learning / Neural Networs don't achieve state of the art results in tabular data problems?

A few years later after this post, some authors wrote a paper at NeurIPS about it. Why do tree-based models still outperform deep learning on typical tabular data? https://proceedings.neurips.cc/...
Carlos Mougan's user avatar
2 votes

Transformers doubt

We should not mistake the K, Q and V vectors received by the multi-head attention block with those received by the scaled dot-product block. The K, Q and V vectors that are fed to the multi-head ...
noe's user avatar
  • 26.7k
2 votes

improving Neural network regression model

First: thanks for a great explanation, it is easy to help when the question is well posed. For the question per se: you can try a polynomial. Given the shape of your data I'd try a ...
Memristor's user avatar
  • 256
2 votes
Accepted

Difference between Word2Vec and contextual embedding

First, some clarifications: Non-contextual word embeddings (e.g. word2vec) assign a real-valued vector to each word. Contextual embeddings (e.g. BERT-based) assign a vector to each "token". ...
noe's user avatar
  • 26.7k
2 votes

Should a Learning Rate Scheduler adjust the learning rate by optimization step (batch) or by epoch?

It very much depends on your training setup. More specifically, the interaction between learning rate and batch-size plays an important role on how the learning rate decay affects the learning ...
Mr Tsjolder from codidact's user avatar
2 votes
Accepted

Correlation between multiple time series

There is not one easy solution to your problem, as it is not well-defined, but there are ways to narrow it down and come closer to a solution. Here are some points that can help you to do so. 1. ...
Broele's user avatar
  • 1,362
2 votes

Implementation of Graph Neural Network for Image Classification

GNNs can be used for image classification and in future may prove to be a better approach, but as of now, CNNs are state-of-the-art. Steps for image classification using GCN: -> Converting the ...
shivani's user avatar
  • 140
2 votes
Accepted

How did Andrej Karpathy make the LSTM output byte values for sampling Shakespeare?

The typical approach to have neural networks generate discrete outputs is to make them generate a probability distribution over the space of options. Then, you can choose the option with the highest ...
noe's user avatar
  • 26.7k
2 votes

Deep learning model produces very different results when classifying the same samples

You're facing a reproducibility issue, I think. In the first case (having clear_session and make_model within the loss) you get ...
Luca Anzalone's user avatar
2 votes
Accepted

Role of stateful parameter vs shuffle parameter in LSTM keras

When we say reset states, we mean hidden ones or cell states? We mean both the initial cell AND hidden state. If I set stateful=False , what's the difference keeping shuffle=True vs shuffle=False in ...
noe's user avatar
  • 26.7k
2 votes

Neural Network for binary classification not working

I think the problem is here y_pred = model.predict(X_test) y_pred_classes = y_pred y_preds = np.argmax(y_pred_classes, axis=1) Your model outputs a single value (<...
Karl's user avatar
  • 671
2 votes
Accepted

Confusion with tensorflow's Sequential Dense Layers

Keras expects the inputs $X$ to be batched, i.e., of shape $B\times D$ in your case (for a feed-forward NN), where $B$ is the batch size (by default is 32, which can be assigned in ...
Luca Anzalone's user avatar
2 votes
Accepted

activation=tf.keras.activations.relu vs activation='relu'

Your results were probably result of the randomness in the training. Both activation=tf.keras.activations.relu and ...
noe's user avatar
  • 26.7k
2 votes
Accepted

CLIP Paper or CLAP Paper (I dont understand the loss function) - Can you help?

Start with a batch of (audio, text) pairs denoted $( X_a, X_t )$. The text data is passed through a text encoder $ f_t $ to produce a text embedding $ \hat{X}_t = ...
Karl's user avatar
  • 671
2 votes

What to do if the learning curve of a loss does not show down trend?

1/4 epoch is not long, but if it lasts for 1-4 epochs, it's usually a sign of a wrong learning rate, either too high or too low. Another problem could be that your data is not scaled, if the variables ...
Max's user avatar
  • 31
2 votes

Why doesn't CLIP use a pretrained large language model as the text encoder?

The link you provide states multiple reason why, namely: "We know from GPT-2 and 3 that models trained on such data can achieve compelling zero shot performance; however, such models require ...
Valentin Calomme's user avatar
2 votes

text extraction from bank statements from pdf format

I think the most convenient way to do that in Python is by using a tabula library. I happened to do similar task with pdf files and probably the easiest way to do so is to use tabula.read_pdf function....
Tomasz Witkowski's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible