New answers tagged

1 vote

issue with data sample

As you can see in the documentation, the data provided to train_test_split need to be arrays of data (e.g. numpy arrays or pandas dataframe), not strings. If your ...
Oxbowerce's user avatar
  • 7,377
0 votes
Accepted

How bootstrapping works for prediction intervals?

You've astutely observed that the PI outputs (represented by the gray lines) can vary significantly depending on the specific model setup and data characteristics. Several key factors influence this ...
Daniel Curtis's user avatar
0 votes
Accepted

What's the purpose of using MLM when pretraining?

When you train any model, you need to train it on a specific task and its associated loss function. Masked-language modelling is the token-level task used for training transformer encoders like BERT ...
noe's user avatar
  • 26.1k
0 votes

What are the differences between contextual embeddings of Bidirectional-LSTM and Transformer?

transformer tech is evolved version of LSTM/rnn based tech by improving on speed and quality. particularly the tokenization, pre-training method, multi-head attention, longer input size are improved ...
ashishfa's user avatar
0 votes

How to find optimized x values (input features) after training in deep learning?

There’s nothing special about your neural network. It is a function of some inputs. You know from (multivariable) calculus how to find the points giving maxima and minima of functions. The fact that ...
Dave's user avatar
  • 3,738
1 vote
Accepted

Fine tuning or just feature extraction or both using Roberta?

Feature extraction does not modify the model's weights, it just uses the model in inference mode to get its outputs and hidden states. Given that you are updating the model's weights, you are fine-...
noe's user avatar
  • 26.1k
1 vote

LIME is observing categorical features even though I am not passing any categorical features

this problem was reported in this github Issue where I proposed the following : This may happen if the training data you give to the LimeTabularExplainer has $n$ ...
Julien CRAMBES's user avatar
0 votes

How to clone Python working environment on another machine?

TLDR: the best method is below: first, in old server rsync -avr /home/xxxx/anaconda3/envs/your_env_name your_login_name@new_server_ip:/home/xxxx/miniconda3/envs/ ...
plo's user avatar
  • 1
1 vote

why validation accuracy is stuck at 75%?

My first inclination is that your dataset size is relatively small, and you have many hidden layers, which increases the likelihood of your model overgeneralizing on the training data, leading to ...
Daniel Curtis's user avatar
2 votes

Affine 2D mapping in python

Bias term $c$ can be eliminated (eg by subtracting the averages from $A$ and $B$) and the approximation be framed as: $$B = M \cdot A$$ Using minimum sum of squared residual errors criterion (similar ...
Nikos M.'s user avatar
  • 2,323
0 votes

TensorFlow LSTM model with lower epoch loss, but higher average RMSE. How/why?

In this case, seeing high RMSE values indicates large errors still happening even though your average MSE is smaller. Remember, RMSE heavily penalizes large errors, so in your case, the frequency of ...
Daniel Curtis's user avatar
1 vote

What ML model is best suited for an intelligent search assistant?

You could use OpenAI's API, GPT3.5 and use those to build a retrieval augmented generation model (RAG) to search a vector store of contexts of your choosing. You can do this using a couple of ...
bdeck8317's user avatar
1 vote
Accepted

What do special tokens used for in Roberta?

The special tokens <s> and </s> are indeed utilized for specific NLP tasks such as question answering, sequence ...
RegressIt's user avatar
  • 361
0 votes
Accepted

Why is there a kmeans and kmeans_plusplus function in scikit-learn?

I assume the purpose of having two functions is to provide users with flexibility and clarity in their choice of initialization method. The default behavior of K-Means uses the k-means++ method, which ...
RegressIt's user avatar
  • 361
0 votes

Customer Lifetime expectancy

You could follow a methodology that involves computing the survival rate and then using it to calculate the lifetime expectancy as follows: Compute Survival Rate: For a given ...
RegressIt's user avatar
  • 361
0 votes

How to optimize my CNN classification architecture

From looking at what you've provided for your problem i'd consider the following; Reduce the Number of Layers: The original model had multiple convolutional and dense layers, which increased the model'...
RegressIt's user avatar
  • 361
-1 votes
Accepted

Illegal action reward strategy for reinforcement learning : reward shaping and termination / truncation

Penalty for Illegal Actions: The penalty for illegal actions is an important aspect of reinforcement learning. In your case, applying a -100 penalty for illegal actions is a reasonable approach. This ...
RegressIt's user avatar
  • 361
0 votes
Accepted

Why was the learning rate decreased for Roberta compared to LSTM?

The learning rate is a hyperparameter, and it has to be tuned for a specific neural architecture. Also, the optimal learning rate is affected by other hyperparameters, like the batch size. We don't ...
noe's user avatar
  • 26.1k
2 votes
Accepted

What do these terms mean in the context of Roberta?

Fine-tuning means that you modify the model to adapt it to your data. Feature extraction means that you don't modify the model. In the case of RoBERTa, it means that you feed your data to the model, ...
noe's user avatar
  • 26.1k
0 votes
Accepted

What are the differences between Embedding Layer and Roberta Embedding?

An embedding layer is just a building block to be used as part of neural architectures. It is just a lookup table whose purpose is to represent tokens as vectors, and to learn these vectors as part of ...
noe's user avatar
  • 26.1k
0 votes
Accepted

What are the differences between contextual embeddings of Bidirectional-LSTM and Transformer?

Here are some differences: Computational complexity: LSTMs have linear complexity $O(n)$, because you need to process input tokens one by one, while transformers have constant $O(1)$ complexity ...
noe's user avatar
  • 26.1k

Top 50 recent answers are included