4

The reason you're seeing BERT and its derivatives as benchmarks is probably because it is newer than the other models mentioned and shows state-of-the-art performance on many NLP tasks. Thus, when researchers publish new models they normally want to compare them to the current leading models out there (i.e BERT). I don't know if there has been a study on the ...


3

Receptive field refers to the number of input pixels that a convolutional filter will operate on. There's a nice distill article about how to calculate receptive field size for your filters (with a nice visualization of receptive field size) and an interactive calculator here if you're only curious about how receptive field size grows with changes to depth ...


2

You have a big dataset and you get new instances//data every 2 months. First you should select with which data you want to train. Since your data is big and there is the probability than data from 2 years ago is not as relevant as the data from the last month you can consider doing a Roll out// slidding window validation. This way you will only select the ...


2

From what I understand - your problem is "sample selection bias" problem. Any kind of pattern to select a subset out of large data may lead to bias. This raises two question. How to choose? Random/stratified random (If you have multiple classes) under sampling to obtain a smaller subset. How big to choose? we can set percentage of undersampling. Reducing ...


2

CNNs like U-Net extract lower level features like edges on lower layers (i.e. the first convolutional layers) and higher level features on higher layers (i.e. convolutional layers closer to the final linear layers). This principle is losely inspired by how visual perception is implemented in the Visual Cortex among humans (and other animals). In a CNN the ...


2

Your model probably overfits. This articles provides an easy to read intro to the topic. As a very first step I suggest to plot your learning curves and look for epochs with lower validation loss. Also, this helps to properly diagnose your model in terms of model capacity. Training a model for less epochs is one way to reduce model capacity and avoid ...


2

There is plenty of methods to calculate feature importance. I recommend trying two of them LIME and SHAP. I don't want to copy-paste material and tutorial provided by the author so please refer to these two repositories.


2

First, I wouldn't use the word "noisy" here because if you know which instances are "wrong" then these are not noise, they are negative examples. In my opinion "noisy" is when positive and negative cases are mixed together in a way that makes it difficult (or impossible) to distinguish between them. I think this matters because you're more likely to find ...


1

It depends how deep technichally you want to go. You can apply a slight modification of a Survival methods/ cox models that relate the time that passes, before some event occurs, to one or more covariates that may be associated with that quantity of time. Also if you group de features you can make the problem look like as a classical binary classification ...


1

I think LSTM based RNN network having series of image data with a speed label, can be useful for training such model and you can try if you get some data set. Please do share your result as well.


1

Let $n$ be a convolutional layer with dimensions $w' \times h' \times c'$. Then each of its $c'$ filters is connected to all $c$ filters (or channels*) of the previous layer. I find it helpful to look at the number of weights here: A single filter of that convolutional layer $n$ with kernel size $k'\times k'$ will have $c \times k' \times k'$ weights. And ...


1

Your reasoning is perfectly correct. Augmentation is just a process, which helps you cover your domain better. You should only pick operators that help you. Abusing augmentation can definitely mess up your model. It's always good idea to print data at those limits, to check yourself. Try also to think, how data will be acquired on production. Albumentations ...


1

There is a particular library called as ReduceLROnPlateau, that will reduce the learning rate, based on the factor value you mention. And this seems working good for all problem cases.


Only top voted, non community-wiki answers of a minimum length are eligible