What's the deal with Deno? We talk with a major contributor to find out. Listen now.
Data Science

Questions tagged [pretraining]

Ask Question

The tag has no usage guidance.

8 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
1
vote
1answer
26 views

Does finetuning BERT involving updating all of the parameters or just the final classification layer?

Currently learning and reading about transformer models, I get that during the pretraining stage the BERT model is trained on a large corpus via MLM and NSP. But during finetuning, for example trying ...
0
votes
0answers
7 views

Any pretrained models for Style Transfer trained with large images (>512px)?

I have found some pretrained models that are prepared for small image sizes. https://tfhub.dev/google/magenta/arbitrary-image-stylization-v1-256/2' (256x256px) https://keras.io/api/applications/vgg/#...
1
vote
1answer
56 views

Deploying multiple pre-trained model (tar.gz files) on Sagemaker in a single endpoint

We have followed the following steps: Trained 5 TensorFlow models in local machine using 5 different training sets. Saved those in .h5 format. Converted those into tar.gz (Model1.tar.gz,...Model5.tar....
0
votes
0answers
10 views

How to use an encoder to do feature extraction

I'm newbie with all of data science. I have a pre-trained U-Net network from which I get its encoder. Now I have to use a picture to get its features. With the whole U-Net I do this with fit method: <...
1
vote
1answer
18 views

Are there any objections to using the same (unlabelled) data for pre-training of a BERT-Based model and the downstream task?

I'm looking to train an Electra model using unlabelled data in a specific field. Are there any objections to using the same data for unsupervised learning and then using the same data downstream for ...
0
votes
0answers
15 views

How to go about fine-tuning BERT using a next-sentence task

I've got a large corpus of documents, and I want to use bert to generate embeddings for a variety of predictive tasks. The documents are multi-sentence, in a non-standard domain, and have labels at ...
0
votes
1answer
20 views

Semantic segmentation with greyscale images

I'm trying to reproduce a research with greyscale images instead of colour images. I have found that there are pre-trained networks, like VGG16, with ImageNet. But that dataset has colour images, and ...
0
votes
1answer
11 views

Question of pretraining text-generation task, it seems that pretraining is not work for a small model?

My task is to generate keywords from sentences. I pretrain a text-generation model. I mask the sentences' tokens and predict the whole sentences' tokens. Pretraining batch_size = 8 and step = 1000000 ...