Questions tagged [pretraining]
The pretraining tag has no usage guidance.
16
questions
0
votes
1answer
11 views
What is the common practice for NLP or text mining for non-English?
A lot of natural language processing tools are pre-trained with corpus in English. What if ones need to analyze, say, Dutch text? The blogs I find online are mostly saying traslating text into English ...
0
votes
1answer
26 views
Where to get models with weights instead of only weights? What's the purpose of .h5 files?
I have downloaded .h5 files from qubvel/resnet and qubvel/efficientnet. I was trying to use some models as a backbone for my model but I'm getting the following ...
1
vote
2answers
51 views
Would there be any reason to pretrain BERT on specific texts?
So the official BERT English model is trained on Wikipedia and BookCurpos (source).
Now, for example, let's say I want to use BERT for Movies tag recommendation. Is there any reason for me to pretrain ...
0
votes
0answers
10 views
(Pre)training an Embedding Layer via prediction (as an alternative to similarity) - does that makes sense?
Similarity is the go-to way to train embeddings - use a similarity matrix (eg dot product) between the embeddings of two inputs, train to increase it for connected inputs and decrease it for inputs ...
0
votes
1answer
60 views
How to access GPT-3, BERT or alike?
I am interested in accessing NLP models mentioned in scientific papers, to replicate some results and experiment.
But I only see waiting lists https://openai.com/blog/openai-api/ and licenses granted ...
0
votes
0answers
24 views
Pretrained CNN model on animal dataset ( turtles images if exist )
I was wondering if there is a pretrained CNN model on an animal dataset. I am participating in a turtle face detection competition and was wondering if there was a pretrained image model to fine tune ...
0
votes
2answers
30 views
Logic behind pre-trained weights and transfer learning
I am not sure about the logic behind, how pre-trained weights actually make sense and translate into a new problem.
To be more specific; for example in a object detection network, how would a model's ...
0
votes
0answers
15 views
With one pretrained CNN model do I get only one vector of descriptors for an image?
I am given a dataset of 2D medical images. I am asked to extract image descriptors from the hidden layer of the neural network pre-trained on the ImageNet dataset. I consider to use two networks: ...
1
vote
2answers
178 views
Does finetuning BERT involving updating all of the parameters or just the final classification layer?
Currently learning and reading about transformer models, I get that during the pretraining stage the BERT model is trained on a large corpus via MLM and NSP. But during finetuning, for example trying ...
0
votes
0answers
33 views
Any pretrained models for Style Transfer trained with large images (>512px)?
I have found some pretrained models that are prepared for small image sizes.
https://tfhub.dev/google/magenta/arbitrary-image-stylization-v1-256/2' (256x256px)
https://keras.io/api/applications/vgg/#...
1
vote
1answer
670 views
Deploying multiple pre-trained model (tar.gz files) on Sagemaker in a single endpoint
We have followed the following steps:
Trained 5 TensorFlow models in local machine using 5 different training sets.
Saved those in .h5 format.
Converted those into tar.gz (Model1.tar.gz,...Model5.tar....
0
votes
0answers
13 views
How to use an encoder to do feature extraction
I'm newbie with all of data science.
I have a pre-trained U-Net network from which I get its encoder.
Now I have to use a picture to get its features.
With the whole U-Net I do this with fit method:
<...
3
votes
1answer
35 views
Are there any objections to using the same (unlabelled) data for pre-training of a BERT-Based model and the downstream task?
I'm looking to train an Electra model using unlabelled data in a specific field. Are there any objections to using the same data for unsupervised learning and then using the same data downstream for ...
0
votes
0answers
146 views
How to go about fine-tuning BERT using a next-sentence task
I've got a large corpus of documents, and I want to use bert to generate embeddings for a variety of predictive tasks. The documents are multi-sentence, in a non-standard domain, and have labels at ...
0
votes
1answer
62 views
Semantic segmentation with greyscale images
I'm trying to reproduce a research with greyscale images instead of colour images.
I have found that there are pre-trained networks, like VGG16, with ImageNet. But that dataset has colour images, and ...
0
votes
1answer
16 views
Question of pretraining text-generation task, it seems that pretraining is not work for a small model?
My task is to generate keywords from sentences.
I pretrain a text-generation model. I mask the sentences' tokens and predict the whole sentences' tokens.
Pretraining batch_size = 8 and step = 1000000
...
