New answers tagged

0

The CTGAN paper (github) use a VAE to generate multivariate synthetic data from tabular data. See section 4.5.


0

Currently, I've implemented the following solution inpired by N-Pair Loss published from NIPS 2016: import torch from torch import nn from matplotlib import pyplot as plt import seaborn as sn class NPairsLoss(nn.Module): """ The N-Pairs Loss. It measures the loss given predicted tensors x1, x2 both with shape [batch_size, hidden_size], and ...


0

In order to verify the encoder efficiency than it is better to design some down stream task like classification, similarity , etc. Based on that u can validate your encoder. U may refer this link for downstream Task.


0

You could use Tensorboard, via the tensorboardX interface. That allows you to load models from PyTorch (and Chainer, MXNet etc.) into Tensorboard. This will then show you the full graph interactively. From the homepage:


0

2) I would try to use this: https://pytorch.org/docs/stable/data.html#torch.utils.data.SubsetRandomSampler It is a sampler that limits the dataloader to certain index. Wrap it around with a batchsampler. Also, modify the dataset, so that it will give (image, target, index) instead of the normal (image, target). 1) collate_fcn() in dataloader does that......


0

That could be due to many reasons. One of the reasons could be gradient vanishing/explosion. Changing your nonlinear function could be a solution. For example, you can use Relu function instead of tanh function. Also, using a early stopping rule could prevent this problem.


0

1) How do I process each sample individually? Do I go about this by setting batch size = 1? Or is there any advantage to disabling automatic batching. If so, how do I go about this. If you set batch size to 1, then you are effectively disabling batching. Samples will be processed one-at-a-time and gradients will be computed for single samples. This is not ...


0

Every word2vec model requires a dictionary that maps an index to a string literal. This dictionary mapping greatly speeds up training because the training is completely integer-based. The string literal values can be any length, from individual characters to phrases.


0

So for Binary Prediction in Pytorch the ideal loss function would be the Binary Cross Entropy Loss which is available along with all the other error functions in the nn submodule in can be called as follows nn.BCELoss() it has parameters reduction(mean and sum) and weights(pre-determined weightages). It's documentation can be found here Ensure that the ...


0

I can't say for sure but I think the issue here is you're not subtracting the mean of the rewards. The idea is that actions with above average reward are positive after mean normalization, while actions with below average reward are negative after mean normalization. Your update step is -log(P(action))*reward, which you then minimize with your optimizer. ...


0

The cudnn implementaton of the LSTM has determinism issues that appear to be fixed in the 7.6.1 release. Check your cudnn version. https://github.com/pytorch/pytorch/issues/18110


0

One thing to try is to create a lean version of Pytorch that only has what you need. I did something similar for an AWS Lambda layer, here's what I was able to delete (this was for Pytorch 1.1, things may have changed since): find . -type d -name "tests" -exec rm -rf {} + find . -type d -name "__pycache__" -exec rm -rf {} + rm -rf ./{caffe2,wheel,...


0

What I've opted for at the moment is packing all of the samples into a single HDF5 table buffer, and keeping a separate table with metadata that tracks each individual sequence's buffer position and length. This works, but I won't be marking this answer as correct because I'm not satisfied with it. This storage method is very poorly suited to editing, and it'...


2

There are not only 2, but many implementations of BERT. Most are basically equivalent. The implementations that you mentioned are: The original code by Google, in Tensorflow. https://github.com/google-research/bert Implementation by Huggingface, in Pytorch and Tensorflow, that reproduces the same results as the original implementation and uses the same ...


Top 50 recent answers are included