3
$\begingroup$

In a conditional GAN, we give a random noise along with a label to the generator as input. In this paper, I don't understand why in one section of the paper, they say they are giving the random noise as input and the in another section of the paper they are saying it is concatenated to the output.

page 2

page 2

page 2 footnote

page 2 footnote

page 3 model setup section

page 3 model setup section

little overview of the paper: Code switching is a phenomenon in spoken language where we switch between two different languages. Mixed language models improve the accuracy of automatic speech recognition to higher degree but the problem is less availability of mixed language written sentences. Thus, as a data augmentation technique, a conditional GAN is developed to synthesize English, Mandarin mixed sentences from a pure Mandarin sentence. The trained generator acts as an agent telling which words in the Mandarin sentence have to be translated. It outputs a binary array (of length equal to input Mandarin sentence length). Both generator and discriminator are BLSTM networks.

#####EDIT: The author accepted that it is a typo, noise should be concatenated after the embedding layer not to the output of BLSTM Author's reply: It is a typo in page 3. The noise is concatenated with the output of the embedding layer. Thanks for your correction. #####

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
+50
$\begingroup$

As stated in 3.2 Model setup

The generator G is made up of embedding layer, one bidirectional long short-term memory (BLSTM) [21] layer, one fully connected (FC) layer.

And

Gaussian noise is 10-dim vector concatenated with the output of BLSTM

So the noise is concatenated to the embeddings computed by the BLSTM for each time step. I think they concatenate the same 10-dim vector to each embedding output of the BLSTM but this is not clear.

The obtained vector (embedding concatenated with noise) is fed to the Fully-connected layer with softmax activation to compute the probability that the word should be translated or not.

Concerning page 2 footnote about "ignoring the noise"

I think they simply ignore it in their explanations of the model, but it is effectively used in the model by simply appending a gaussian noise vector to the embeddings output of the BLSTM.

As you see in the following figure they do not show the noise, but it is actually concatenated to the output embeddings of the BLSTM in the Generator.

enter image description here

Improve this answer
$\endgroup$
3
  • $\begingroup$ I understood the ignoring noise part, could you verify this? say if my sentence length = 8, vocab size = 5000, embedding matrix = 100 x 5000, batch size = 20 input to embedding layer = (5000, 20, 8), output of embedding layer = (100, 20, 8), concatenate noise = (110, 20, 8), fc layer weights = 1 x 110, output = (1, 20, 8) $\endgroup$
    – Mathav Raj
    Sep 2, 2020 at 15:30
  • $\begingroup$ Yes this seems correct to me $\endgroup$
    – Adam Oudad
    Sep 3, 2020 at 7:42
  • $\begingroup$ Why is the noise needed here though after the embedding layer? The authors of this paper have adopted this GAN from SeqGAN but there I did not see any mention of noise $\endgroup$
    – Mathav Raj
    Nov 16, 2020 at 17:34
1
$\begingroup$

They say they ignore the noise z in the input. Is that why they concat it later?

So the over all model is more simple?

Improve this answer
$\endgroup$
5
  • $\begingroup$ but don't we need the noise at the input of the generator? $\endgroup$
    – Mathav Raj
    Aug 27, 2020 at 3:49
  • $\begingroup$ True, but they explicitly mention in the paper they do it to reduce the complexity. Can you contact the author? Another option would be to find a paper with a similar equation and see whether they have implemented it with z. Then you can compare both (possibly run it) and see why they have done so! $\endgroup$
    – Academic
    Aug 27, 2020 at 4:29
  • $\begingroup$ I have mailed all the authors a week ago, still waiting for a reply.Yes I tried cross referencing with papers that cited this paper, they have not touched about this aspect, maybe I should try mailing them too $\endgroup$
    – Mathav Raj
    Aug 27, 2020 at 5:45
  • $\begingroup$ Also what good is adding the noise at the lstm output side. From what I have learnt so far, we are deriving a distribution as close as possible to the original from random noise , isn't it? $\endgroup$
    – Mathav Raj
    Aug 27, 2020 at 5:47
  • $\begingroup$ oh okay, lets see if they do! So the noise is added to the output of the generator (which is the lstm output). So that is basically for the discriminator. $\endgroup$
    – Academic
    Aug 27, 2020 at 7:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.