Rephrase Neural Network : Find the Fittest Word for Given Meaning/Explanation

Question

I am a sophomore student who's interested in deep learning and its method layering up some linear/non-linear operations and constructs up the complex function through the network.

I'd like to comprise a network that finds the word upon the given explanations :

e.g.1 input: a spherically shaped fruit which tastes sour and sweet and gets red when matured well output: apple

e.g.2, input: to make up the boundary between multiple options for the base of one's behavior or decision output: determine.

So basically it's a classification problem but the class might be more than 20,000.

I'd like to name this task "rephrase neural network".

Any good reference or starting point, or, any network recommendation for comprising this neural network?

Answer 1

In a perfect setting, where you have infinite sentences (definitions).

You could use some kind of attention models by chunking sentences into smaller pieces.

[a spherically shaped] [fruit] which [tastes sour] and [sweet] and gets [red] when matured well.

With a distributed representation framework, every little piece of information is simply like a binary indicator (the feature exists or not).

Maybe each piece is vague by itself and it could not tell you anything about the target yet by combining them together they define useful information, see products of experts

To learn p(Y|X) where x consists of the sentence, y is the embedding of some class, so may similar classes share similar sentences.
If you have a model trained on the sentences and learned to project similar sentences near to each other, and assume you learned the word classes by some word2vec like algorithm, you could generalize to new pair even if the sentence and that class target were never paired (never seen in your training data), their respective feature vectors have been related to each other by the mapping function.

At the end of the day if you have a smooth function and your input is continuous then any small change of the input will introduce small change at the output, yet how could we truly extract the knowledge exists in the data. see Generating Sentences from a Continuous Space