Take the sentence:
If you see a green light then you may cross the road.
I propose a neural network which produces as output from this sentence 3 masks and a classifier as follows:
input : If you see a green light then you may cross the road. output 1: XX XXXX output 2: XXXXXXXXXXXXXXXXXXXXX output 3: XXXXXXXXXXXXXXXXXXXXXXX output 4: [if-then-statement]
So basically we train the neural network on a corpus to split the sentence into it's most general structure. Then we take the masks and use it on the input to produce a new phrase. e.g. let's take the mask from output 2:
input : you see a green light output 1: xxx output 2: xxx output 3: xxxxxxxxxxxxx output 4:[event]
This time the neural network classifies the input as an event and creates masks for splitting the sentence into subject-object-verb.
So we repeat the process using the masks created to change the input until we have parsed the sentence into a tree-like representation. We use the same neural network each time.
If it fails to find a classification for an input it could go back up one level and try a different classification. (Much like a human might do if it doesn't quite understand a sentence).
The result would be an internal representation which might consist of a tree-like structure of masks and classifications. Which might then be more useful for reasoning about the meaning of the sentence.
This is kind of a top-down approach. I'm not sure how it would learn these masks from un-marked speech, but certainly I think it could learn them from a tagged corpus. I think maybe a similar approach might be used backwards to generate sentences.
Has this sort of thing been tried before?