1
$\begingroup$

I have the natural language sentences as follows:

This is a black chair. It is next to the table.

Each phrase that represents an object is annotated with an object Id. For example, in the above sentence, we have:

This: 15, black chair: 15, It: 15, table: 14

(where, 14 and 15 are object Ids)

I would like to train a model to predict the object Id of each phrase representing an object for a new sentence. From what I understand, each training example will consist of the following structure:

Inputs: sentence + object phrase

Output: object id (from 18 available ids)

I would need to repeat the above for each object phrase in a sentence

My question: How do I prepare the training data for this task? How do I represent each object phrase (eg: 'black chair') and each sentence for training the neural network?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Nlp like spacy remove stop works and identify proper nouns and nouns. It does fairly well at identifying nouns but is not perfect. Try a sample of your data and see how accurate your percentages become. Use the medium model

Improve this answer
$\endgroup$
3
  • $\begingroup$ Post a paragraph and I will run it through some code and I will post back the results $\endgroup$
    – Golden Lion
    Commented Feb 19, 2021 at 0:49
  • $\begingroup$ Thank you for your response. I'm able to extract noun chunks for the sentences. My question is how do I convert this data into vectors so that I can train a classifier? $\endgroup$
    – Sid
    Commented Feb 19, 2021 at 1:17
  • 1
    $\begingroup$ Use count vectorizor or tfidf tokenizers $\endgroup$
    – Golden Lion
    Commented Feb 19, 2021 at 4:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.