5
$\begingroup$

I have data containing both numbers and raw text related differently like:

  1. The power of diesel generator is 15kva. I need a single phase generator.
  2. Three phase generator required of 140 kva.
  3. Need 70g/m^2 GSM a4 size copier paper.

I removed the numerical data and trained a classifier on vector obtained from Doc2Vec and got around 80% accuracy.

Is there a way to take in account the numbers (here 15,140,70) involved in text while creating a doc2vec vector? or if there is any way to take in account these numbers in order to improve on the accuracy?

$\endgroup$
  • $\begingroup$ if you feel that one of us has answered your question sufficiently please mark it as answered :) $\endgroup$ – HFulcher Feb 22 at 16:44
2
$\begingroup$

Having read the original research on word embeddings published by Google and others, I'm sad to say that it made no mention of how to handle such data. While you could hypothetically just run a model over the raw, unedited text, depending on your desired end-use (and the dimensionality of your data), preprocessing might be highly beneficial.

If I was trying to establish a word-embedding model and didn't have enough raw data containing numbers to allow the model to figure out differences between different numbers and their uses, I'd likely use regular expressions to help simplify the inputs. While you could replace all numbers (of any length) into a uniform label (i.e. 'NUM' as @HFulcher suggested), depending on your application you may be losing data by doing this. You may want to differentiate between prices, fractions/percentages, ordinal numbers, dates, etc. This can be done by varying the labels that you replace the raw numbers with ('PRICE', 'DATE', etc). Hypothetically you could also swap between numerals and spelled-out numbers if they fit a certain category (like falling under a certain ceiling or being ordinal).

$\endgroup$
3
$\begingroup$

If you are passing in the numbers as a string type Doc2Vec should just accept these as part of the vocabulary without an issue. Have you tried running the Doc2Vec training with the numbers still included?

One way to solve the issue if you can't just pass the values in as is is to give them a unique identifier (e.g. < NUM >). It shouldn't matter what the value is as Doc2Vec should in theory not be able to draw any information (and really shouldn't) from that. Instead if you teach it when a number occurs in a document (by using the identifier) that might improve accuracy.

$\endgroup$
  • 1
    $\begingroup$ But OP wants Doc2Vec to draw information from the numbers, that's kind of why they posted $\endgroup$ – I_Play_With_Data Feb 21 at 19:29
  • $\begingroup$ @I_Play_With_Data yes but without a sufficient amount of numbers in enough contexts then the Doc2Vec model simply won’t be able to derive enough useful patterns. The easiest way to account for numbers is to give them a unique identifier which could then be substituted in post-processing. $\endgroup$ – HFulcher Feb 21 at 21:18
  • $\begingroup$ Oh I didn't say it would be easy, I just said that's what OP wants :-). The unique identifier of <NUM> wouldn't necessarily be helpful because you still need to associate both the scalar values and the units. For example, you would have to know that 4.7K is the same as 4,700 or that "A4" can be a type of paper or a type of metal - the unique identifier wouldn't address any of that. $\endgroup$ – I_Play_With_Data Feb 21 at 22:32
  • 1
    $\begingroup$ @I_Play_With_Data yes it certainly won’t be easy haha! I agree the identifier isn’t the optimal solution but might go far enough. To differentiate all the different permutations that number forms can involve you would need a substantial training set and a sufficiently large vocabulary. Not impossible but could make the problem certainly harder. $\endgroup$ – HFulcher Feb 21 at 22:47
  • $\begingroup$ As @I_Play_With_Data rightly identified, i want to draw information from the numbers as in Doc2Vec should differentiate between 15kva and 140kva . Is there a way I can do so? $\endgroup$ – Manasvi Duggal Feb 25 at 3:50
2
$\begingroup$

Welcome to the site! I think that's a great question and there's probably no quick answer. My gut tells me that you would have to create your own custom instance of Doc2Vec with some training on the numbers and units. That's especially the case when you're talking about domain specific items like "kva" and "a4" - Doc2Vec isn't going to know what to do with those OOTB, but you can train it.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.