5
$\begingroup$

I am looking for a method to parse semi-structured textual data, i.e. data poorly formatted but usually having a visual structure of a matrix which may vary a lot in content and number of items in it, which may have headers or not, which may be interpreted sometimes column-wise or row-wise, and so on.

I have read about the WHISK information extraction paper : https://homes.cs.washington.edu/~soderlan/soderland_ml99.pdf

but unfortunately, it is not very detailed and I have not been able to find a real-system implementing it, or even snippets of code.

Has anybody have an idea where I can find such help? Or suggest an alternative approach which may be suited to my problem?

Thank you in advance for your reply!

Improve this question
$\endgroup$
2
  • $\begingroup$ Without a sample it's hard to see what tool could help you. UTAH is a java library that's able to parse semi-structured text. It might be useful for you. $\endgroup$
    – user17540
    Apr 2, 2016 at 15:48
  • 2
    $\begingroup$ Sorry, I forgot to link to UTAH ;) github.com/sonalake/utah-parser $\endgroup$
    – user17541
    Apr 2, 2016 at 15:51

1 Answer 1

Reset to default
5
$\begingroup$

Without a sample of your data, it's unclear what's the structure of your data and what tool is suitable to process it.

Here are some blind recommendations based on my experience:

  • If you just need some flexibilty parsing the text record, such as variable repeat number of certain field, or conditional parsing of fields, then you should check out this python library: http://construct.readthedocs.org/en/latest/ it allows you to first define a hirachcal structure of your data, and then apply this structure to parse information from a text file. It's especial useful when parsing binary files.
  • If you're looking for an algorithm that can actually "understand" your text data and "infer" the structure in a smart way. Then you might want to try graph based approach: http://kavita-ganesan.com/opinosis
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.