1
$\begingroup$

This is the same question that I posted on stackoverflow, but I wondered stackexchange would be appropriate for this question.

I would like to convert text data to CoNLL format.

words.txt

I was born in 1981.
From 12 to 24.

tags.txt

O O O O B-DateTime O
O B-DateTime I-DateTime B-DateTime O

CoNLL(.conll file)

I    O
was  O
born O
in   O
1981 B-DateTime
.    O

However, I only found a library for CoNLL-U format(conllu) and a library looked like for CoNLL(pyconll) but no sample code, so I have no idea to apply for it to text-CoNLL conversion.

I'm stacked with how to convert the data to CoNLL and how to write Python3 script to do it.

$\endgroup$
1
  • $\begingroup$ Have you looked into using spaCy to parse the string into the CoNLL format? $\endgroup$
    – Oxbowerce
    Aug 2 '21 at 10:20
1
$\begingroup$

I think there's a bit of confusion here: the sample you're showing is not a full "conll" format, at least not any recent one. It's simply a BIO format for NER.

As far as I know conllu has been the standard "conll" format for probably at least 10 years, so if you're using some old data it's possible that it used the name "conll" for something different. Or maybe somebody just used the name "conll" because the data was related to conll even though the format is not a standard conll format. Btw the library you link is for parsing the conllu format, not for generating it.

Normally The conllu format includes several columns for every token: at least token, lemma, POS and usually some dependency tree information (index of the head of the dependency). It is used in particular by the Universal Dependencies project.

So as far as I know there's no particular conll standard in this format. But it's a pretty simple conversion that can be implemented manually: you can simply iterate over the words and the tags in parallel and print the token and BIO tag as columns. The only posssible issue is whether the words are already tokenized or not.

$\endgroup$
2
  • $\begingroup$ Thank you for your answer. So in this case, do I need to write words for the first line and BIO tags for the second line for a text file, and then covert the file extension from text format to .conll format? $\endgroup$
    – Mahler
    Aug 2 '21 at 10:40
  • 1
    $\begingroup$ @Mahler yes, but mind that it would be words for the first column and BIO tag for the second column, not line. Sure you can call the file .conll if you want, but it doesn't matter, it's not the real conll format. You didn't say which task you're doing but it looks like you're going to train a NER model, it's not especially related to conll. $\endgroup$
    – Erwan
    Aug 2 '21 at 12:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.