How can I convert text data to CoNLL format?

Question

This is the same question that I posted on stackoverflow, but I wondered stackexchange would be appropriate for this question.

I would like to convert text data to CoNLL format.

words.txt

I was born in 1981.
From 12 to 24.

tags.txt

O O O O B-DateTime O
O B-DateTime I-DateTime B-DateTime O

CoNLL（.conll file)

I    O
was  O
born O
in   O
1981 B-DateTime
.    O

However, I only found a library for CoNLL-U format(conllu) and a library looked like for CoNLL（pyconll） but no sample code, so I have no idea to apply for it to text-CoNLL conversion.

I'm stacked with how to convert the data to CoNLL and how to write Python3 script to do it.

Answer 1

I think there's a bit of confusion here: the sample you're showing is not a full "conll" format, at least not any recent one. It's simply a BIO format for NER.

As far as I know conllu has been the standard "conll" format for probably at least 10 years, so if you're using some old data it's possible that it used the name "conll" for something different. Or maybe somebody just used the name "conll" because the data was related to conll even though the format is not a standard conll format. Btw the library you link is for parsing the conllu format, not for generating it.

Normally The conllu format includes several columns for every token: at least token, lemma, POS and usually some dependency tree information (index of the head of the dependency). It is used in particular by the Universal Dependencies project.

So as far as I know there's no particular conll standard in this format. But it's a pretty simple conversion that can be implemented manually: you can simply iterate over the words and the tags in parallel and print the token and BIO tag as columns. The only posssible issue is whether the words are already tokenized or not.