Translation as a pre-processing step is usually sufficient for many tasks (e.g. sentiment classification), but naturally undesirable for other tasks e.g. grading someone in written Dutch fluency.
Hence, for these tasks, the objective is:
Be able to train a language model for your specific language
However, you want to be able to do this with minimal ...
Since the main goal is to extract headers(which has different characteristics from other text). It would be best to convert pdf to xml as you will get features like style,hpos,vpos for each text in the file.
To work with xml you can refer ElemenTree.
To complements @Phil's answers and the excellent SPMF Library:
Seq2Pat: Sequence-to-Pattern Generation Library might be relevant to your case.
The library is written in Cython to take advantage of a fast C++ backend with a high-level Python interface. It supports constraint-based frequent sequential pattern mining.
Here is an example that shows how to mine a ...
IMO. Firstly, you make the hypothesis that cliche sentences are in the same part of the document, which makes sense. But how do you determine the weight of the position and what if the students use different structures (e.g., some write the conclusion and the discussion sections separate, others write them together)?
Then you could get more up to date ...
It's not a great dataset for NER because normally NER relies on trigger words close or inside the NE. For example "he went to X" would indicate that X is a location. Here there's no usable context and even the order of the lines can change.
What I would try here is to classify the lines, because apparently there's a kind of structure to each sample:...
I think we would need more information about the model architecture you are using and the features you engineered for training.
Something I like to do when starting to work on text classification problems is to gradually increase the complexity of the architecture and start from simple approaches.
So if you are using tf-idf features and some sort of ...