9

Doccano is an open source simpler alternative to Prodigy. Its native python via Django. I found it suitable for simple implementations.


2

You can try Prodigy by explosion.ai, creators of spacy or brat an open source alternative to it. You may also refer to this post on qoura.


2

I'm not sure if it meets all your criteria (mostly because I'm not sure I understand all your criteria!), but you could have a look at ELAN: Description: With ELAN a user can add an unlimited number of textual annotations to audio and/or video recordings. An annotation can be a sentence, word or gloss, a comment, translation or a description of any ...


1

There are many manual text annotation tools available, but you will probably have to search around in order to find the one which suits your precise needs. Here are a few pointers: Gate Text annotation tools Doccano a recent review


1

You ideally want a copy of The Handbook of Linguistic Annotation which covers the issues you’re up against in detail. The basic idea is: Create annotation guidelines as a training tool to increase interannotator agreement as far as possible Measure interannotator agreement among people using your guidelines to get an idea of the irreducible error Generate ...


1

Most machine learning algorithms are designed with complete trust in the labels. There is no standard way to model uncertainty in data labels. Thus, create a project-specific threshold for uncertainty to omit data or labelers. For example, a trusted classification data label would require n of m ensemble voting. One major issue is re-labeling. Systems tend ...


1

So, I'm still not sure about what you consider to be an annotation on curlie.org. What I understood, you'll correct me if I'm wrong, is that you would like to annotate some text, more specifically you would like to annotate/identify concepts, and you would like to follow the structure on curlie. From what I see, curlie doesn't contain any annotation, it's ...


1

It always depends on each specific problem. The amount of data depends on a number of factors including (but not limited to) the complexity of the problem, the number of features, the quality of the training data, the ratio of the training classes (i.e. class imbalance), etc. If you have no idea where to start, sometimes the best thing is just to experiment....


1

You can use sequence labeling feature to annotate the text:


1

Try prodigy. An annotation tool powered by active learning.


1

Shannon entropy is a common measure of uncertainty among a fixed set of choices, as are the ones provided by Brian Spiering. Regarding your question -- "some approach to compare named entities regarding how difficult to disambiguate?" -- note that the difficulty to disambiguate an entity is completely context and domain dependent. To give a truly useful ...


1

Entity Linking is a type of supervised machine learning, thus many of the common performance metrics could be used. In particular, creating a confusion matrix would identify where one label was predicted but the ground-truth was different. Confusion matrices can be calculated with counts or normalized, a normalized data would be an estimate of "ambiguity ...


1

I have been working with the spaCy extenstion on INCEpTION from Technische Universität Darmstadt. Seems pretty good so far.


1

Diffgram can be used for creating and managing training data.


Only top voted, non community-wiki answers of a minimum length are eligible