2
If you want to be up to date with the new advancements, a good way is skimming through the accepted papers of the major NLP conferences, namely ACL, EMNLP, and the regional EACL, NAACL, AACL.
If you want even more information, you can skim through the papers uploaded to the arxiv. One way to do that is via Twitter, by following bots that tweet papers in ...
2
For any kind of Machine Learning task or a NLP task (which is what you are doing), you need to convert string/text values to numeric values. The machine cannot uderstand or work with string values. It only understands numeric values.
So for example if you are doing a machine learning task, you would use libraries like OneHotEncoder, LabelEncoder etc to ...
1
You can either use a sentence embedding model to associate a vector to each of your inputs, and use a clustering algorithm like KMeans, or build a similarity matrix between your strings using a string distance metric, and use a similarity-based algorithm like Spectral Clustering or Agglomerative Clustering.
The first one using KMeans might not work the best ...
1
Once I assume you are using text data as your input matrix X. The first point is that you have to include your preprocessing step as you would do when not using a calibrated classifier, so as you already know you can use a Pipeline like so:
calibrated_svc = CalibratedClassifierCV(linear_svc,
method='sigmoid',
...
1
I think simple regex matching is all you need.
Pass the tweet into a series of regular expressions that match emojis and hashtags and if nothing remains, discard.
1
https://nlpprogress.com/ aims to provide pointers to the state of the art papers and datasets for the main NLP tasks.
It seems to be updated regularly so far. However it depends on the efforts of volunteers so there's no guarantee about completeness or future updates.
Only top voted, non community-wiki answers of a minimum length are eligible