7
votes
Should I use regex or machine learning?
I would say depends on the requirements and how much effort you want to give the task.
Using regex is definitely easier but in the some time you will not be able to cover everything specially if the ...
- 408
5
votes
Accepted
How to summarize a long text using GPT-3
Is there already a popular open-source script to do that?
The Python library GPT Index (MIT license) can summarize a large document or collection of documents with GPT-3.
From the documentation:
<...
- 5,573
4
votes
Should I use regex or machine learning?
If you foresee to build something for a big public, definitely you cannot use regular expressions. There is no way you can write a regular expression that can span the variance that a class of ...
- 1,754
3
votes
Accepted
How to use df.groupby() to select and sum specific columns w/o pandas trimming total number of columns
I think the answer depends on what you want to do with column 6. Keep in mind that the values for column6 may be different for each groupby on columns 3,4 and 5, so you will need to decide which value ...
- 1,889
3
votes
Accepted
Should I use regex or machine learning?
You receive an email from a friend that says, "let's have lunch next Tuesday" and your email program detects it and asks if you want to save a new calendar entry for "lunch on Tuesday".
What you ...
- 18.7k
2
votes
Should I use regex or machine learning?
Not every Tuesday is lunch day even if you are talking about some lunch on some Tuesday. It should be a NLP system using NER to provide the best prediction.
- 367
2
votes
Should I use regex or machine learning?
If in case 1 there would be one template for organizing information on CVs then you can go for regex, but in order to have really helpful tool for real world CV you have to train ML solution. Just ...
- 285
2
votes
Should I use regex or machine learning?
In scale, unless you are expecting to receive only a particular format, it is machine learning. For the first task, you should first parse the text and then scan it, probably with a Named Entity ...
- 411
2
votes
Extractive text summarization, as a classification problem using deep networks
TL;DR:
Is something like this feasible? (I know that nothing can be said for sure in data science unless tried out, but is it worth the shot?)
Yes
What kind of features should I feed into the ...
- 444
2
votes
Accepted
finding themes from text documents
The best method to find themes in a collection of documents is topic modeling. Topic modeling finds the hidden (aka, latent) themes beyond just keyword counts.
There are many approaches to topic ...
- 20.2k
2
votes
Accepted
Measuring the success of text summarization
You will first need a human written summary as your correct output. You can then compare the original and generated ones using Rouge scores. They compare the similarity between two giving paras by ...
- 2,205
2
votes
Is there a dataset with news articles and their headlines?
The most widely used ones in text summarization research is the DUC dataset. If you see a paper using dataset "DUC 2015" or "DUC 2016" that's from here.
I have also personally used the Reuters ...
- 2,214
2
votes
Accepted
Reducing text input size into word2vec without affecting performance too badly?
I don't think there really is a right or wrong answer to the "removing stopwords" question. Some people will argue that throwing away information will reduce model performance, while others argue ...
- 2,350
1
vote
Topic modelling or Keyword extraction for a small dataset
Maybe a simpler solution like below would help? I am not a fan of topic modeling because i feel results wont be worth the effort.
To understand what the people are talking about here:
Use ngram ...
- 408
1
vote
Ideal Windows Size in Pk Evaluation Metric
I am also new to NLP, but according to some research papers:
Window size k should follow the next expresion:
This applies for WinDiff and Pk scores.
Reference:
https://books.google.es/books?id=...
- 11
1
vote
Should I use regex or machine learning?
Ive been able to use a set of regex rules that feed a scoring system to profile Pubmed abstracts. For example, any instance of 'increased risk', 'increased association', etc., adds to an 'association' ...
- 111
1
vote
Accepted
UniLM - Unified Language Model for summarization
Here's what you should do
Prepare your dataset: Follow similar instructions as described in the paper and preprocess your dataset. This will be your major task as after this you will only have to ...
- 352
1
vote
Accepted
Text summarization with limited number of words
You sure can,
for example in latent semantic analysis you can fixate number of topics (which is actually size of the decomposition matrix) beforehand.
- 5,609
1
vote
Detect sensitive data from unstructured text documents
Welcome to the site! Assuming that I understand your problem correctly, I think you can achieve a working model.
If I was in your position I would:
Obtain the cleanest data possible from the ...
- 2,079
1
vote
Extractive text summarization, as a classification problem using deep networks
I have tried markovify, a markov chain library in python for automatic text summarization for text documents. You can check that out for a simple unsupervised approach. For me it did not give ...
- 33
1
vote
Extract sentences from beginning of news in single document summarization
Keselman, Schubert
Computational models for text summarization
The paper deals with methods (models) for text summarization. The reference (base) model was "first sentence model":
As a baseline ...
- 636
Only top scored, non community-wiki answers of a minimum length are eligible