Skip to main content
6 votes
Accepted

regex to remove repeating words in a sentence

Since you were working with RegEx, I will ofer a RegEx solution. I will also show that you need to also take care to first remove punctuation. (I will not go down the rabbit-hole of re-sinserting the ...
n1k31t4's user avatar
  • 14.9k
4 votes

How to improve OCR (Scanning) results?"

Build a dictionary of common words that frequently appear in these documents (e.g., MEDICAL, SEX, AGE, etc). Then, for each word in the output from OCR, check whether it is similar to a word in your ...
D.W.'s user avatar
  • 3,381
3 votes
Accepted

How to customize word division in CountVectorizer?

It's possible if you define CountVectorizer's token_pattern argument. If you're new to regular expressions, Python's documentation goes over how it deals with ...
Tom M.'s user avatar
  • 671
2 votes
Accepted

Facing a difficult regular expression issue in cleaning text data

To remove multiple white space matches, you will need [\s\n]+, note the inclusion of the + (match one or more). Code: Here is ...
Stephen Rauch's user avatar
  • 1,783
2 votes

R: remove single characters from string

This splits the string and filters the substrings by length, no regex: ...
rcs's user avatar
  • 720
2 votes

How to filter out a portion of the text data with re.match or any other regex method with Python?

nice that you solved it. This would also work: ...
El Burro's user avatar
  • 800
2 votes
Accepted

Machine learning or NLP approach to convert string about month ,year into dates

I got my answer , NLTK is good to go for this problem. You may use sutime with python wrapper : Python wrapper for Stanford CoreNLP's SUTime The usual approach in NLP is to collect a dataset ...
Bipul's user avatar
  • 201
2 votes

Is the search for a specific n-gram the same like a string search?

Well, as you state the problem, it is true that the search for a certain sequence of strings/words is the same as looking for the corresponding n-gram. However, keep in mind that an n-gram, when you ...
Peter's user avatar
  • 7,526
1 vote

regex to remove repeating words in a sentence

From what I understood, doing this with regex rules can be tricky and a bit slow. In python, I use this and it works perfectly : ...
Adrien D's user avatar
  • 1,113
1 vote

Classify E-commerce URLs into predefined classes

There is not enough information for a comprehensive answer, but I will try to assist you. You can try to hard code it without using any machine learning algorithms. Try looking at specific parts, ...
Nemo_the_scientist's user avatar
1 vote

Regex in R as a list for Quanteda

This doesn't seem like a great task for regex--even your pattern would miss very close typos like COWID-19 or potential OCR mistakes like ...
Gregor Thomas's user avatar
1 vote
Accepted

regex for JSON only for one nested data

You need to apply two regexes: first, get r'^test:.*$' with m option, then your original regex on the result of the first.
Alexander Mashin's user avatar
1 vote

pandas series match multiple keywords

here's what I've tried: ...
user3016638's user avatar
1 vote
Accepted

Text comparison: spot the differences

You could look at string similarity measures and TFIDF (usually with cosine). If you want a measure which works at both levels of words and sentences, there are more advanced options such as SoftTFIDF....
Erwan's user avatar
  • 25.5k
1 vote
Accepted

How to validate regex based Resume parser efficiently

Is there any better way to validate this piece of python code. ? No, because any automatic method would be equivalent to creating another learner, and there would be no way to know if it's good or ...
Erwan's user avatar
  • 25.5k
1 vote

How to generate abbrevations from shortend words in medical records

First a warning: is your data anonymized? Even if it is, be extremely careful because medical history is super-sensitive personal information. There are legal requirements about how to handle this ...
Erwan's user avatar
  • 25.5k
1 vote

How to improve regex while fetching record dynamically

...
Robin Nicole's user avatar
1 vote
Accepted

Keeping part of a string in R

Use strsplit(). If you want to remove the space between the two words (along with 'NSW'): ...
Vishal's user avatar
  • 268
1 vote
Accepted

Replace method in pandas not giving expected result

Try this if you decide to use pandas: ...
Johan's user avatar
  • 26
1 vote

Replace method in pandas not giving expected result

This is pure(almost) python: list(map(lambda x: x[0] + '.'.join(['X' * len(c) for c in x[1:].split('.')]), my_df['IP'])) Explanation: - Use the map to iterate ...
yoav_aaa's user avatar
  • 993
1 vote

Tagging Unix/Non-Unix logs using NLP

PoS tagging works for natural language only and identifies grammatical parts of the sentence, nothing more. LSTM is an algorithm that can be used to predict series. Named Entity Recognition (NER) and ...
wacax's user avatar
  • 3,410
1 vote

Regex-style pattern-matching for time series

If you need optional parts in the pattern then regular expressions were specifically built for those kind of problems. You could probably speed up regular expressions by using loops in Cython, but the ...
keiv.fly's user avatar
  • 1,269
1 vote

Using regex in redshift to find dollar values

I am getting that you want extract numbers with either '$' or even not without a dollar. Why are not using just [0-9]+ for that field? The results will be like ...
Kiritee Gak's user avatar
  • 1,799
1 vote

Using training data generated with pure regular expressions - Can machine learning surpass the accuracy of your regular expression?

No- not if your regex is actually generating the data. If this is the case, the regex (or a complementary one) should be able to classify the resulting text perfectly. That is, you know what the rules ...
Thomas Cleberg's user avatar
1 vote
Accepted

Regular expression in python -

It may not be perfect but does the job almost. ...
Jil Jung Juk's user avatar
1 vote

Regular expression in python -

As @spacedman correctly mentioned, this will answered quicker at StackOverflow. But you can use this to create a dictionary like this. There might be a better way but this is a quick work around. <...
Kiritee Gak's user avatar
  • 1,799
1 vote

What is the best approach for specified optical character recognition?

Optical character recognition is a well-studied problem with many possible solutions (ressources). CNNs have proven to work extremely well even for hand-written character recognition. Take a look at ...
tony's user avatar
  • 591

Only top scored, non community-wiki answers of a minimum length are eligible