I want to analyse the Price situation for flat/appartements in my City.

How can I process the Text from flat ads (like in newspapers or online with its typical structure!) to get Features like size or number of rooms?

Is it simple to take a Package in R or Python (what is suitable?) and to Train the typical structure?

  • $\begingroup$ Yes. That should be the assumption. Becsuse Text scraping is Not topic. $\endgroup$
    – Tido
    Nov 17 '17 at 8:55
  • $\begingroup$ Ok, i get it... hmm it is just theoretical. We can discuss First that they exist in in a text file in a Format whrere we can work with like Every row consist of the whole Text.... $\endgroup$
    – Tido
    Nov 17 '17 at 9:06

Look for entity recognition. The entities in your case would be price, number of bedrooms, size etcetera. What I would do is label a number of training examples with:

  • From character 12 to character 22 indicates the number of bedrooms
  • From character 39 to character 41 indicates the price


After this you can train a model to classify entities. Sometimes the number of bedrooms will be in text instead of numbers so then you would need to write something that translates it to numerical values but you could likely just use some dictionary to do this for you.

Entity recognition models are plenty, but I think a character based LSTM approach could work well if your texts are not too long.

  • $\begingroup$ Thanks! But ad Text has no fix structure...or what You mean with the Characters? What i want is also a Package in R or tensorflow for those tasks. $\endgroup$
    – Tido
    Nov 18 '17 at 8:07

You can use Apache OpenNLP for your purpose. First, you need to train your model with training data. In your case the training data may look like:

<start:size> Big <end> Apartment at <start:address>Pavelle Road <end> on sale at just <start:price> $400k <end>.

If you can manage to have a large collection of annotated data like this, you can use the model to get data thereon. For more information look in Apache OpenNLP, its Open Source.


Contrary to what others are suggesting, trying to extract data on square footage and number of rooms from apartment ads is not a problem for machine learning - especially not when you don't have any training data. You will spend 10 times as long trying to build a machine learning system for this task than you would manually extracting the data yourself.

If you really want an automated system, consider building some simple regex rules based on ads you have seen. There are only so many ways people describe these features, and you should be able to get most of them when they are present in the ad. You won't get 100% accuracy, but you're not going to do any better another way, in my opinion.

You can start with some simple rules, and improve them as you look at more and more ads. In Python, for example:

import re

ad = '$3582 / 1br 1ba - 724ft2 - Breathtaking Seaport District'

sqft_pattern = '([0-9]+)\s?(sq|ft)'
beds_pattern = '(([1-9](\.5)?)\s?(br|bd|beds?|bedrooms?))|(beds|bedrooms):?\s?[1-9]'
bath_pattern = '(([1-9](\.5)?)\s?(ba|baths?|bathrooms?))|(baths|bathrooms):?\s?[1-9]'

m = re.search(sqft_pattern, ad)
sqft = m.group(1) if m else None
m = re.search(beds_pattern, ad)
beds = m.group(2) if m else None
m = re.search(bath_pattern, ad)
bath = m.group(2) if m else None

print('square  feet: {}'.format(sqft))
print('bedrooms: {}'.format(beds))
print('bathrooms: {}'.format(bath))


square feet: 724
bedrooms: 1
bathrooms: 1

Not the answer you're looking for? Browse other questions tagged or ask your own question.