1
$\begingroup$

I have data that formatted like this below:

{"input": "250 Hartford Avenue, Bellingham, MA, 2019", 
    "output": "{
    'address': 'Hartford Avenue',
    'city': 'Bellingham',
    'state': 'MA',
    'zip': '2019'   }" },   

{"input": "700 Oak Street, Brockton, MA, 2301",  
    "output": "{
    'address': 'Oak Street',
    'city': 'Brockton',
    'state': 'MA',
    'zip': '2301'   }" },

This can be handled by regex, but I'm requested to make it can predict if the input just- and given output like this:

"input": "Hartford Avenue"

the output should be:

  "output": "{
    'address': 'Hartford Avenue',
    'city': 'Bellingham',
    'state': 'MA'}

or

"input": "Bellingham"

the output should be:

  "output": "{
    'city': 'Bellingham',
    'state': 'MA'}

and the others clueless input, output should gives completed address.

anyone can give me a method in deep learning side for handle this data? is this appropriate to using classify approach? or tagging? or text generation?

$\endgroup$

3 Answers 3

3
$\begingroup$

Fuzzy address matching and/or record linkage could be a more efficient solution than deep (DL) learning. The training data for DL would be a collection of addresses. Probabilistically searching through that collection of addresses is probably easier (and faster) than training a DL system.

There are a variety of algorithms and paid services that provide this kind of functionality. One commonly used package is Python Record Linkage Toolkit.

$\endgroup$
1
  • $\begingroup$ If using fuzzy matching, does it mean it must use looping for searching the highest score (represent the most similar)? I think about avoiding looping way due to performance (speed, etc.). Or is it the best way? $\endgroup$
    – Mico S
    Jul 28 at 2:04
0
$\begingroup$

https://www.kaggle.com/discussions/general/30364#169086

This kaggle user tried to something similar. I am trying to get the code myself to check out his approach.

Apart from that, it would be interesting to know the scope of the problem. Are we just sticking to all addresses with one city (Bellingham, MA) or the scope is for all global street address.

Narrowing down the scope of the problem, would help tremendously in finding a suitable approach, be it simple (using excel) or Deep learning methods.

$\endgroup$
1
  • $\begingroup$ I can't found any code project on there. $\endgroup$
    – Mico S
    Jul 28 at 4:00
0
$\begingroup$

At first, I would have recommended a next-word generation with a model like GPT-2, but it seems more like a mask-filling topic with models like BERT.

BERT is actually great for finding the most probable fields according to input with specific positions.

Hence, after training a BERT model from scratch with plenty of addresses like this, you can ask to fill masks:

import torch
from transformers import BertTokenizer, BertModel,BertForMaskedLM
 
tokenizer = BertTokenizer.from_pretrained('my_model')
#Field 1: address, Field 2: city, field 3: state
input_txt = "Hartford Avenue [MASK] [MASK] "
inputs = tokenizer(input_txt, return_tensors='pt')

Note that BERT requires a continuous text field to get the correlations between words and you can extract those words from the fields, and after the Bert predictions, collect the predicted words to copy them to the correct fields, if possible as proposals.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.