0
$\begingroup$

I'm experimenting with a concept I saw in this research paper. That is, first I train a CRF-model for named entity tagging, then I do implement an identical model, except for that one also takes the prediction of the previous one as a feature. Everything else is the same.

During training, the cross-validation results show drastic improvements for the second model, and when the weights are visualized, the added feature ranks by far the most significant one.

However, when the two models are tested and compared on my test set, using the CRF.predict() function, the second model performs exactly the same as the first one. What could be wrong?

I'm basing my code on this tutorial and here's the most relevant part of the code I think:

def sent2features(sent):
    return [word2features(sent, i) for i in range(len(sent))]

def sent2features_second_guess(sent, old_sent2features, oldpredictor):
    features = old_sent2features(sent)
    prediction = oldpredictor.predict_single(features)
    return [word2features_second_guess(sent, features[i], prediction, i) for i in range(len(sent))]


def word2features(sent, i):
    word = sent[i][0]
    features = {
        'bias': 1.0,
        'word.lower()': word.lower(),
        'word[-2:]': word[-2:],
        'word[-3:]': word[-3:],
        'word.isupper()': word.isupper(),
        'word.istitle()': word.istitle(),
        'word.isdigit()': word.isdigit(),
    }
    if i > 0:
        word1 = sent[i-1][0]
        features.update({
            '-1:word.lower()': word1.lower(),
            '-1:word.istitle()': word1.istitle(),
            '-1:word.isupper()': word1.isupper(),
        })
    else:
        features['BOS'] = True

    if i < len(sent)-1:
        word1 = sent[i+1][0]
        postag1 = sent[i+1][1]
        features.update({
            '+1:word.lower()': word1.lower(),
            '+1:word.istitle()': word1.istitle(),
            '+1:word.isupper()': word1.isupper(),

        })
    else:
        features['EOS'] = True

    return features

def word2features_second_guess(sent, word_features, predicted_labels, i):
    word_features.update({'model1label': predicted_labels[i]})

    return word_features



What am I doing wrong? Let me know if you need more of the code to be able to help.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.