# Add extra term weight when grouping strings by similarity?

I'd like to convert a set ducoments to term weight matrix(features) by tf-idf. Then calculate the similarity of two document by their features.

similarity is compute by result = matrix*matrix.T > 0.9 detail here, and group them by the result(loop result, if result[1,0] >0.9, then index 1 is similar with index 2)

Now I have a lot of resource to group.

For example, I have some books with different names(more complex in fact) I can roughly group these books name by similarity, like below:

step1:

group 1:
1.The Three Body Problem vol 1
2.[Chinese]The Three Body Problem no 1
3.The Three Body Problem 2
4.The Three Body Problem vol 3[Japanese]
5.Problem of Three Body vol 3
6.(xx)The Three Body Problem 2
7.The Three Body Problem 1[English]
group 2:
1.Another book 1
....


But xxx 2 and xxx vol3 is needless when I want to find xxx vol 1 , So have to do

step2: tokenize each book name again, use some patterns/rules to extract the book number to distinguish them.

Is there any way to add some term(such as Arabic numerals:0-1, English number:one- twenty) with high dissimilarity weight, to make step1 result

group 1:
1.The Three Body Problem vol 1
2.[Chinese]The Three Body Problem no 1
7.The Three Body Problem 1[English]
group 2:
3.The Three Body Problem 2
6.(xx)The Three Body Problem 2
group 3:
4.The Three Body Problem vol 3[Japanese]
5.Problem of Three Body vol 3
group 4:
another book 1
[xx]another book vol.1


update

If there are some titles like below:

1. There are 2 man vol.1
2. (xx)There man 2 boy 2


I need add a lot detection(number position or something else), that's why I want a way to add a extra weight to somewhere(to avoid step 2, the redundant tokennizing and custom extraction rule).

I think the similarity weight may work like:

tfidf weight two title, plus each number weight, then calculate the similarity matrix.
But now I am using tfidf matrix power to get similarity matrix,
I don't know how to add a extra weight to tfidf weight result, the weight meaning is different between tfidf weight and what mentioned extra weight.

I want to know where and how to add the proper extra weight, how calculate it value?

• I have already group similar book together.But what I need is add some function to make it more correctly. xxxxx 1 and xxxxx vol.one can be accept as similar , but xxxxx 1 with xxxxx 2 or xxxxx vol.II not.That's the problem. Mar 13, 2016 at 2:49
• That is what I try to avoid.You must extract the number from book title(as I do now), rather than add a weight to number at step 1. Extract number have a lot work to do , such as when title like there are 2 man vol.1.I wouldn't fair such problem when using a similarity weight. Mar 17, 2016 at 7:47