# Datamodel for cluster analysis terms & segmentation

Hey fellow data lovers!

I have a data modelling problem for a clustering analysis that I can't wrap my head around. Perhaps I'm thinking to advanced and I should simplify my analysis.

I have 2 data sets:

• Persons with their properties:
• PersonId
• Gender
• Age
• Country
• Etc.
• Conversations and their properties:
• ConversationId
• Sentences of the conversations
• Keyword of a sentence + it's frequency
• Polarity of the sentence (pos, neg or neutral)

I would like to cluster terms that have the same polarity for a group of people. Eg. male persons between 20 and 35 speak positively about economy and neg about privacy Eg. male persons from BE, NL and DE speak neg about climat changes.

Problem is that I need to cluster multiple terms for an undefined group of people. For one term (eg. privacy) it would be 'easy' to determine the properties of the different segments as this is a classification problem. As I would like to cluster multiple terms together, I'm strugling to model my data as this means I have multiple records per person (they can talk about multiple terms).

Assumption: you can assume I have one record per person per term.

• The data model I'm referring to in my question, is how I should model my data to feed it to a clustering algorithm. Not how my data model of the 'raw data' should look like. – Tankske Jan 13 '16 at 14:28
• Wouldn't how exactly you model your data depend on the implementation you are using for clustering? – Shagun Sodhani Jun 11 '16 at 14:17

clustering algo would take any data type as long as it is measurable, ideally shouldn't be string (which can be translated into vectors using topics). If you tell me what application you are using i can guide you on how to go about it step by step

• Welcome to Data Science SE. This remark would better fit in a comment. – Stereo Mar 8 '17 at 9:06

If you're just looking for a data model, this one should handle your stated requirements:

    Person
-------------------
PersonId
Age
Gender
Country
Etc
--------------------

Person_Conversation
--------------------
PersonId
ConversationId
--------------------

Conversation
--------------------
ConversationId
....
--------------------

---- if a sentence can be part of more than one conversation
---- use:

Conversation_Sentence
--------------------
ConversationId
SentenceId
--------------------

Sentence
--------------------
SentenceId
Keyword
Polarity
--------------------

----Otherwise use:

Sentence
--------------------
SentenceId
ConversationId
Keyword
Polarity
--------------------

• I agree this is the original data model. I was wondering however how I should model my data to feed it to a clustering algorithm.. I was thinking of one table with the following elements: Term, Polarity, Frequency, Person, Age, Gender, Country. But I'm afraid that feeding this to an clustering algorithm wouldn't work as terms occur multiple times (as do persons) – Tankske Jan 13 '16 at 10:04
• @Tankske- I am having similar problem. Could you please let me know your approach? – Neil Aug 10 '16 at 2:16