1
$\begingroup$

Hey fellow data lovers!

I have a data modelling problem for a clustering analysis that I can't wrap my head around. Perhaps I'm thinking to advanced and I should simplify my analysis.

I have 2 data sets:

  • Persons with their properties:
    • PersonId
    • Gender
    • Age
    • Country
    • Etc.
  • Conversations and their properties:
    • ConversationId
    • Sentences of the conversations
    • Keyword of a sentence + it's frequency
    • Polarity of the sentence (pos, neg or neutral)

I would like to cluster terms that have the same polarity for a group of people. Eg. male persons between 20 and 35 speak positively about economy and neg about privacy Eg. male persons from BE, NL and DE speak neg about climat changes.

Problem is that I need to cluster multiple terms for an undefined group of people. For one term (eg. privacy) it would be 'easy' to determine the properties of the different segments as this is a classification problem. As I would like to cluster multiple terms together, I'm strugling to model my data as this means I have multiple records per person (they can talk about multiple terms).

Assumption: you can assume I have one record per person per term.

Improve this question
$\endgroup$
2
  • $\begingroup$ The data model I'm referring to in my question, is how I should model my data to feed it to a clustering algorithm. Not how my data model of the 'raw data' should look like. $\endgroup$ – Tankske Jan 13 '16 at 14:28
  • $\begingroup$ Wouldn't how exactly you model your data depend on the implementation you are using for clustering? $\endgroup$ – Shagun Sodhani Jun 11 '16 at 14:17
1
$\begingroup$

clustering algo would take any data type as long as it is measurable, ideally shouldn't be string (which can be translated into vectors using topics). If you tell me what application you are using i can guide you on how to go about it step by step

Improve this answer
$\endgroup$
1
  • $\begingroup$ Welcome to Data Science SE. This remark would better fit in a comment. $\endgroup$ – Stereo Mar 8 '17 at 9:06
0
$\begingroup$

If you're just looking for a data model, this one should handle your stated requirements:

    Person
    -------------------
    PersonId
    Age
    Gender
    Country
    Etc
    --------------------

    Person_Conversation
    --------------------
    PersonId
    ConversationId
    --------------------

    Conversation
    --------------------
    ConversationId
    ....
    --------------------

    ---- if a sentence can be part of more than one conversation
    ---- use:

    Conversation_Sentence 
    --------------------
    ConversationId
    SentenceId
    --------------------

    Sentence
    --------------------
    SentenceId
    Keyword
    Polarity
    --------------------

    ----Otherwise use:

    Sentence
    --------------------
    SentenceId
    ConversationId
    Keyword
    Polarity
    --------------------
Improve this answer
$\endgroup$
2
  • $\begingroup$ I agree this is the original data model. I was wondering however how I should model my data to feed it to a clustering algorithm.. I was thinking of one table with the following elements: Term, Polarity, Frequency, Person, Age, Gender, Country. But I'm afraid that feeding this to an clustering algorithm wouldn't work as terms occur multiple times (as do persons) $\endgroup$ – Tankske Jan 13 '16 at 10:04
  • $\begingroup$ @Tankske- I am having similar problem. Could you please let me know your approach? $\endgroup$ – Neil Aug 10 '16 at 2:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.