Which insights a data scientist could derive from text-analysis? [closed]

Question

I have many texts and I am trying to analyse them. After tokenising them, studying words frequency, spotting any typos, studying punctuations, I have been working on POS tagging. Since it is my first time in text mining and manipulation, I would like to know which kind of insights I could get from this information and what the best approach to present this analysis would be.

For example: if I had many texts like the following:

Hemingway =["When spring came, even the false spring, there were no problems except where to be happiest. The only thing that could spoil a day was people and if you could keep from making engagements, each day had no limits. People were always the limiters of happiness except for the very few that were as good as spring itself.","Most people were heartless about turtles because a turtle’s heart will beat for hours after it has been cut up and butchered. But the old man thought, I have such a heart too.","Perhaps as you went along you did learn something. I did not care what it was all about. All I wanted to know was how to live in it. Maybe if you found out how to live in it you learned from that what it was all about.","The people that I liked and had not met went to the big cafes because they were lost in them and no one noticed them and they could be alone in them and be together."]

Shakespeare=["These violent delights have violent ends And in their triump die, like fire and powder Which, as they kiss, consume","Let me not to the marriage of true minds Admit impediments. Love is not love Which alters when it alteration finds, Or bends with the remover to remove. O no, it is an ever-fixed mark That looks on tempests and is never shaken; It is the star to every wand'ring barque, Whose worth's unknown, although his height be taken. Love's not Time's fool, though rosy lips and cheeks Within his bending sickle's compass come; Love alters not with his brief hours and weeks, But bears it out even to the edge of doom. If this be error and upon me proved, I never writ, nor no man ever loved.","O serpent heart hid with a flowering face!Did ever a dragon keep so fair a cave? Beautiful tyrant, feind angelical, dove feather raven, wolvish-ravening lamb! Despised substance of devinest show, just opposite to what thou justly seemest - A dammed saint,honourable villain!","Lord Polonius: What do you read, my lord? Hamlet: Words, words, words. Lord Polonius: What is the matter, my lord? Hamlet: Between who?  Lord Polonius: I mean, the matter that you read, my lord."]

Would it be useful a word frequency distribution, POS tags distribution? I have never presented any results, so I would like to know more on how a data scientist/analyst may 'tell' and look at this data.

Answer 1

Well, obviously the use cases depends on the industry. Also, I am assuming you are thinking of use cases that are somehow useful. But let's think of some examples:

• I once worked with a book distributor that tagged each book they sold with keywords (Fantasy, Horror, etc). You can automate the tagging process if you have a sufficiently large dataset of already labeled books. You can do the same with your phrases (Inspiring, Funny, etc) but probably you don't have labeled data. It would be nice to be able to ask an app for a phrase of certain type :).

• Something easier is sentiment analysis: are most of the phrases positive? Negative? You don't need labels for this.

• Style transfer: maybe you have several phrases by, let's say, Shakespeare. You can try transferring his style to Einstein's phrases. This is hard but feasible: look up Generative Adversarial Networks.