0
$\begingroup$

I am trying to use the labeled Enron dataset (link) but I am really confused about the labeling system they use. I understand the Cat_[1-12]_level_weight is some form of confidence level. This dataset labeled by multiple students. Cat_[1-12]_level_weight increases with the number of the same label assigned by multiple students to a certain row (sample). But, what is for instance the Cat_1_level_2? From the overview (in the link), I guess Cat_1 mean "Course genre" and level_2 mean "Purely Personal (49 cnt.)" ? If so, why the Cat_1_level_2 has values like 1, 2, 3, 4, etc.? I am really confused.

$\endgroup$
1
$\begingroup$

Looking at the first link from your post there is an in-depth overview of the different categories. Cat_#_level_1 denotes the top level category (12 in total) whereas Cat_#_level_2 denotes the second level category (up to 19 categories, depending on the level 1 category). E.g. an email with the labels Cat_3_level_1 and Cat_6_level_2 has the label california energy crisis / california politics (category 3.6). You therefore need to combine the two levels to get to the label that belongs to the second level category.

$\endgroup$
4
  • $\begingroup$ That makes sense! Thank you $\endgroup$ – MehmedB Mar 30 at 11:46
  • $\begingroup$ If my answer solved your question please mark my answer as the solution for other who may have the same question. $\endgroup$ – Oxbowerce Mar 30 at 12:10
  • $\begingroup$ Yes, that makes sense but not that much. So, there is an email with label 2 in the column of Cat_3_level_1. What does that 2 mean? $\endgroup$ – MehmedB Mar 31 at 7:58
  • $\begingroup$ From what I understand, first, I need to look at the Cat_#_level_1. From my example, it is 2. And from there, I need to look at the Cat_2_level_2 because Cat_1_level_1 is 2. And in that email, the Cat_2_level_2 is 3. Does that mean this email belongs to category 2.3??... Omg I have no idea $\endgroup$ – MehmedB Mar 31 at 8:09
0
$\begingroup$

Ok, I guess here how it should be read. A sample could be in multiple categories. First, you look into a sample. Below is an example (at row 12) and the mail content:

Content Cat_1_level_1 Cat_1_level_2 Cat_1_weight Cat_2_level_1 Cat_2_level_2 Cat_2_weight
Jennifer, Thank-you for stepping in on this and guiding the process! ---------------------- Forwarded by Sarah-Joy Hunter/NA/Enron on 12/12/2000 05:03 PM --------------------------- From: Patrick Tucker@ENRON COMMUNICATIONS on 12/12/2000 02:52 PM PST To: Sarah-Joy Hunter/NA/Enron@ENRON cc: Subject: Re: HP -- confidential internal document Sarah-Joy, thanks for your excellent recap of progress to date. I really appreciate the organization and order you have brought to this process. It's great to work with you again after all of this time! Patrick 1 2 2 4 6 2

Here, you need to combine the Cat_1_level_* to get a single category that this mail belongs to. You can do the same combination for other categories as well (like Cat_2_level_*). If you combine the first category as Cat_1_level_1 (which is 1) and Cat_2_level_2 (which is 2), you get an actual category label which is Category 1.2 (Purely personal). As I said before, a sample could be in multiple categories. If you combine the second category labels, the Cat_2_level_1 (which is 4) and Cat_2_level_2 (which is 6), you get another category label which is Category 4.6 (gratitude). If you check the content, you can see they are talking about something personal and they try to express how grateful they are to each other.

About the weights (Cat_1_weight, Cat_2_weight, Cat_*_weight...). Since this dataset samples were labeled by multiple people, in some cases, there were different labels assigned to the same sample (or in some cases a sample assigned with the same label by multiple people). The weights show that how many people assigned the same label in a category for a certain sample. Here, the Cat_1_weight is 2, which means 2 people assigned this Cat_1_level_* labels to this sample. (I guess...)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.