1
$\begingroup$

I'm doing a little project of bugs prediction. My goal is to predict which bug will be (eventually) assigned to which relevant group (this is my label obviously).

For training, I'm relaying on a bugs database where I'm extracting various features (as much as possible) from each bug.

  1. traces
  2. panics
  3. git blame (if available)

While most of the above features will always be available for me during prediction, there's another feature I thought I can use, which is "comments" between group members. This feature will probably won't be available for me during prediction (since I'm planning to predict the bug on its early stage).

Now I'm a bit confused. Is it ok for me to relay on it during training? Am I cheating myself? Needless to say, the score is much higher when using it (without it I'm around 80% vs 90% or more while using it).

$\endgroup$
2
  • $\begingroup$ probably or possibly?! $\endgroup$ Jun 22, 2021 at 8:13
  • $\begingroup$ @KasraManshaei probably $\endgroup$
    – Ben
    Jun 22, 2021 at 8:38

1 Answer 1

1
$\begingroup$

3 points:

  1. If the feature is certainly or very most of the time is not available during prediction the no you cant use it
  2. If it is sometimes available and sometimes not, you must include no-comment bugs into your training as well and choose a default value which means no-comment (e.g. 'no-comment' string! or None)
  3. In case they are available only for training, you can still benefit from them during EDA. Extracting keywords, topics, etc. from them will help you understand the situation of different labels and possibly helps you validate your labels, score them and/or understand the relation between other features (by clustering-like analysis)
  4. If you want to use them, be careful how you do it. You have a bunch of categorical and/or numerical features and if you want to put text feature next to them you need to thing about feature representation. For example if you want to use TF-IDF, you suddenly introduce potentially thousands of features which may kill the information of your main features. So try to make it as sparse as possible like extracting keywords or topic from those texts and use those as categories. If you use any embedding to model them, check if you need to normalize your feature set as the scale of embedding values and other numerical features might be different

Hope it helped. Good Luck!

$\endgroup$
2
  • $\begingroup$ All my features are text. Again - the probability of whether or not this feature will be available or not is directly affected by when the prediction is being made on the bug life cycle. I can easily apply your suggestion in the 2nd point, but I'm not sure I got the bottom line I was looking for. $\endgroup$
    – Ben
    Jun 22, 2021 at 9:09
  • $\begingroup$ Then you can certainly use it. Just take care of the points i mentioned in modeling as they may affect your result significantly. $\endgroup$ Jun 22, 2021 at 9:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.