2
$\begingroup$

I have a dataset which contains data for about accidents. The dataset consists of about 15.000 entries and I can't get more. The Distribution is as follows:

  • 88.6% of the data are class 1 accidents
  • 10.6% of the data are class 2 accidents
  • 0.8% of the data are class 3 accidents

As you can see, the biggest part of the training data belongs to one class. I have only very few examples for class 3 accidents (about 100 rows out of 15.000), but it would be most important to classify class 3 accidents correctly.

I trained a pretty standard deep neural network on the data and got an accuracy of ~93% on the validation set. I used a custom Tensorflow estimator with an AdamOptimizer and tuned the parameters as good as possible. The Problem is, the network still classifies most of the accidents as class 1 accidents. So if I have for example 25 class 3 accidents in the validation set, the network misclassifies 10 of them as class 1. I want to improve that.

Are there any methods to improve the performance in this case? The obvious choice would be to get more data of class 3 accidents, but sadly that is not possible. Does it make sense to show the existing class 3 data multiple times? So for example, train 5 Epochs with all data and then 3 additional Epochs with just the class 3 accidents?

Or could I do something during the data preprocessing? Right now I'm MinMax-Scaling the input data to get to the [0, 1] interval. Is there maybe any other way to emphasize outliers more? (If you assume outliers mostly belong to class 3)

I hope someone knows some methods to increase the accuracy in this case.

EDIT: The Dataset has mostly categorical columns like:

  • Street Class (e.g. highway or country road)
  • Light (e.g. "good")
  • Weather (e.g. "rainy" or "sunny")
  • ...

Additional it has these columns:

  • Accident Date (just month and day)
  • Age
  • Time of the day
  • Number of injured persons
  • Number of Vehicles

So an entry might look like this:

{
 street_class: 'highway',
 light: 'daylight',
 date: '23. Jan',
 age: 59,
 injured_persons: 2,
 vehicles: 2,
 time: 1724,
 label: 1
 ...
}
Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

I'd try some kind of data augmentation, but from your question is not clear what type of data you have and it's impossible to suggest a solution.

Try to add an example of the data to your question.

Improve this answer
$\endgroup$
5
  • $\begingroup$ I added a brief description of the dataset $\endgroup$
    – Nils Schlüter
    Commented Sep 20, 2018 at 21:12
  • 1
    $\begingroup$ You are using machine learning because you wouldn't be able to classify an accident looking at the data? I guess not, I think you know what makes an accident of class 1 or 2 or 3. If that's the case I would simply use an if statement to discern the accidents $\endgroup$
    – Francesco Pegoraro
    Commented Sep 20, 2018 at 21:17
  • 1
    $\begingroup$ Note to Nils Schlüter: I think Francesco's comment here is IMMENSELY important. You should NEVER just dive in with an ML algorithm and hope you get what you want. You have to know your data first. Then you need to come up with a simple-as-possible algorithm to do what you want. Go to ML when the so-called "silly stuff" doesn't work. A very simple decision tree, especially when you have the exact characterization of the accident classes, could easily get you $100\%$ accuracy! $\endgroup$
    – Adrian Keister
    Commented Sep 21, 2018 at 13:08
  • $\begingroup$ I agree with you, but sadly i don't know what makes an accident class 1 or 2 or 3. I just got the data and the assignment to try it with some ML algorithm $\endgroup$
    – Nils Schlüter
    Commented Sep 21, 2018 at 15:34
  • $\begingroup$ Yeah, since you don't know what makes an accident what, I quote @Adrian Keister. You should use decision trees. Maybe remove the excessive class1 samples. $\endgroup$
    – Francesco Pegoraro
    Commented Sep 22, 2018 at 15:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.