I have a dataset collected in a smart home and I would like to do Activity Recognition. I am used to classification but in this case data comes from 4 different sources:

  • Accelerometer from wrist
  • Binary sensors around the house
  • proximity beacons (to understand which room the user is at any moment)
  • ground pressure from floor

Data is labelled and has 25 activities.

At the moment I am working with a CNN on the activity images coming from the sensors and I can reach 45% accuracy, but I need to add the other sources to achieve better results.

My questions:

  • how to use all (or a group of) different sources for example in a NN?
  • Can I train 4 different classifier that works together to make a prediction?
  • Maybe some classifiers try to understand simpler activities and other try to find more complex activities?

Data-fusion is the most probable way to "fight" with all of the data sources. However, additionally, I would strongly suggest imposing an ontology and some set of rules.

Basically what you can do is the following:

-proximity beacons (to understand which room the user is at any moment) based on this sensor data you can introduce localization, decision trees may the most suitable way to go here while imposing some clear manually added rules(if the person is in this room, then goes into this room, etc). Basically, before doing the overall data fusion the localization may be really helpful in increasing the accuracy because you can introduce the ontology with specific activities that can be done in some of the rooms. (The person is not expected to be running in the kitchen or the bathroom for example).

ground pressure from the floor - this type of sensor can be really useful for localization as well.

Also, before data-fusion, the Accelerometer from wrist data readings should be carefully pre-processed. I would strongly suggest finding a nice way how to remove the bias(left handed/right-handed people) and how to remove the DC component(that is the overall gravity of the Earth gravitational field, remove this bias as well)


I ended up using multi input neural networks, where each input is used for each source.

Also called Data-fusion.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.