Some feedback/tips/tricks/opinions here:
Problem setup
Including requirement analysis. Gotta decide how the system/solution should work, how to know ho how well we are doing, and then how to get there.
Model evaluation.
It is very desirable to have a quantitative way to evaluate our model performance.
For that we want some labeled data. It is very quick to count the number of ON events (given assumptions) for each house. House 1, 4 times. House 2, 3 times. House 3, 2 times.
Then at least we can use that as an evaluation metric.
Could also mark the times in the dataset to compute a percentage of ON time. That would be a more granular/precise metric to evaluate by. Strongest evaluation would be to check that each event time was also correct (within some tolerance).
Generalization
A model trained on N TVs/houses will also work on unseen houses. Very desirable, as allows the same model to be used for all houses, including future ones.
For this reason one should do possibly do Cross Validation grouped by house.
Might also want to use Time-Series splitting, but a bit hard on such a tiny dataset.
System behaviour assumptions
We assume that that Power(ON) >> Power(Standby) > Power(Off).
However each TV might have different Power levels, as well as differences/ratios between the three levels. Different houses may also have different behavioral patterns in turning on/off.
Minimum on time.
It is reasonable to assume/set some minimum time for TV on/off. Can be informed from data, or set based on what you know about TVs. Maybe 10 seconds is OK?
Technique feedback
EDA
Really missing a histogram over power level!
Denoisning
KNearestNeighbours is too complex/opinionated for this. A running mean or running median would be more suitable.
Classifier
GradientBoostingClassifier wayy to complex. You have a single feature and tiny amounts of data.
GaussianMixtureModel is quite relevant, as this is a classic case of Hidden States and transitions. However it is still not clear if the complexity is justified - ie if the problem cannot be solved with just state-independent thresholding
Report feedback
Your notebook indicates heavy work-in-progress, it does not have the qualities of a report. Of course if you did this live, on-the-spot or with a very tight deadline, that is natural. But if you had several hours on this task, I would expect a much cleaner notebook.
- The intro should define/re-state the problem(s). So that the report is self-container
- An Exploratory Data Analysis section should be before models. Question -> Data Exploration -> Findings.
- All non-interesting findings or explorations should be removed!
- Low use of global variables, proper use of functions. Models should use pipelines where appropriate
- Use pandas time series functionality. All time-series plots should have time on axis!
- After the EDA and models, Results, and then a Discussion/Conclusion
- You may want to also have the most important findings/conclusions summarized on top
