4
$\begingroup$

I have a large dataset for the activities performed by multiple staff in a factory over a long period of time - 01/01/2017 - present. The activities performed by the different staff are recorded at each point in time (since they interact with software). I have tabulated these to record the number of activities performed by each operator for each day.

My table looks something like this:

Date         Name    Activity  UnitsProcessed   Shift      Team
01/10/2017   MMouse  Soldering   1000           Shift A   Team A
01/10/2017   MMouse  Welding     3000           Shift A   Team A
01/10/2017   DDuck   Soldering   4000           Shift A   Team B
01/10/2017   DDuck   Welding     1000           Shift A   Team B
...
02/11/2017   MMouse  Soldering   1100           Shift A   Team A
02/11/2017   MMouse  Welding     2900           Shift A   Team A
02/11/2017   DDuck   Soldering   4010           Shift A   Team B
02/11/2017   DDuck   Welding     900            Shift A   Team B

Each line represents the activities performed by each operator each day they worked, and the number of units processed during each activity, along with a classifier for the shift and team that the operator was a member of (for simplicity, assume that this does not change over time).

For brevity, I have omitted values from Shift B, but essentially, the team members of Shift B will (usually) work different days from Shift A, but the proportion of tasks performed will usually be similar to the corresponding Team on Shift A - Team A or Team B.

The problem I want to solve is: I have the date, activity, and units processed - as well as various other variables - for another operator, "GOofy". But I don't know which shift/team Mr. GOofy works. So I am using the data for Mr. Mouse and Mr. Duck (obviously, in reality, scores of people) to see if I can try to train kNN to correctly guess the Shift and Team pattern for each Name, given the Date, Activity, and Units Processed fields, among others.

My thinking is:

a) There is no particular value in keeping Team and Shift separate, so I should create a joined column that combines both

B) If I create separate columns for "Soldering", "Welding" and the other activities, I can have the number of units processed for each day for each activity by date. Given that the kind of activity performed is (I hypothesize) a strong predictor of the shift/team combination, I should be able then to train to recognize shift/team based on both the date of that line (perhaps converted to a numeric date) and the relative values for each line of soldering, welding, etc.

The problem is that, when extended to the test set, this will only give me predictors for each individual day. But I suspect that the days worked are also a significant predictor, so I would like to include this as well. But I'm not sure how to approach this.

The two approaches I can think of are:

Option 1: To use kNN to generate estimators for shift/team for each operator/day combination, with the x variables being the date, units soldered and units welded per day, and then calculate, for each operator, the proportion of times "Shift A, Team A", "Shift A, Team B", etc were assigned to that operator - so if DDuck is assigned "Shift A, Team A", on 90% of days, it seems most likely that DDuck is indeed "Shift A, Team A".

Option 2: To somehow create a variable that represents all of the days worked for each operator - although I'm not sure how to do this - and thus create a set that contains one row for each operator.

Is either of these approaches valid? Is there a widely-used alternative approach for this?

Note that this is not a duplicate of previous questions relating to kNN and time-series data, since those were concerned with forecasting, whereas I am trying to analyze retrospective data.

$\endgroup$
2
  • $\begingroup$ Will you have units processed for all dates and all activities? Same list of dates for training and classifying GOofy? Or will there be some unknowns? If not, you could put the data for each operator into a $D*A$ length flattened vector where D is number of days and A number of activities. $\endgroup$
    – AHusain
    May 14 '18 at 13:28
  • $\begingroup$ To use kNN you will need numerical data. Search for tabular embedding or graph embedding for some methods on converting your kind of data to a numerical embedding $\endgroup$
    – Jon Nordby
    Oct 16 '20 at 7:55
0
$\begingroup$

Do you have more numeric features? KNN works off distances (euclidean, L1, etc.) which requires numeric data. It doesn't take in categorical data. Your activity feature will not work with KNN and even converting the date to say an integer will just create a time series that will not perform well with KNN. I think you're approaching this wrong. Try logistic regression. It will take in both categorical and numeric features.

There is no particular value in keeping Team and Shift separate, so I should create a joined column that combines both.

I would build two separate models for team and shift. You'll get higher accuracy because you're using the data to predict fewer classes and letting the models specialize on each task.

Overall I would use logistic regression, not KNN. It's more appropriate for your problem. Hope this helps.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.