I'm working on a problem which is a multiple equation. I have a group of people and each person in the group is working on different tasks (e.g. n tasks in total). Each person in this group is working on multiple tasks and complete them. I'd like to find an estimation for the time each type of task takes.

I have equations like below:

#of days person i worked = time(task1) * #task of type 1 completed + time(task2) * #task of type 2 completed +...+ time(task n) * #task of type n completed

The only unknown in these equations are time (task i).

I have one equation per person in the group (there are 100k people). Does it make sense to use regression to find time (task i)?

  • $\begingroup$ Is there a way to break down your equation so that the number of features is the same for each record, and that each record has only one label? Or, is n always the same for each record? $\endgroup$ – The Lyrist Jul 13 '18 at 23:18
  • $\begingroup$ Each record has the same numberbof features/tasks, obviously many of them are zero. $\endgroup$ – H.Z. Jul 15 '18 at 1:10

Yes, linear regression is perfect for this.

If you knew that all the quantities were measured with perfect accuracy (no errors), you could solve using linear algebra; the system amounts to solving $Ax=y$, where $A$ is a matrix where each row corresponds to the number of each task for one person, and $x$ is a vector that reflects the time of each task, and $y$ is a vector that records the number of days each person worked.

If you suspect there might be small errors in the measurement of $y$, then using linear regression is appropriate.

You'll need that the number of people exceeds the number of tasks.

If you have a different error model, the optimal procedure might be something other than linear regression. For instance, if you have errors in both the measurements of $y$ and $x$, PCA (also called orthogonal regression) might be more suitable. If you have a more complex error model, you might want to ask on Stats.SE for the most appropriate inference procedure, or consider using stochastic gradient descent to minimize an appropriate loss function chosen based on your error model.

  • $\begingroup$ There is one issue, using linear regression might returns some negative coefficients which does not make sense in this context. Should I use something like Lasso and force the coefficients to be positive? Does Lasso make sense in this context? $\endgroup$ – H.Z. Jul 15 '18 at 0:14
  • $\begingroup$ @H.Z., you could consider a custom procedure that does optimization with the side requirement that all coefficients be non-negative (see the last sentence of my answer), but this might be more complex than necessary. The lasso is doing something different and isn't about ensuring coefficients are non-negative; it doesn't seem relevant here. $\endgroup$ – D.W. Jul 15 '18 at 1:25
  • $\begingroup$ There is an implementation of Lasso in scikit lear ln which allows non-negative coefficients but you’re saying it doesn’t make sense in this context and I shold avoid it? $\endgroup$ – H.Z. Jul 15 '18 at 2:09
  • $\begingroup$ @H.Z., no, there's no reason you need to avoid it, it's just that the Lasso part has nothing to do with non-negative coefficients. $\endgroup$ – D.W. Jul 15 '18 at 2:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.