0
$\begingroup$

I have a regression problem in which for each week I have several rows (variable between rows i.e 1 week might have 1800 rows and other might have 5000 rows).

My target is to predict a value at end of each week's data. Here's an example of what I need : x is a feature y is the target.

Week 1 ; x1, x2, x3.. x90

Week 1 ; v1, v2, v3... v90

.... 100 more rows

Week 1 ; z1, z2, z3... z90

Week 1; y1

Week 2; a1, a2, a3.. a90

Week 2; c1, c2, c3.. c90

.... 500 more rows

Week 2; r1, r2, r3.... r90

Week 2; y2

And so on..

I have tried aggregation to weekly values but the results are too bad to be believable, the best model could make a guess at random.

The target values also contain a lot of zero's much like zero-inflated data.

Can you guys help me format this problem into a solvable format and any recommendations for the models to use for such a data?

Total number of weeks are around 120 and total number of feature for each instance (rows) are around 90.

Improve this question
$\endgroup$
6
  • $\begingroup$ Welcome to DataScienceSE. The question is not very clear to me: what the different letters represent? are all the $y_i$ part of the target? if not, what are the $y_i, z_i$, are they also features? And finally are you sure that these features can help predict the target? $\endgroup$
    – Erwan
    Commented Apr 23, 2022 at 11:48
  • $\begingroup$ @Erwan Thank you for your reply, I have edited the question for a little more clarity, here y1 & y2 are targets (Regression problem), x, v, c & a, c, r are instances of same week . Here's an example of what I'm trying to accomplish, I want to predict the next week completion in a project, in any week there may be 100 activities or even 10000, since the target is just next week prediction these activities either needs to be aggregated or a ML approach that can handle such instances such as multi-instance-learning can be used. $\endgroup$
    – Aneeq
    Commented Apr 23, 2022 at 22:07
  • $\begingroup$ I think there might be simply too much information, probably a lot of it redundant and some of it contradictory. But also in your example there's no guarantee that the target can be predicted reliably from the features: it's possible that one activity hits an obstacle and the whole project can't be completed. Anyway what you could try is feature engineering to simplify the features, or feature reduction with something like PCA or SVD. Maybe it's even possible to do some kind of embedding but I don't know much about this. $\endgroup$
    – Erwan
    Commented Apr 24, 2022 at 13:51
  • $\begingroup$ You've already identified the right keyword, multi-instance learning (I've added the tag). To know how best to approach modeling, I think we'd need to know more about the context; see the Assumptions section of the wikipedia article. $\endgroup$
    – Ben Reiniger ♦
    Commented Apr 24, 2022 at 14:39
  • $\begingroup$ Thank you for your replies. I'll try out different approaches, how can I make sure that my target is even predictable ? (I know I might sound naive here). The reason I ask is because whatever method I try it doesn't seem to work better than a random guess. $\endgroup$
    – Aneeq
    Commented Apr 24, 2022 at 22:46

1 Answer 1

Reset to default
1
$\begingroup$

Try to make all the rows of the week one row.

Considering the max number of rows/weeks:

Week 1; $x_1, x_2, x_3...x_{90}, v_1, v_2, v_3...v_{90}...z_1, z_2, z_3...z_{90}; y_1$

Week 2; $x_1, x_2, x_3...x_{90},v_1, v_2, v_3...v_{90}...z_1, z_2, z_3...z_{90}; y_2$

$y_i$ depends on all the rows of the week.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.