Tackling an automatic labeling (optimization) + classification problem

Question

I'm aware this isn't quite programming-focused, but I did not know any other place to ask this question. It's more about approach than a technical issue.

Context

I have an optimization + classification task. So essentially, my data has the following columns: ['Model ID', 'Q', 'refinement', 'avg_time', 'lattice', 'radius'] (there are more but for brevity, let's keep it just those)

• Model ID: represents a "design", each Model ID will have multiple rows

• Q: This is the target variable

• refinement: this is a setting variable; it could take values 1-8, and this directly affects Q, the (Model ID, refinement) pair is unique. So a Model ID will have multiple rows with varying refinements

• avg_time: this is the time taken for the simulation to finish, this is affected only by refinement, the higher the refinement, the more time it takes. This value is design-agnostic, it only depends on refinement, so all designs at a specific refinement have the same time taken.

• lattice and radius: These represent a "Design", essentially changing those will change Q

Dataset

My dataset is from a simulation of randomized designs. For each design, we could have the following behavior:

• continuously increasing (the Q values at each refinement is higher than the previous refinement level)
• continuously decreasing (the Q values at each refinement is lower than the previous refinement level)
• Zig-Zag in which the Q values follow this current pattern (high, low, high, low) or (low, high, low, high)
• Bowl in which the Q values follow this current pattern: (high, low, low, high)
• Trapezoid in which the Q values follow this current pattern: (low, high, high, low) I have code that detects these shapes and returns a boolean:

def is_zigzag(q_values)

def is_bowl(q_values)

def is_trapezoid(q_values)

The range of values for refinement in the dataset is 2-5, but the refinement could take values 1-8.

Task

So what I'm trying to achieve is to automatically label each Model ID (by grouping the rows) with the optimal refinement value (in the range 1-8) that maximizes the change in Q (higher delta is better) and minimizes the time taken (lower is better). The problem is that because the data is cut at refinement level 5, I had the idea of using probabilistic methods (e.g., MLE) to create an expected change in future refinements and an expected time taken. But I can't seem to "tune" it, so it's useful. After getting the expected values, I need a cost function to calculate (optimize) the return on Q compared to the increased time taken for that refinement level to finish.

In the next part, I will develop a classifier that will take the designs and the optimal refinements. It should, theoretically, predict the optimal refinement level for an unseen design

I'd appreciate any guidance/help in tackling this problem.