How to curve fit, Z variable dependent on X and Y?

Question

I'm trying to find the function for this visualization: I would like to get feedback if I'm taking the right approach. My approach:

1. These data points are created by a person. They are two independent variables (x and y axes) and one dependent variable (the colored dots).
2. I asked ChatGPT what I should do and it told me to try to fit the curve. Try different models and evaluate.
3. I randomly splitted the training and test data sets (80/20).
4. The models I tried: Decision tree, random forest, curve fit, linear regression, logistic regression and polynomial features.
5. All models gave bad results (negative r squared value) except for linear regression with PolynomialFeatures (r squared value of 0.339). But this results still seem low right?

Here is my data and code:

z,x,y
0.6,0,1
0.55,0.1,1
0.5,0.2,1
0.4,0.3,1
0.3,0.4,1
0.2,0.5,1
0.9,0,3
0.75,0.5,3
0.35,1,3
0.15,1.5,3
0.05,2,3
1,0,7
0.9,1,7
0.75,2,7
0.35,3,7
0.2,4,7
0.1,5,7
0.02,6,7
1,2,14
0.8,4,14
0.5,6,14
0.05,9,14
0.01,10,14
1,5,25
0.65,9,25
0.1,13,25
0.01,17,25
1,8,50
0.8,13,50
0.3,18,50
0.1,23,50
0.01,30,50

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
import pandas as pd
from sklearn.preprocessing import PolynomialFeatures
from sklearn.metrics import r2_score
import matplotlib.pyplot as plt

df = pd.read_csv("scores.csv")
X = df.iloc[:, 1:].values
y = df.iloc[:, 0].values

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

poly = PolynomialFeatures(degree=3)
X_train_poly = poly.fit_transform(X_train)
X_test_poly = poly.transform(X_test)

poly_reg = LinearRegression()

poly_reg.fit(X_train_poly, y_train)

y_pred = poly_reg.predict(X_test_poly)

y_pred = np.clip(y_pred, 0, 1)

I have a feeling I'm doing something wrong. I think this problem could be solved easier because when you look at the visual you can see a clear pattern. I'm looking for some confirmation to see if I'm taking the right approach. Any feedback?

Answer 1

Overall, your approach makes sense.

There are a couple of ways of reframing it:

• Change the evaluation metric from R² to one that is prediction oriented (e.g., MSE or MAE). R² is useful but has limitations.

• Given the figure shows evidence of a more complex relationship, it might be more useful to fit a more expressive model (e.g., kernel regression or isotonic regression).