Is the dataset fit for Linear and Logistic Regression

Question

I am trying to check the correlation in a red wine quality dataset via a scatter plot but it seems it just doesn't seem to be linear.

I have applied the preprocessing steps below:

1. Standard Scaler since the range was different for all the columns.
2. Treated outliers

So the standard scaler code is as below:

df_red = pd.read_csv('wine+quality/winequality-red.csv', sep=';')

def preprocess_data(df): # Trying to scale the data since columns likely have different ranges. 
    res_df = df.copy()
    std = StandardScaler()
    for i in list(res_df.columns):
        res_df[i]= std.fit_transform(res_df[[i]].values)
    return res_df

encoded_df = preprocess_data(df_red)

Original data:
Data looks like the below after applying the standard scaler:

Where quality is the target variable.

Now below is my code for treating outliers:

Q1 = encoded_df['fixed acidity'].quantile(0.25)
Q3 = encoded_df['fixed acidity'].quantile(0.75)

IQR = Q3 - Q1

LL = Q1 - 1.5 * IQR
UL = Q3 + 1.5 * IQR

ul_outlier_count = encoded_df[encoded_df['fixed acidity'] > UL].shape[0]
ll_outlier_count = encoded_df[encoded_df['fixed acidity'] < LL].shape[0]

total_outliers = ul_outlier_count + ll_outlier_count

perc_outliers = total_outliers * 100 / encoded_df.shape[0]

print(f'UL Outlier Count: {ul_outlier_count} | LL Outlier Count: {ll_outlier_count} | Total Outlier Count: {total_outliers} | Outlier%: {perc_outliers}')

#since my outliers are less than 5%
encoded_df.loc[encoded_df['fixed acidity'] > UL] = UL
encoded_df.loc[encoded_df['fixed acidity'] < LL] = LL

So below is my boxplot before treating outliers:

Below is my boxplot after treating outliers:

I have used a pair plot to identify which feature is strongly correlated with the target so that I can use that feature for training the model.

By looking at this pair plot can I say that the data is not fit for linear regression? Since the data is not fit for linear regression it would also not be fit for logistic regression, as in logistic regression we need to create a linear regression model first and then pass that model to the sigmoid function/logit function internally.

Or do I need to get more features for the training? I don't think the features are anywhere related to the target.

Answer 1

First things first: that's definitely not how you use the StandardScaler(). You don't have to wrap it around a function and iterate your dataset like that, Scikit will handle the different ranges in each column for you. By doing that you're refitting the scaler to each column and won't be able to use that instance when scaling other subsets of data (e.g. a holdout set). Just do something like:

df_red = pd.read_csv('wine+quality/winequality-red.csv', sep=';')
X, y = df_red.iloc[:,:-1], df_red['quality'] # extracts the target variable before scaling.
scaler = StandardScaler()
X_norm = scaler.fit_transform(X)

You don't need to manually treat outliers either, just use something like Feature Engine's Winsorizer. Take your time when reading these modules' documentations.

Second: That's not how a logistic regression works either, nor a linear regression for that matter. Generally speaking, a linear regression optimizes the mean squared error using a least squares formulation and is used for a continous output in regression problems, and a logistic regression optimizes the likelihood of the predictions using a Bayesian approach and is used for a discrete output in classification problems. Just because a problem isn't fit for a linear regression it doesn't mean it's not fit for a logistic regression. For this particular dataset I'd start with a logistic regression, but that doesn't mean that a linear regression wouldn't work.

Third: I (really) don't think you'll be able to evaluate your future model's performance by looking at scatter plots of the target and the input. It might have some value as some sort of step in an exploratory data analysis, sure, but otherwise it's mostly meaningless. There are more meaningful approaches such as evaluating the input variables' distributions via histograms, the boxplots you're doing are also a good approach. After fitting your model, you must evaluate it with previously unseen data, that means be careful with data leakages. If you can't evaluate your model's generalization, then it is for all intents and purposes worthless.