0
$\begingroup$

I'm trying to work on a ML project and I have a dataset and I'm trying to see if there is a correlation between some of the features in my dataset.

The dataset contains inspection notes for car parts in an industrial process along with some information about the car parts themselves.


I have posted a Jupyter Notebook with all of my code on GitHub. The data used is in the same folder. But to get back to my question, I've plotted a simple correlation matrix to show how the features correlate.

I've done some data augmentation so that I have all numerical values and be able to calculate and plot the matrix.

But after giving this a thought and reading about correlation, I'm not sure that my the correlation makes much sense (in the real world).

If my understanding is correct, correlated variables would be useful in a

The below plot is a representation of correlation between variables for all of the car parts that have been thrown away after inspection.

However, I'm not 100% sure that my following statement is correct, based on the correlation matrix below:

"There seems to be some correlation between the MONTH and the MINUTE features in my dataset."

But although I believe the numbers show this, I don't think this is valuable information and after reading I believe that correlation would work best with measurement values, not inspection values.

I'm seeking help to confirm or help me improve my understanding of what correlation is and how it would apply in my scenario.

enter image description here

I have a hunch that it could be applicable, but I need to tweak my dataset. Like I need to do a count aggregate on all features and build the correlation matrix and plot that, but I'm not sure yet...

$\endgroup$
1
$\begingroup$

I see you have done encoding of feature line using the below code:

line_dict = {'INSPECTION':'0', 'REPAIR':'1' }

if we see the type of the value as below:

   >>> line_dict['INSPECTION']
       '0'
   >>> type(line_dict['INSPECTION'])
   <class 'str'>

it seems it is of object type. This could be the problem. Suggest you do to one hot encoding technique for this feature

Also it is good to an info on your dataset to identify the object columns

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 153381 entries, 0 to 153380
Data columns (total 12 columns):
bodyId        153381 non-null int64
bodyTypeId    153381 non-null int64
color         153381 non-null int64
line          153381 non-null object
shiftNo       153381 non-null int64
Inspection    153381 non-null object
mYear         153381 non-null int64
mMonth        153381 non-null int64
mDay          153381 non-null int64
mHour         153380 non-null float64
mMinute       153380 non-null float64
mSecond       153380 non-null float64
dtypes: float64(3), int64(7), object(2)
memory usage: 14.0+ MB

if you check the data type line it says object which means the line feature is String

|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.