How to visualize data after performing OneHotEncoding and normalization?

Question

I have a dataset and on that, I have performed OneHotEncoding and Standardization using standard scalar, Now that I have preprocessed data I have to visualize it, but on converting it to pandas dataframe it is showing error. What is the way to visualize preprocessed data?

steps that I have taken

df = pd.read_csv(filepath)

categorical_features = [column names which are categorical in nature]

numerical_features = [column names which are numerical in nature]

one_hot = OneHotEncoder()
scaler = StandardScaler()

tranformer = ColumnTransformer([("one_hot", one_hot, categorial_features),("standard_scaler", scaler, numerical_features)], remainder = "passthrough")

transformed_X = tranformer.fit_transform(X)

Now, how to view this transformed_X in a tabular manner that we see using the .head() function of pandas dataframe?

Answer 1

Try this,

col_names = list(tranformer.named_transformers_['one_hot'].get_feature_names())+numerical_features

df1 = pd.DataFrame.sparse.from_spmatrix(transformed_X)
df1.columns = col_names
df1.head()

Answer 2

After applying one-hot encoding and standardization, the transformed_X data is no longer a pandas DataFrame, but a numpy ndarray. To convert it back to a pandas DataFrame, you can use the pd.DataFrame() function as follows:

transformed_X_df = pd.DataFrame(transformed_X, columns=tranformer.get_feature_names())

This will give you a DataFrame with the same number of rows as the original dataset X, but with transformed columns. You can then use the .head() function to view the first few rows of this DataFrame:

transformed_X_df.head()

Alternatively, you can also use the pandas.concat() function to concatenate the transformed data with the original data and create a new DataFrame with all the columns:

X_df = pd.DataFrame(X, columns=[categorical_features + numerical_features])
transformed_X_df = pd.DataFrame(transformed_X, columns=tranformer.get_feature_names())
final_df = pd.concat([X_df, transformed_X_df], axis=1)
final_df.head()

This will give you a DataFrame with all the original columns as well as the transformed columns.