I am currently using tensorflow to create a neural network that does basic binary classification, and I would like to examine the bias of the model after training.

I have a dataset of about 300,000 samples, and each sample has a list of 112 features. Prior to training, we've selected 20 features that we want to use for prediction - race is a feature in the dataset, but it is not used for training the neural network.

What I'd like is to be able to see how often a variable with a different values for "race" gets misclassified, without including the race feature in the training dataset. Is there a way to essentially drop that feature during training, then re-append it to the dataset to see what the accuracy is on specific groups? Or to tell tensorflow to ignore that variable while training?


1 Answer 1


If you have

  • df the dataset with all the columns
  • df_train and df_test the datasets for train and test with all the columns.
  • X_train and X_test the datasets for train and test with only the selected columns.
  • y_train and y_test the target variables.
  • model your model.

you can do something like

# make predictions
model.fit(X_train, y_train)
predictions = model.predict(X_test)

# create a dataframe with predictions and race
# here you can add as many columns as desired
race = df_test['race'].values
results = pd.DataFrame(
        {'predictions': predictions,
         'race': race

# inspect the effect of race in predictions - even if race is not used in the model

Also, if you want to go deeper in the field of AI fairness I recommend you this project: https://fairlearn.org/


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.