0
$\begingroup$

I am currently training an XGBoost model for binary classification. I have fitted and predicted with the model but when I try to get the "gain" type feature importances, the results differ based on what method/function I call to get the importances. The following chunks of code illustrate what I am talking about.

chunk1:

# Access the booster object
booster = model_tuned.get_booster()
#get importances
importance = booster.get_score(importance_type='gain')

print("Feature Importance's(gain)", importance)

Output(truncated for space):
Feature Importance's(gain) {'Value_Engage_Bus_Number_cmd_l_121001000_110_0.0+10': 2493.173143016667, 'Value_ADF_Bearing_121322000_170+8': 1444.65723, 'Value_Speed_Brake_Panel_9_Position_71472000_110': 861.343201, 'Value_Total_Cumulative_Flight_Dur__Seconds_92029000_140+8': 428.6793822, 'Value_Hybrid_EW_Velocity_True_80341000_110+10': 353.6710613333334, 'Value_Hybrid_Wind_Speed_80031000_110+7': 468.978516, 'Value_HMU_Torque_Motor_lane_a_22469000_140+1': 222.413793325, 'Value_Low_Limit_Valve__Dead_Band_Offset_Error_200249000_180+4': 331.243164, 'Value_Yaw_Servo_Torque_71671000_110+10': 323.87207, 'Value_Magnetic_Variation_80121000_110+8': 251.5, 'Value_Avionics_Ch_2_Timeout_Status_fdbck_l_121021000_180+7': 4044.4652635, 'Value_Engage_Bus_Number_cmd_l_121001000_110_1.0+10': 976.6852322999999, 'Value_Total_Cumulative_Flight_Dur__Seconds_22029000_140+4': 907.075684, 'Value_Cabin_Altitude_121711000_110+4': 766.206543, 'Value_Engage_Bus_Number_cmd_l_121001000_110_2.0+8': 663.79834, 'Value_N1_Red_Line_Trimmed_lane_b_91449000_160+10': 443.595215, 'Value_RVDT_Position_40539000_140+6': 405.405273, 'Value_N1_Max_Cruise_Rating_123601000_150+5': 188.24823778, 'Value_Arm_1_72040000_110_3.0+7': 1800.49854, 'Value_Stall_Warning_Speed_Ratio_120411000_140+4': 482.21429450000005, 

chunk 2:

from xgboost import plot_importance
plot_importance(model_tuned ,max_num_features=30, title="Feature Importance-Gain",importance_type="gain")

Output: Gain Feature Importance Why are the outputted features different when both methods use the same model and importance type?

$\endgroup$
1
  • $\begingroup$ Please format your code. // What do you mean "the outputted features [are] different"? With a quick scan they look the same to me. $\endgroup$
    – Ben Reiniger
    May 25, 2023 at 18:54

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.