I've read and heard about the mighty XGBoost which is one of the most famous models people are using today to solve Kaggle challenge. This makes me interest to develop my own intuition about the model so I decided to try XGBoost on my dataset, but unfortunately, I experienced a lot of dependency related problems when I tried to install XGBoost on mac :(
However, I did a bit more researches and found out another related library called LightGBM from Microsoft which they claim to achieve better result (or at least, equivalent) over XGBoost. Therefore, I decided to go with the LightGBM instead since I didn't encounter any problems during the library installation.
I expected that the model should outperform, or at least, perform similarly compared to other models on the same dataset. My dataset contains 14k of text documents and has 0 or 1 values which refer to positive and negative sentiment respectively as target variable.
As a result showing below, LightGBM with count vectorizer as input got the 6th place with an accuracy of 94.60%. I'm quite curious since the accuracy of LightGBM is not only lower than SVC but also lower than Extra Tree model (300 esitimators).
I don't have enough foundation and solid understanding behind these models so I couldn't come up with an answer about these outcomes. Can someone please give me an idea or assumption according to the result?
PS. All models was performed with 5-fold cross validation
model score -------------- ------- extraT 0.9528 svc 0.9514 sgd_elas 0.9481 extraT_tfidf 0.9476 svc_tfidf 0.9473 lightGBM 0.9460 sgd_elas_tfidf 0.9458 lightGBM_tfidf 0.9420 randomF 0.9409 randomF_tfidf 0.9345 mulNB 0.9307 berNB 0.9087 berNB_tfidf 0.9087 mulNB_tfidf 0.9036