Is anyone using transformer based models for tabular data in real data science jobs as of 2024

I mean models like

  • Tabnet
  • Tabtransformer
  • ARM-net
  • FT-Transformer
  • Non-Parametric Transformers

I got these models from a blog post https://levelup.gitconnected.com/tabula-rasa-could-we-have-a-transformer-for-tabular-data-9e4b238cde2c and it also lists the paper "Why do tree-based models still outperform deep learning on tabular data?" Which claims tree-base models would still be superior.

I wonder if this still holds true in 2024. Maybe also somebody found a suitable transfer learning task where he profited from the deep learning methods a lot.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.