4
$\begingroup$

Field-aware factorization machines (FFM) have proved to be useful in click-through rate prediction tasks. One of their strengths comes from the hashing trick (feature hashing).

When one uses hashing trick from sci-kit-learn, one ends up with a sparse matrix.

How can then one work with such a sparse matrix to still implement field-aware factorization machines? SKLearn does not have an implementation of FFM.

EDIT 1: I want to perform feature-hashing/hashing-trick for sure in order to be able to scale FFM to millions of features.

EDIT 2: Pandas is not able to scale to many fields. I also want to convert an arbitrary CSV (containing numerical and categorical features) into LIBFFM (field:index:value) format and perform hashing trick at the same time (preferably without using Pandas). Pandas2FFM does not scale even after performing the Hashing Trick.

$\endgroup$
1
+50
$\begingroup$

One option is to use xLearn, a scikit-learn compatible package for FFM, which handles that issue automatically.

If you require feature hashing, you can write a custom feature hashing function:

import hashlib

def hash_str(string: str, n_bins: int) -> int:
    return int(hashlib.md5(string.encode('utf8')).hexdigest(), 16) % (n_bins-1) + 1
$\endgroup$
7
  • $\begingroup$ xLearn also takes as input a file of the form field:feature:value and not a sparse matrix. I want to perform a hashing trick for sure. This is the only way to scale FFM to millions of features. $\endgroup$ – Sumit Sidana Jun 5 '20 at 20:24
  • $\begingroup$ It is not evident that xLearn handles large number of features automatically. $\endgroup$ – Sumit Sidana Jun 5 '20 at 20:34
  • 1
    $\begingroup$ Good point. I amended my answer to handle feature hashing. $\endgroup$ – Brian Spiering Jun 5 '20 at 23:13
  • $\begingroup$ Is it also possible for you to confirm the whole picture: This is what I understood: Input: Pandas Dataframe 1. Apply hash_str to categorical columns with a lot of values 2. Convert data frame to FFM-based input file using pandas2ffm 3. Train and predict using FFM $\endgroup$ – Sumit Sidana Jun 6 '20 at 10:48
  • 1
    $\begingroup$ If scalability is an issue, avoid using Pandas. $\endgroup$ – Brian Spiering Jun 6 '20 at 14:04
1
$\begingroup$

I normally dont use sklearn for the encodings but "category encoders package":

Have you consider using their Hashing Encoder?:

The advantage of this encoder is that it does not maintain a dictionary of observed categories. Consequently, the encoder does not grow in size and accepts new values during data scoring by design.

The output are int64 features. Category encoders API es easy to use and can be implemented in a transformer

$\endgroup$
1
  • 1
    $\begingroup$ Yes, it contributes to the answer (upvoted). But, I am looking for an answer without Pandas as I haven't managed to scale Pandas with many fields, when working with FFM. But, thank you for your contribution. $\endgroup$ – Sumit Sidana Jun 10 '20 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.