4
$\begingroup$

Field-aware factorization machines (FFM) have proved to be useful in click-through rate prediction tasks. One of their strengths comes from the hashing trick (feature hashing).

When one uses hashing trick from sci-kit-learn, one ends up with a sparse matrix.

How can then one work with such a sparse matrix to still implement field-aware factorization machines? SKLearn does not have an implementation of FFM.

EDIT 1: I want to perform feature-hashing/hashing-trick for sure in order to be able to scale FFM to millions of features.

EDIT 2: Pandas is not able to scale to many fields. I also want to convert an arbitrary CSV (containing numerical and categorical features) into LIBFFM (field:index:value) format and perform hashing trick at the same time (preferably without using Pandas). Pandas2FFM does not scale even after performing the Hashing Trick.

$\endgroup$

2 Answers 2

1
+50
$\begingroup$

One option is to use xLearn, a scikit-learn compatible package for FFM, which handles that issue automatically.

If you require feature hashing, you can write a custom feature hashing function:

import hashlib

def hash_str(string: str, n_bins: int) -> int:
    return int(hashlib.md5(string.encode('utf8')).hexdigest(), 16) % (n_bins-1) + 1
$\endgroup$
7
  • $\begingroup$ xLearn also takes as input a file of the form field:feature:value and not a sparse matrix. I want to perform a hashing trick for sure. This is the only way to scale FFM to millions of features. $\endgroup$
    – learner
    Jun 5, 2020 at 20:24
  • $\begingroup$ It is not evident that xLearn handles large number of features automatically. $\endgroup$
    – learner
    Jun 5, 2020 at 20:34
  • 1
    $\begingroup$ Good point. I amended my answer to handle feature hashing. $\endgroup$ Jun 5, 2020 at 23:13
  • $\begingroup$ Is it also possible for you to confirm the whole picture: This is what I understood: Input: Pandas Dataframe 1. Apply hash_str to categorical columns with a lot of values 2. Convert data frame to FFM-based input file using pandas2ffm 3. Train and predict using FFM $\endgroup$
    – learner
    Jun 6, 2020 at 10:48
  • 1
    $\begingroup$ If scalability is an issue, avoid using Pandas. $\endgroup$ Jun 6, 2020 at 14:04
1
$\begingroup$

I normally dont use sklearn for the encodings but "category encoders package":

Have you consider using their Hashing Encoder?:

The advantage of this encoder is that it does not maintain a dictionary of observed categories. Consequently, the encoder does not grow in size and accepts new values during data scoring by design.

The output are int64 features. Category encoders API es easy to use and can be implemented in a transformer

$\endgroup$
1
  • 1
    $\begingroup$ Yes, it contributes to the answer (upvoted). But, I am looking for an answer without Pandas as I haven't managed to scale Pandas with many fields, when working with FFM. But, thank you for your contribution. $\endgroup$
    – learner
    Jun 10, 2020 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.