1
$\begingroup$

I am about to kick off a large hackathon event.

We have a dataset that is comprised of one continuous variable with high precision, and a number of categorical variables qualifying these data 3-levels deep.

Data provider wants to 'mask' the data such that the original values cannot be reverse-engineered. I'm not worried about the categorical variables, this is simple. But the continuous variables are tricky.

  1. a logarithmic transformation is easily reverse engineered
  2. a nonlinear transformation is better, but will mess with the relationship of values between categories
  3. a pure linear transformation would work, but doesn't seem to 'mask' enough.

I need to preserve the relationships between numbers whilst also protecting the actual, true values.

Ideas greatly appreciated.

Improve this question
$\endgroup$
1
  • $\begingroup$ theoreticaly any one-one transformation is invertible (no matter linear or not), on the other hand a not one-one transformation messes up the problem $\endgroup$
    – Nikos M.
    Feb 3, 2021 at 8:15

1 Answer 1

Reset to default
1
$\begingroup$

I think you can use a much more complicated monotonic transformation, like

log(1.234578 + sqrt(x + 7.4142) ** 3)

which will be harder to invert than a simple log. But, as Nikos says, strictly monotonic functions are invertible, so all you can do is make it very hard to compute the inverse by composing many monotonic functions.

Improve this answer
$\endgroup$
6
  • $\begingroup$ maybe use some "unique" hash (cryptographic) function with unique key/salt? $\endgroup$
    – Nikos M.
    Feb 3, 2021 at 16:24
  • $\begingroup$ is it possible not to lose the monotonicity? can you provide an example? $\endgroup$
    – David Masip
    Feb 3, 2021 at 16:43
  • $\begingroup$ I like this approach, but doesn't this transform, while monotonic, screw with the relationships between numbers? e.g., a regression run on the transformed numbers won't yield the same parameter estimates as the original numbers, right? $\endgroup$
    – HEITZ
    Feb 3, 2021 at 16:52
  • $\begingroup$ yeah, the parameters will change, but they change when you apply a logarithm too. However, given a model you'll always be able to recover the model on the original variables by calculating the inverse of the transformation function $\endgroup$
    – David Masip
    Feb 3, 2021 at 16:57
  • $\begingroup$ I implied using a "one-one" cryptographic hash (with unique key/salt) as the desired transform, but.. $\endgroup$
    – Nikos M.
    Feb 3, 2021 at 17:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.