# Neural network obfuscation

Neural networks are trained to minimize some error function over the weights of the neural connections. In some applications, these weights could be considered intellectual property. Is there a way to encrypt these weights and still have an operational neural network?

Some context: I'm trying to scale a neural network algorithm, but right now we're doing all the computations on a centralized server and it's getting bogged down. We can shift the computation to the client side, but we don't want someone to unpack the executable and obtain the weights of the network. Is there some way to distribute an "encrypted neural network" such that our IP is protected?

To clarify: I am not talking about "oblivious" neural networks that operate on encrypted data, I am talking about the weights of the neural network themselves.

I'm fine with alternative obfuscation techniques.

Edit: I found this paper, but it states

Great attention is paid to avoid any unnecessary disclosure of information, so that at the end of the protocol the user only knows the final NN output, whereas all internal computations are kept secret. In this way, the possibility for a malevolent user to provide a set of fake inputs properly selected to disclose the network secrets is prevented. A solution is also sketched that permits to obfuscate the network topology, however, a deeper investigation in this direction is left for future research.

indicating that this paper is on a related concept, but I'm looking for a resource where the topology is obfuscated.

• When you say "unpack the executable" do you mean if the NN was a C program and you shipped just an executable binary (and no source code) then that would not be a sufficiently obfuscated solution for you? – Spacedman Sep 3 '16 at 8:29

No. What you want is probably not achievable in practice, because the approach you are considering obfuscates only the weights but does not obfuscate the inputs and outputs to the network. In any reasonable ischeme I can imagine, the input $x$ to the obfuscated network will be known and under the attacker's control, and the output $y$ from the obfuscated network will be known to the attacker. In particular, the attacker can pick any $x$ of his choice, and observe the output $y$.

If that is true, your goose is cooked. The attacker can use the ability to invoke your neural network to learn his own neural network that is just as good as yours. In particular, the attacker can assemble an arbitrarily large training set by picking many potential inputs $x$ and for each one computing the corresponding $y$ by running your obfuscated network on $x$, then putting the pair $(x,y)$ in the training set. This doesn't require labelled examples; it only requires the adversary to be able to put together a large set of unlabelled instances (and then use your obfuscated network to label those instances). Typically, it's not too hard for someone to create a large set of unlabelled instances.

Finally, once the adversary has assembled this training set, the adversary can train their own neural network using standard techniques. It's likely that the resulting neural network will be approximately as good as yours -- i.e., have approximately the same accuracy. (This is what seems to happen in practice.)

As a result, no obfuscation scheme for obfuscating the weights is likely to be terribly effective, because that doesn't hide the inputs and outputs to the network. The most you can hope for is a scheme that acts as a "speed bump" that slightly increases the cost of de-obfuscation or that raises the bar a little bit, but nothing you do will provide strong security against a knowledgeable adversary. So, don't spend too much time or energy or money on trying to make this work. Instead, you might be better off looking for other ways to deal with this issue.

P.S. Even if you could hide the outputs and reveal only the ultimate classification (i.e., hide the continuous probability values from the softmax and just reveal the highest-probability class), that's probably still not enough. Revealing the class is still enough for an adversary to label a bunch of instances, create a training set, and then train their own network.

My answer only applies to one niche, the database server market. If you're storing your weights in a SQL Server database, you can use Transparent Data Encryption (TDE). This will encrypt your whole database so that only authorized users can read its contents; it is also possible to encrypt individual columns to further restrict access, so that even authorized users can't read the contents of a table without first decrypting it.

Here's a link to some documentation on how to use TDE. It's actually really easy to use, but as with anything, there are a few potential "gotchas" when setting it up. For a short list of some of those I encountered, see my column Misadventures in TDE: How to Restore an Encrypted SQL Server Database Completely Wrong.

Normally you don't have to do any coding whatsoever in a front-end application (such as a GUI user access interface with spreadsheets, etc.) to use a TDE-enabled database; the user accessing the data just needs sufficient permissions. I know that .Net languages like VB and C# also have their own encryption commands you can leverage too if for some reason you needed to do encryption on the front-end too; I suspect other programming ecosystems like Java etc. have their own counterparts. I also fairly certain that SQL Server's competitors in the database server market, such as Oracle and MySQL, have their own versions of database encryption, but I'm not familiar with how they work. I firmly believe that in due time, a lot of neural net and other machine learning algorithms will end up running on database servers of this kind because of this and many other advantages. I'm pretty sure that Oracle etc. have free versions of their database akin to SQL Server's, which lets you store and operate on about 10 GB. If that's enough space for your neural nets, then you could just set them up on a dedicated server without having to pay for a database server license. Disclaimer: I'm not in any way connected with Microsoft or in a position to profit from it; I just know this one technology well. I use it to encrypt my own neural net weights, so I know from personal experience that this solution will work quite well.

If you're merely storing the weights in files rather than a database you can probably make use of disk and file encryption software, such as TrueCrypt or EFS and BitLocker for Windows. I'm weak in networking so I can't comment on transport security across the Internet, although I suspect technologies like SSL etc. would do the job. I hope this helps.

• So we need the neural network to be operational even with encrypted weights. It sounds like what you're describing just encrypts data when it's stored, but to actually make use of the data you have to decrypt it and ship it to the client, which is not actually what I want to do. The neural network needs to work without being decrypted. – Maxwell Johansen Aug 3 '16 at 21:34
• No, that's the thing - you don't need to decrypt it to use it - that why it's called "Transparent." I set up my own TDE encryption long ago and haven't had to mess with it at all. I wrote VB.net code to import the data and operate on the weights, display them in a GUI, etc. without once having to take the encryption into account or mess with it. I forgot it's even there. The data would be obfuscated, however, if I were a user without sufficient permissions tried to access it. – SQLServerSteve Aug 3 '16 at 23:01
• More clarification:any decryption occurs under the hood,so the neural net algorithms that access the weights don't even have to take TDE into account - as long as I've got the right permissions. I can change the decrypted values any way I like or display them etc. They only appear encrypted to unauthorized users; since I have the right permissions, I never see or operate on the encrypted values. Your neural net will work the same. I could also authorize the program to act as a middle man and display the decrypted data to unauthorized users just as seamlessly, should the use cases call for it. – SQLServerSteve Aug 3 '16 at 23:10
• We can't trust the user with the unencrypted weights though. Let's say Alice wants to use Bob's neural network, but Bob can't afford to rent out a server to crunch Alice's data. The weights for the NN are intellectual property, so Bob doesn't want to share the weights with Alice. Bob sends Alice some sort of holomorphic encrypted neural network so Alice can run the network on her own machine, without being able to derive the weights. Here Alice would need to be an "authorized user" that sees the weights in cleartext, i.e. Bob must trust Alice. Is there a trust-less way to do this? – Maxwell Johansen Aug 4 '16 at 4:34
• If you're an admin who can manage the Windows OS-level permissions for these neural net users, you just have to lock down those so they can't hack the encrypted database you've installed on their system. If they have admin rights and you don't, I don't think there's any foolproof way of encrypting data on another user's machine that a determined, knowledgeable user can't hack in time. As far as out-of-the-box solutions go, however, this is a reasonable option to consider, one that at least throws a lot of roadblocks in the way of anyone who wants to hack your neural weights. – SQLServerSteve Aug 4 '16 at 6:32