2
$\begingroup$

I know this question is broad, but I need an advice to know if it's possible to achieve what I want to do.

The problem is that I have around 2500 documents with sensitive data being replaced by four dots. I do not have the original documents, so I wonder if there is a way to build a model that can detect sensitive data from any new documents (with sensitive data not being removed) using the previous documents? I want to apply machine learning or deep learning approaches. And what I know is that the original data set with annotated sensitive data should be used for training, which I can't obtain.

I am new at this field so any advice would be very appropriated

|improve this question
$\endgroup$
1
$\begingroup$

Welcome to the site! Assuming that I understand your problem correctly, I think you can achieve a working model.

If I was in your position I would:

  1. Obtain the cleanest data possible from the documents. For example, you don't state if the docs are already in simple text or if you need to do something like OCR or whatnot. Having the cleanest set possible will be key for this.
  2. Make sure you have a consistent marker for the sensitive data. You mention four dots - is that the case for ALL instances? If not, clean that data now
  3. You're going to need to do standard NLP cleansing stuff like removing punctuation but you may or may not want to keep stop words (this will be part of your model testing). Also, this is key, be 100% certain that the four dots are viewed as a single work in your tokenization process - you should be able to verify this prior to committing to your tokenization file.
  4. I would take all my documents and create 3 word ngrams. I would then separate out ngrams that contain sensitive data and not sensitive data. That, essentially, becomes your labeled dataset and you should label them accordingly.
  5. My base model would use all entries that contain sensitive data in the second position of the ngram (the middle of the three words). I would train a neural network on that and see what kind of results I achieve with that. NOTE that your four dots will not be an input, only the word previous and after will be your inputs. You could almost treat this as a binary classification model - the middle word is either sensitive or it's not.
  6. Future iterations of my model would maybe use a multi-classification approach with something like (1) No sensitive data (2) sensitive data in first position (3) sensitive data in second position and (3) sensitive data in third position and so on and so on.
  7. From there, you can play with variations on the size of the ngram since the immediate words may or may not actually have an effect on the predictions. There's no limit to how crazy you can get with this - you won't know until you start modeling.

Finally, your entire project becomes even more interesting when you go to the prediction phase with new data. You will do the same and break down your document into ngrams and create a prediction for each one and output the result. In other words, you will need to break down your document only to turn around and build it up again - that should be a fun script to write! Good luck with this, let us know how it turns out.

|improve this answer
$\endgroup$
  • 1
    $\begingroup$ Thank you very much for your informative answer. Forgive me if I'am asking basic questions, what you suggested is that I approach the problem as classification task where given the first and third words the model should predict the probability that the middle word is sensitive ? Do have any suggestion in which neural network architecture to use? Also, I want to know if you think using models like LSTM would be possible? $\endgroup$ – user971961 Mar 18 '19 at 18:29
  • $\begingroup$ @user971961 In effect, yes. Train the model on two words combos that do and don't have a sensitive word in between.That would be your binary classifier and you can use that as a baseline model (and it may very well be all you need for your problem). From there, I would change my training set to a multi-classification problem where for string containing X grams it would tell you if there is a word there that precedes a sensitive word. That should be more than enough to get you started and get a working model up and running. $\endgroup$ – I_Play_With_Data Mar 18 '19 at 18:33
  • $\begingroup$ @user971961 Yes, LSTM would absolutely be a possible model to use. They usually work well for NLP problems and I could see a path forward that would have you using an LSTM neural network to solve this. But, that shouldn't be your concern right now. Focus on getting the datasets you're going need and try a variety of models, not just LSTM $\endgroup$ – I_Play_With_Data Mar 18 '19 at 18:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.