I've been participating in a project where I had to cut most of the dataset due to mismeasurements/invalid values and we ended up with a small dataset.

The dataset is regarding the following:

  • Tool Deflection : Metrology - Sensor Measured
  • Tool Wear : Metrology - Sensor Measured
  • Piece - Form Accuracy : Metrology - Sensor Measured
  • Piece - Roughness : Metrology - Sensor Measured
  • Machine Input - Parameters : System Inputs

I thought of generating more data for each metrology dataset, using the Machine Inputs as reference, however, I feel like it could potentially bias my model.

As you can see, I'm quite confused with respect to this subject.

  1. What should I expect if I move towards this solution?
  2. Is there any restriction I should be aware of?

It's as you said there is a chance of bias in the dataset. To avoid this you'll have to conform to an algorithm to generate data. Right off the bat, the solution I would propose is parametric methods. Find the statistical distribution of data and depending on that distribution fill in data accordingly.

If you want to know a bit more about parametric methods you can look at my answer here.

What does it mean for the training data to be generated by a probability distribution over datasets

For some machine learning methods look at this answer I posted here.

What are the best way to handle missing values

There is never one single correct way to do what you're asking, but there are good ways and it depends on the nuances of your data.

  • $\begingroup$ Yes, Absolutely. I know there is not a single approach and that is why I asked before moving even further. There are so many methods and I could potentially miss some details. Thanks for your help. $\endgroup$ Jan 11 '18 at 11:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.