0
$\begingroup$

I'd like to apply some machine learning on 3D CAD data. File format should ideally be mesh-based like STL. Keras offers 3D convolutional layers (https://keras.io/layers/convolutional/), so it can handle 3D (5D) input data. The problem I have is the preprocessing of the input data. There seem to exist integrated methods in Keras for image preprocessing (https://keras.io/preprocessing/image/). However that does only seem to apply to 2D images.

I plan to voxelize the STL files somehow, and would appreciate input on the process of voxelization and preprocessing the data to be suitable for Keras.

PS: I know there are CNNs which directly take mesh data. However I'd like to approach the voxelation way to compare performance and results.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

I found a way that works now.

Keras Conv3D can be fed with a numpy array of the voxelization and the corresponding class of the file.

So first the files have to be voxelized with a tool like binvox.

Afterwards those files can be read into a numpy array with binvox-rw-py:

np.int32(binvox_rw.read_as_3d_array(f).data)

In combination with the classification, the files can be used as input for a Conv3D layer in Keras.

A complete example can be found here.

Improve this answer
$\endgroup$
0
$\begingroup$

Voxelization will give you rather sparse data. I'd rather discretize mesh by volume, which will give you cloud of points. This way you'll preserve volume information and make it less sparse.

Improve this answer
$\endgroup$
9
  • $\begingroup$ Can you elaborate more on that? Why is a voxel grid more sparse than a point cloud? The problem of converting mesh files to voxels or point cloud and more importantly preprocessing that so it is suitable as an input for Keras still is not answered. I hope, you can also give advise on that. $\endgroup$
    – clel
    Commented Nov 18, 2019 at 12:56
  • $\begingroup$ Voxels are surface only. Basically you label points as: 1 (inside object), -1 (outside object). Labeling negative points as 0, will kill your weights and you'll encounter vanishing gradient. Downside of this method is: you're gonna need some arbitrary coordinate system, if you want to know the difference between smaller and larger objects. On the other hand you can tune discretization resolution to your needs. $\endgroup$
    – Piotr Rarus
    Commented Nov 18, 2019 at 13:10
  • $\begingroup$ Well, the STL format is surface only. Since voxels also "label" points on the inside as inside, not only on the surface, it should not be surface only, but the result will be a "solid" object. I understand the problem of differenciating the object size. So thanks for the hints! Still, can you also answer my question, how to achieve all of this? Currently we are only discussing, which method works best. $\endgroup$
    – clel
    Commented Nov 18, 2019 at 15:09
  • $\begingroup$ You need to create 3D grid, let's say 1000x1000x1000. Then you iterate over each point and check, whether it's inside the object. It's rather computationally expensive, but it should suit nicely for NNs. Just shoot a ray in random direction and check number of intersections. If it's odd, you're inside. $\endgroup$
    – Piotr Rarus
    Commented Nov 18, 2019 at 15:16
  • $\begingroup$ That is a bit abstract. I also know programs to convert STL to voxel data. For your example: How do I create that grid, how do I check whether a point is inside the object? The question is, what does keras expect (numpy array?) and how to preprocess the data so it fits this expectation. So how to convert many files to that, save them and load them into Keras? $\endgroup$
    – clel
    Commented Nov 19, 2019 at 14:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.