I need to, given an indoor image of a room, to detect the materials of the floor and the wall (and maybe the roof too). I've started looking for open source repositories similar to this and found https://github.com/CSAILVision/unifiedparsing. Using that code I can identify materials for some objects like a sofa or a window but the problem is that (after some days of code debugging) I can't seem to extract the materials of the wall and the floor. Another problem of that code is that it takes a lot of time to run and that I don't really understand it in order to fix it.

So I'm thinking about building a classifier and I've been looking the ADE20k http://groups.csail.mit.edu/vision/datasets/ADE20K/ and the OpenSurfaces http://opensurfaces.cs.cornell.edu/ datasets. The problem is that I don't really know how to use them in order to accomplish my goal. I'm a Software Engineer and I've a basic idea of Deep Learning (I've applied ML for some projects) but it's difficult for me to establish a step by step plan to solve this.

I've been thinking about starting an online course on Deep Learning, but I need to solve this as soon as possible to show it (as a MVP) to a potential customer in order to validate the product and then invest more time improving it.

My first idea is the following one (let me know what you think):

Use this (https://github.com/CSAILVision/semantic-segmentation-pytorch) semantic segmentation library to extract the location of the floor and the walls.

The ade20k dataset has a .txt file related to each image containing each object on the image (the floor for example) and the attributes of the object (tile or wood for example). So I could get the location of the floor (step 1) and the material of the floor.

With the image-object-material I could train a deep learning algorithm to learn to classify new images.

I don't know about you, but it sounds weird to me (especially step 3). I think I'm pretty lost here and don't know how to go on.

What do you think? What would you do?

Your Answer


By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.