9
$\begingroup$

I want a GAN to train on "before and after" images of dental surgeries; so that it can generate "after" pictures for fresh patients.

Input images are like these: https://img.webmd.com/dtmcms/live/webmd/consumer_assets/site_images/article_thumbnails/slideshows/cosmetic_dentistry_slideshow/650x350_cosmetic_dentistry_slideshow.jpg

Now being a web developer, I am getting into AI.

I have read a hell lot on Basic ML, AI, GANs, VAE, StyleGANs, etc.

Made small projects on Tensorflow, etc.

I have my basics and theoretical knowledge in place. But I dont know how to get started practically on the above idea I have.

What should be my approach?

Following are my questions:

How many "before and after" images will I need?

What architecture should I use? i.e. StyleGAN, VAE?

As I will need my custom model; training a fresh model would need a good GPU. What platform should I go for? Colab, Floyhub, or something else

One approach that comes to my mind is that - Process the image, do the color segmentation or something.

Add a static tooth image to area where tooth is missing. Then feed this image to a GAN, so that it can polish that image to look real.

Please guide me in the right direction. Sorry if any of my question is nonsense.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
5
$\begingroup$

It's a very specific problem and there's no right or wrong solution. I'll just write what I'd do in your position and hope that it is useful.

How many "before and after" images will I need?

You'll need a lot of images to consistently get good results, in the range of tens or preferably hundreds of thousands.

What architecture should I use? i.e. StyleGAN, VAE?

StyleGAN seems to be the most popular option for these sorts of tasks these days, so I'd stick with this.

As I will need my custom model; training a fresh model would need a good GPU. What platform should I go for? Colab, Floyhub, or something else

The platform you'll choose depends on whether or not you're willing to pay. Colab, Floydhub and Kaggle are the only free options with GPU afaik, but also offer paid versions (1, 2). All have their pros and cons. For example colab, which I'm more familiar with, offers only 12-hour sessions (after that it disconnects you and you have to reconnect).

Approach

Regarding the process you said, that's an interesting approach. I would have gone with something more basic like a segmentation mask, but I think your idea is better.

Improve this answer
$\endgroup$
3
  • $\begingroup$ hey Djib thanks for your answer! i m sorry if i sound noobish.. when i go to stylegan page..it just shows me they turned horses to zebra.. i dont know how to get started?.. if u know any open source project that does what I want do, that wud be really helpful $\endgroup$
    – Lakshay Dulani
    Mar 21, 2020 at 10:37
  • 1
    $\begingroup$ I've never used this particular architecture before. There seems to be an official implementation in TensorFlow. From a first look it seems to be a mess. I found another unofficial implementation which seems to be less complex. I'd suggest getting familiar with TensorFlow and then starting with this implementation to see how it works exactly. $\endgroup$
    – Djib2011
    Mar 21, 2020 at 11:46
  • $\begingroup$ you're welcome! $\endgroup$
    – Djib2011
    Mar 21, 2020 at 12:13

Not the answer you're looking for? Browse other questions tagged or ask your own question.