I’ve been following the development of the next Stable Diffusion model, and I’ve seen this approach mentioned.
Seems like this is a way in which AI training is analogous to human learning - we learn quite a lot from fiction, games, simulations and apply this to the real world. I’m sure the same pitfalls apply as well.
Synthetic data was used here with impressive results: https://programming.dev/post/133153
There is a lot of potential in this approach, but the idea of using it for training AI systems in MRI/CT/etc. diagnostic methods, as mentioned in the article, is a bit scary to me.