Proposal-based:
- Let's look for a car, define its boundary etc. Okay, found a car.
- Cluster all the pixels belonging to that car.
Proposal-free:
- Let's label each pixel as some uncategorized instance.
- Based on the results from semantic segmentation, that instance probably belongs to the "Car" category.
You can also check the II. Related Work in the paper below where it is explained in better detail, with some additional sources mentioned in it:
Hsu, Yen-Chang, et al. "Learning to cluster for proposal-free instance segmentation." 2018 International Joint Conference on Neural Networks (IJCNN). IEEE, 2018.