This effectively reverts 22690efcc2378222503cb8aaad26a6f4a539f5aa,
re-opening #11818.

This had the following problems:
 * Buggy on some targets (macOS, Windows)
 * Messy output to the terminal

These problems need to be solved before moving forward with this.