This makes rolling this out across extensions a far bit easier and also
safer, because we don't have to manually set `run_tests` for every
extension (and never have to consider this when updating these).

Release Notes:

- N/A