We finally have a cancellation mechanism to use! Made the non
side-effectful handlers stop their work if we get a cancel request
notification.

Release Notes:

- N/A