The right-hand side was incorrectly cast to a pointer, since only signed
ints were being interpreted correctly as pointer arithmetic.

Fixes #20285.