Previously we would use the address of the slice itself, which would
result in miscompilations and accidently setting the memory region
of the slice itself, rather than based on the `ptr` field.