The code assumed that every ?T had a pointer child type T, add some more
checks to make sure the type is effectively a pointer.

Closes #4789