Use an unsigned pointer compare, as the highest bit may sometimes be set.

This fixes intermittent connection issues on 32-bit Linux. Also related
to issues #2557 and #2600.