I did have to offset it by the Ethernet header. The typecast ended up looking like this:
pIPHeader := TIPHeaderPtr (@Buffer.m_IBuffer[15]);
