Just found something out by accident.
I've logged into my 9024D-M switch as root, and ifconfig is showing the default IP address of the ipoib0 interface as "192.168.2.100".
If you configure one of your Infiniband cards with IPoIB, and put it on the 192.168.2.x subnet, you might be able to get to 192.168.2.100 through it.