ibdump unable to capture MPI traffic on MLNX ConnectX-3 FDR HCAs

Hi all,

I have been trying to to capture MPI traffic using ibdump for one way latency measurements (at the physical layer).

But all ibdump captures are IPoIB packets interleaved with Subnet monitoring packets such as perfqueries from the UFM/SM sweeps.

Specs:

ibdump, 2.0.0-5, built on Aug 21 2013, 13:09:58. GIT Version: 41a4dd2

I have also tried: ibdump, 2.0.0-8, built on Apr 28 2014, 17:06:51. GIT Version: f8e989d

I have also used --mem-mode and --max-burst to prevent packet dropping but with no luck.

Ofed_info: MLNX_OFED_LINUX-2.0-3.0.0.3 (OFED-2.0-3.0.0)

bash-4.1$ ibv_devinfo

hca_id:    mlx4_0

    transport:            InfiniBand (0)

    fw_ver:                2.30.3200

    node_guid:            0002:c903:0044:e220

    sys_image_guid:            0002:c903:0044:e223

    vendor_id:            0x02c9

    vendor_part_id:            4099

    hw_ver:                0x1

    board_id:            IBM1100110019

    phys_port_cnt:            1

    port:    1

    state:            PORT_ACTIVE (4)

    max_mtu:        4096 (5)

    active_mtu:        4096 (5)

    sm_lid:            1

    port_lid:        15

    port_lmc:        0x00

    link_layer:        InfiniBand

Using tshark -r sniffer.pcap| grep -i infiniband to parse the .pcap files, I typically get this:

cat /etc/modprobe.d/mlnx.conf:

blacklist mlx4_core

blacklist mlx4_en

blacklist mlx5_core

blacklist mlx5_ib

I would highly appreciate anyone who can help understand this behavior.

Thanks,

Fabrice