Does this ethtool output for ib0 in IPoIB mode look correct?

I have set up four servers recently for testing:

• Switch: SB7700
• HCA: MCX455A-ECAT ConnectX-4 VPI adapter, EDR IB and 100GbE, single-port QSFP28
• Mellanox cables

[root@fs00 ~]# cat /etc/redhat-release CentOS Linux release 7.2.1511 (Core) 
[root@fs00 ~]# uname -a Linux fs00 3.10.0-327.22.2.el7.x86_64 #1 SMP Thu Jun 23 17:05:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux 

• CPU: 2 x Intel E2620v3 2.4Ghz
• RAM: 8 x 16 GiB DDR4 2133Mhz DIMMs => 128 GiB

The following is a typical ip a output on a server.  The sole IB interface ib0 is also configured with IPoIB. It is assigned an IPv4 address in the same subnet as another Ethernet port, enp129s0f0, which is a regular 10G LOM port. The rest three Ethernet interfaces are kept down - we don't need them. The 10G is connected to the bastion host. The server is meant to provide storage services over IB, which the ib0 is for.

[root@fs00 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
  link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
  inet 127.0.0.1/8 scope host lo
  valid_lft forever preferred_lft forever
  inet6 ::1/128 scope host
  valid_lft forever preferred_lft forever
2: enp15s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
  link/ether 00:15:b2:a8:d2:cb brd ff:ff:ff:ff:ff:ff
3: enp129s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP qlen 1000
  link/ether 00:15:b2:a8:d2:c8 brd ff:ff:ff:ff:ff:ff
  inet 192.168.11.200/24 brd 192.168.11.255 scope global enp129s0f0
  valid_lft forever preferred_lft forever
  inet6 fe80::215:b2ff:fea8:d2c8/64 scope link
  valid_lft forever preferred_lft forever
4: enp0s25: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
  link/ether 00:15:b2:a8:d2:ca brd ff:ff:ff:ff:ff:ff
5: enp129s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
  link/ether 00:15:b2:a8:d2:c9 brd ff:ff:ff:ff:ff:ff
6: ib0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 2044 qdisc pfifo_fast state UP qlen 1024
  link/infiniband a0:00:00:27:fe:80:00:00:00:00:00:00:7c:fe:90:03:00:29:28:8e brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
  inet 192.168.11.150/24 brd 192.168.11.255 scope global ib0
  valid_lft forever preferred_lft forever
  inet6 fe80::7efe:9003:29:288e/64 scope link
  valid_lft forever preferred_lft forever

When I reviewed the ib0 using ethtool, I saw the following. Despite the Speed entry is listed at 100000Mb/s, when running e.g. iperf3 over ib0, the throughput tops out around 8 - 9 Gbps. Three questions:

1. Does the ethtool output look correct?
2. Is the iperf3 throughput limited by the "Supported link modes"? 
3. Is there a way to get ib0 to support speed higher than 40Gbps?  It's connected to SB7700, which is fully capable of 100Gbps?

[root@fs00 ~]# ethtool ib0

Settings for ib0:

  Supported ports: [ ]

  Supported link modes:   1000baseT/Full

                         1000baseKX/Full

                         10000baseT/Full

                         10000baseKX4/Full

                         10000baseKR/Full

                         40000baseKR4/Full

                         40000baseCR4/Full

                         40000baseSR4/Full

                         40000baseLR4/Full

  Supported pause frame use: No

  Supports auto-negotiation: No

  Advertised link modes:  1000baseT/Full

                         1000baseKX/Full

                         10000baseT/Full

                         10000baseKX4/Full

                         10000baseKR/Full

                         40000baseKR4/Full

                         40000baseCR4/Full

                         40000baseSR4/Full

                         40000baseLR4/Full

  Advertised pause frame use: No

  Advertised auto-negotiation: No

  Speed: 100000Mb/s

  Duplex: Full

  Port: Other

  PHYAD: 1

  Transceiver: internal

  Auto-negotiation: on

  Link detected: yes