Hi,
Even if you have two 40GbE ports. each port can reach close to 40GbE, but when both of them pass traffic you are limited to the PCIe capacity with is ~54-55Gb/s.
in addition, try to use high MTU side if possible.
Are you aware of the performance tuning guide for Mellanox adapters?
Thanks,
Ophir.