Hello Zak,
Check/Do the following:
1. Make sure you have the latest Mellanox Driver/Firmware
2. Make sure you are able to reach line rate speed with "nd_write_bw.exe" which comes with Mellanox WinOF( http://www.mellanox.com/page/products_dyn?product_family=32&mtag=windows_sw_drivers )
3. Please consult our Performance tuning guide. http://www.mellanox.com/related-docs/prod_software/Performance_Tuning_Guide_for_Mellanox_Network_Adapters.pdf
4. Perform the Ntttcp between two server back-to-back(without SX6036 switch), What are your performance results?
Cheers,
~R