11 Replies Latest reply on Mar 6, 2015 3:22 AM by vince_intel

    Intel i350 NIC Teaming Question

    MattJH

      Hi

       

      I have a question surrounding the native NIC Teaming in Windows Server 2012 R2 with regards to the i350 Quad Port adapters.

       

      In our Hyper-V implementation, we have 3 x quad port NICs per Hyper-V Cluster node.

       

      With two of these adapters, we have balanced a Switch Independent / Dynamic (Sum of Queues mode) team across 4 of these ports for our VM Switch. Here's the thing... two of these ports are on one physical adapter and the other two ports are in the second physical adapter. Each port has VMQ enabled with suitable processor core allocations for a Sum of Queues team. VMQ is required as the team is connected to a VM Switch which is then supporting a reasonable number of Virtual Machines.

       

      The remaining 4 ports in these two adapters (2 in one card and 2 in the other) are then allocated to iSCSI MPIO use for access to our SAN. These remaining ports have RSS enabled.

       

      The questions I have are as follows:

       

      A) What is Intels Stance when mixing RSS and VMQ modes on different ports, on the same physical adapter?

       

      B) Is splitting a VMQ enabled NIC team between ports from physical NICs supported by Intel when using the native Windows NIC Teaming in Server 2012 R2?

       

      We are seeing intermittent latency issues and packet loss from within virtual machines with VMQ enabled. Disabling VMQ does work around the issue, however this isn't really a solution.

       

      Are there any known issues with these adapters when using Virtual Machine Queues?

       

      Kind Regards

       

      Matt

        • 1. Re: Intel i350 NIC Teaming Question
          MattJH

          No takers at all?

          • 2. Re: Intel i350 NIC Teaming Question
            MattJH

            Can I have an answer here please Intel? These cards were not cheap and we have purchased them specifically for their VMQ abilities. Being Intel cards, reliability is one thing that I would expect however I am disappointed thus far with both customer service and reliability.

             

            We are seeing constant issues here with Virtual Machine Queues. The four adapters in the team are configured correctly. RSS is disabled and VMQ is enabled. As this is a Switch Independent team in Dynamic mode, Sum of Queues mode is in operation, therefore we have been careful not to overlap processors for each team member.

             

            Even with the text book configuration, latest drivers and firmware, we still see problems with Virtual Machine Queues when they are enabled on the physical NICs and Microsoft Multiplex Team Adapter. Packet loss, high latency or complete loss of network connectivity for VMs is the order of the day. This mainly happens when a VM is migrated from host to host. Secondly, during a Live Migration, we see a longer than usual delay in network connectivity. Usually, we lose about a single ping during a Live Migration with VMQ turned off. With VMQ enabled, we see a good 5 or 6 lost ping requests before connectivity is restored. Clearly this isn't acceptable.

             

            Answers please....

            • 3. Re: Intel i350 NIC Teaming Question
              sage99

              Hi MattJH, have you tried using VMLB teaming mode of Intel on your setup? kindly refer to the site below for details.

               

              Network Connectivity — FAQs: VMDq, VLANs, and Teaming on Intel® Ethernet Adapters in Hyper-V*

               

              hope this help.

              • 4. Re: Intel i350 NIC Teaming Question
                MattJH

                That's not really a valid answer if I'm completely honest with you.

                 

                My questions were:

                 

                A) What is Intels Stance when mixing RSS and VMQ modes on different ports, on the same physical adapter?

                 

                B) Is splitting a VMQ enabled NIC team between ports from physical NICs supported by Intel when using the native Windows NIC Teaming in Server 2012 R2?

                 

                I guess now I should be asking whether or not these are adapters are even compatible at all with Windows NIC Teaming in 2012 R2???

                 

                Regards

                 

                Matt

                • 5. Re: Intel i350 NIC Teaming Question
                  vince_intel

                  Hi MattJH, i'm currently checking on your 2 questions related to VMQ and RSS. For the mean time, would like to clarify if i captured your issue and setup correctly.

                   

                  Issue:

                  Packet loss, high latency on VMs such as 5-6 lost pings request during live migration

                   

                  Setup:

                  3 I350-T4 installed on 1 system and configured as one of the Hyper-V. 2 out of 3 I350-T4 configured as Native Win2k12 Teaming (Switch Independent/Dynamic distribution). Total of 4 ports in 1 team. 2 ports belongs to I350-T4 #1 and 2 ports belongs to I350-T4 #2

                   

                  With VMQ enabled and processor core allocations. Remaining ports from I350-T4 1 and 2 were configured for ISCSI MPIO to access SAN with RSS enabled.

                   

                  Troubleshooting done:

                  Turned off VMQ

                   

                  Please share the output of the following

                  Get-NetAdapterVmq

                  Get-NetAdapterRss

                  • 6. Re: Intel i350 NIC Teaming Question
                    MattJH

                    Hi Vince

                     

                    Yes that's correct. With VMQ enabled in conjunction with the native Windows Server 2012 R2 NIC Teaming, we see intermittent packet loss, lengthy timeouts when Live Migrating VMs. With VMQ disabled, things seem to behave as expected.

                     

                    I can also confirm that out of the two quad port i350 adapters, two ports from each physical adapter are in a Switch Independent / Dynamic team (Sum-of-Queues mode) for the core VM Switch (4 interfaces), whilst the remaining 4 ports across the two physical adapters are ISCSI MPIO interfaces to the SAN. RSS is enabled on the ISCSI ports and VMQ is enabled on the VM Switch Team ports.

                     

                    We are also allocating a single CPU core to each of the VMQ enabled VM Switch Team interfaces (separate cores for each interface with no overlap). Hyper-Threading is enabled on the CPUs, however HT Logical execution units are not being allocated to VMQ or RSS, only actual cores.

                     

                    Here are the Get-NetAdapterRSS and Get-NetAdapterVMQ outputs as requested:

                     

                    Kind Regards

                     

                    Matt

                    • 7. Re: Intel i350 NIC Teaming Question
                      vince_intel

                      Hi MattJH,

                       

                      Please try the version 17.4 driver, it should help with your issue.

                       

                      https://downloadcenter.intel.com/Detail_Desc.aspx?DwnldID=21228

                       

                      Regards,

                      Vince

                      • 8. Re: Intel i350 NIC Teaming Question
                        HimanshuTank

                        Hey,


                        Can you help me with core allocation process. I have 4 pcie gen 2.0 slots. Each slot is having 2x 10G Flexi ports.

                        I am using E5-2680V2 (Ivy Bridge- EP) which has  10 cores and 2.8GHz.

                        I am familier with RSS but a little more description would be helpful.

                        Also want to mention we are using Linux.

                        Can i be able to implement VMQ on this platform in order to enhance the performance. I am not that much familier with VMQ.


                        Please help me with this situation ASAP.


                        Thanks,

                        Himanshu

                        connect: himanshutank75@gmail.com

                        skype: himanshu.tank3

                        • 9. Re: Intel i350 NIC Teaming Question
                          vince_intel

                          Hi Himanshu, RSS distributes the load of data processing to multiple CPUs, please share your linux platform and kernel version, also the model of Intel network adapter involved.

                           

                          For VMQ, this is being implemented on Virtualized environment.

                           

                          regards,

                          vince

                          • 10. Re: Intel i350 NIC Teaming Question
                            HimanshuTank

                            Can you get connected with me on skype or hangouts. It would be much better and fast to resolve the issue. As it is on high priority.

                             

                            Skype: himanshu.tank3

                            himanshutank75@gmail.com

                             

                            Thanks,

                            Himanshu

                            • 11. Re: Intel i350 NIC Teaming Question
                              vince_intel

                              Hi Himanshu, don't have skype or hangout apps on my system, for faster resolution, suggest to coordinate with your system manufacturer or Linux platform as they might have sample case studies / best practices using similar setup.

                               

                              regards,

                              Vince