6 Replies Latest reply on Apr 30, 2013 4:34 PM by Patrick_Kutch

    82599 VF to Linux host bridge

    jhaller1

      We are configuring VFs for our high-volume network traffic, but want to continue to use Linux bridging for other VMs. Is there some configuration we need to do to enable packets from a VF to be switched to a bridge off the eth0 PF?

       

      We create several VMs. One of them has a high volume of traffic, as it serves as a load balancer. It takes external traffic on one VLAN, and distributes it to other VMs on a different VM. If the different VM is on the same host, the packets are never switched in the 82599 back to the eth0 for feeding into the Linux bridge. We are using Red Hat EL 6.4. We have been able to get VM to VM traffic working on the same host, if both VMs use VFs, but are unable to get VM to VM traffic if one is using VF and the other is using bridges.

       

      Suggestions?

        • 1. Re: 82599 VF to Linux host bridge
          Patrick_Kutch

          Hi John,

          Thanx for posting to our site.

           

          You are experiencing this issue because the VF's are connected to a kind of small, light-weight 'switch' within the Intel(r) 82599.  When the Hypervisor creates a VF and a MAC address is assigned to it (either automatically or manually) this 'switch' is notified via the driver.

           

          With this internal switch, VF's can talk to VF's because when one VF transmits a packet and the destination MAC address matches another VF the 'switch' will automatically send it to the VF rather than out to the wire.

           

          However VM's and bridges that are using emulated ethernet devices do not have their MAC addresses added to this 'switch'.  That is the source of your issue.

           

          I'm doing some digging - I believe there have been, or is soon to be, some updates that will allow this kind of traffic to be passed between a VF and an emulated path.

           

          - Patrick

          1 of 1 people found this helpful
          • 2. Re: 82599 VF to Linux host bridge
            Patrick_Kutch

            Found some more data:

             

             

             

            The functionality you are after may not be included in RHEL 6.4.   Our development team is not
            sure which patches from upstream kernel RedHat team has back ported to RHEL 6.4 kernel. 

             

            This functionality is included in upstream kernel.  I’d suggest updating your kernel to the 3.8.5
            available at kernel.org.  You  will also need to update IPROUTE2 utility to latest version.  You  will need to add the VF and PF mac addresses to bridge forwarding database using IPROUTE2 utility’s bridge
            commands.  

             

            Example steps below:

            • Create 10 VF on eth0 (assuming eth0 is a Intel 82599port).  Each VF all will have a mac
              address. You can find the VF MAC address by executing “ip link showeth0”. 
            • Create a Linux bridge using brctl  command. 
            • Add eth0 interface to the newly created bridge using “brctl addif switchname eth0”
            • Use “bridge fdb add” command to add VF mac addresses and eth0 mac address to bridge forwarding
              database.

              Latest ixgbe & ixgbevf drivers are included in stable 3.8.5 upstream kernel. 

             

            Hope this helps!

             

            - Patrick

            • 3. Re: 82599 VF to Linux host bridge
              jhaller1

              Thanks for the information, Patrick. We will try just updating the drivers and iproute2 first, and try updating the kernel second.

              • 4. Re: 82599 VF to Linux host bridge
                Patrick_Kutch

                Sure thing.  If you can come back and let us know how it went, would be appreciated.

                 

                - Patrick

                • 5. Re: 82599 VF to Linux host bridge
                  jhaller1

                  We verified that the proper support is not in Red Hat EL 6.4. We were able to get a newer version of the driver, which helped.

                  • 6. Re: 82599 VF to Linux host bridge
                    Patrick_Kutch

                    Great!  I love success stories.  Thanx for posting the update!

                     

                    -  Patrick