14 Replies Latest reply on Mar 6, 2018 10:59 PM by Intel Corporation

    X710 strips incoming vlan tag with SRIOV

    slitkowski

      Hi,

       

      I have a VM using SRIOV interfaces based on X710 interfaces. This VM is connected to a physical router through the VF.

      I would like to enable vlan tagging on the VM.

       

      I can ping the physical router from the VM if I'm not using VLAN tagging on the interface.

      When I add VLAN tagging on the interface (on the VM and on the physical router), the network connectivity is broken.

       

      The physical router correctly receives the packets/frames from the VM.

      The VM receives the packets/frames but without a VLAN tag, it seems that the VLAN tag has been stripped.

       

      Here are some informations on my setup:

       

      [root@SLI-SRV3 sysadmin]# ethtool -i eth1

      driver: i40e

      version: 1.6.27-k

      firmware-version: 6.00 0x800034e6 18.3.6

      expansion-rom-version:

      bus-info: 0000:19:00.0

      supports-statistics: yes

      supports-test: yes

      supports-eeprom-access: yes

      supports-register-dump: yes

      supports-priv-flags: yes

       

      [root@SLI-SRV3 sysadmin]# ip link show eth1

      3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq portid 246e967eaabe state UP mode DEFAULT qlen 1000

          link/ether 24:6e:96:7e:aa:be brd ff:ff:ff:ff:ff:ff

          vf 0 MAC 52:54:00:5d:6a:60, spoof checking on, link-state auto, trust off

       

       

      VIRSH network config:

          <interface type='hostdev' managed='yes'>

            <mac address='52:54:00:5d:6a:60'/>

            <driver name='vfio'/>

            <source>

              <address type='pci' domain='0x0000' bus='0x19' slot='0x02' function='0x0'/>

            </source>

            <alias name='hostdev0'/>

            <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>

          </interface>

       

       

      Any idea about what could be the issue ?

       

      Thanks,

       

      Stephane

        • 1. Re: X710 strips incoming vlan tag with SRIOV
          Intel Corporation
          This message was posted on behalf of Intel Corporation

          Hi Slitkowski,

           Thank you for posting in Wired Communities. What is the exact Linux operating system used here? Please try run the SSU utility and send me the log file.
          https://downloadcenter.intel.com/download/26735/Intel-System-Support-Utility-for-the-Linux-Operating-System

          Thanks,
          Sharon T
           

          • 2. Re: X710 strips incoming vlan tag with SRIOV
            slitkowski

            I'm running a RHEL 7.4.

             

            # uname -a

            Linux SLI-SRV3 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 13 10:46:25 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux

             

             

            SSU result is attached.

             

            Thanks

            • 3. Re: X710 strips incoming vlan tag with SRIOV
              Intel Corporation
              This message was posted on behalf of Intel Corporation

              Hi Slitkowski,

               Thank you for the information. Let me further investigate on this.

              Regards,
              Sharon T 

              • 4. Re: X710 strips incoming vlan tag with SRIOV
                Intel Corporation
                This message was posted on behalf of Intel Corporation

                Hi Slitkowski,

                  Please be informed I am still investigating on this, will update the post once there is any finding.

                Thanks,
                Sharon T 

                • 5. Re: X710 strips incoming vlan tag with SRIOV
                  Intel Corporation
                  This message was posted on behalf of Intel Corporation

                  Hi Slitkowski,

                   

                      Please try check on the information regarding VLAN tag stripping in our READMe file at https://downloadmirror.intel.com/24693/eng/readme.txt (extracted here for your reference). Try turning off the rxvlan and check if it works.

                   

                  Setting VLAN Tag Stripping
                  --------------------------

                   

                  If you have applications that require Virtual Functions (VFs) to receive
                  packets with VLAN tags, you can disable VLAN tag stripping for the VF. The
                  Physical Function (PF) processes requests issued from the VF to enable or
                  disable VLAN tag stripping. Note that if the PF has assigned a VLAN to a VF,
                  then requests from that VF to set VLAN tag stripping will be ignored.

                   

                  To enable/disable VLAN tag stripping for a VF, issue the following command
                  from inside the VM in which you are running the VF:
                    ethtool -K <if_name> rxvlan on/off
                    or alternatively:
                    ethtool --offload <if_name> rxvlan on/off

                   

                       The MAC address shown is from Dell, can you confirm if the NIC is an embedded NIC on the system? or a standalone NIC comes with Dell system?

                   

                     Please feel free to update me.

                   

                  Thanks,
                  Sharon T

                  • 6. Re: X710 strips incoming vlan tag with SRIOV
                    slitkowski

                    I tried, but it does work. VLAN is still stripped:

                     

                    [root@SLI-SRV3 sysadmin]# ethtool -k eno1

                    Features for eno1:

                    rx-checksumming: on

                    tx-checksumming: on

                            tx-checksum-ipv4: on

                            tx-checksum-ip-generic: off [fixed]

                            tx-checksum-ipv6: on

                            tx-checksum-fcoe-crc: off [fixed]

                            tx-checksum-sctp: on

                    scatter-gather: on

                            tx-scatter-gather: on

                            tx-scatter-gather-fraglist: off [fixed]

                    tcp-segmentation-offload: on

                            tx-tcp-segmentation: on

                            tx-tcp-ecn-segmentation: on

                            tx-tcp6-segmentation: on

                            tx-tcp-mangleid-segmentation: off

                    udp-fragmentation-offload: off [fixed]

                    generic-segmentation-offload: on

                    generic-receive-offload: on

                    large-receive-offload: off [fixed]

                    rx-vlan-offload: off

                    tx-vlan-offload: on

                    ntuple-filters: on

                    receive-hashing: on

                    highdma: on

                    rx-vlan-filter: on [fixed]

                    vlan-challenged: off [fixed]

                    tx-lockless: off [fixed]

                    netns-local: off [fixed]

                    tx-gso-robust: off [fixed]

                    tx-fcoe-segmentation: off [fixed]

                    tx-gre-segmentation: on

                    tx-ipip-segmentation: on

                    tx-sit-segmentation: on

                    tx-udp_tnl-segmentation: on

                    tx-mpls-segmentation: off [fixed]

                    fcoe-mtu: off [fixed]

                    tx-nocache-copy: off

                    loopback: off [fixed]

                    rx-fcs: off [fixed]

                    rx-all: off [fixed]

                    tx-vlan-stag-hw-insert: off [fixed]

                    rx-vlan-stag-hw-parse: off [fixed]

                    rx-vlan-stag-filter: off [fixed]

                    busy-poll: off [fixed]

                    tx-gre-csum-segmentation: on

                    tx-udp_tnl-csum-segmentation: on

                    tx-gso-partial: on

                    tx-sctp-segmentation: off [fixed]

                    l2-fwd-offload: off [fixed]

                    hw-tc-offload: off [fixed]

                     

                    11:43:12.537863  In 10:f3:11:1c:f8:ff > 52:54:00:49:82:40, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 255, id 0, offset 0, flags [none], proto: ICMP (1), length: 100) 66.0.0.5 > 66.0.0.6: ICMP echo request, id 61806, seq 0, length 80

                    11:43:14.539047  In 10:f3:11:1c:f8:ff > 52:54:00:49:82:40, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 255, id 1, offset 0, flags [none], proto: ICMP (1), length: 100) 66.0.0.5 > 66.0.0.6: ICMP echo request, id 61806, seq 1, length 80

                    11:43:16.540645  In 10:f3:11:1c:f8:ff > 52:54:00:49:82:40, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 255, id 2, offset 0, flags [none], proto: ICMP (1), length: 100) 66.0.0.5 > 66.0.0.6: ICMP echo request, id 61806, seq 2, length 80

                    11:43:18.542226  In 10:f3:11:1c:f8:ff > 52:54:00:49:82:40, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 255, id 3, offset 0, flags [none], proto: ICMP (1), length: 100) 66.0.0.5 > 66.0.0.6: ICMP echo request, id 61806, seq 3, length 80

                    11:43:20.544015  In 10:f3:11:1c:f8:ff > 52:54:00:49:82:40, ethertype IPv4 (0x0800), length 114: (tos 0x0, ttl 255, id 4, offset 0, flags [none], proto: ICMP (1), length: 100) 66.0.0.5 > 66.0.0.6: ICMP echo request, id 61806, seq 4, length 80

                     

                     

                    I have a PCIE X710 Dual port 10G card and also a combo i350 DP+X710 DP as a LOM card.

                    Both have the same issue.

                    • 7. Re: X710 strips incoming vlan tag with SRIOV
                      slitkowski

                      I found a way to get the VLAN tag.

                      I need to set a VLAN on the VF and remove it:

                       

                      [root@SLI-SRV3 images]# ip link set eno1 vf 1 vlan 10

                      [root@SLI-SRV3 images]# ip link set eno1 vf 1 vlan 0

                      [root@SLI-SRV3 images]#

                       

                      15:18:26.214811  In 10:f3:11:1c:f8:ff > 52:54:00:49:82:40, ethertype 802.1Q (0x8100), length 118: vlan 100, p 0, ethertype IPv4, 66.0.0.5 > 66.0.0.6: ICMP echo request, id 24942, seq 27, length 80

                      15:18:28.216687  In 10:f3:11:1c:f8:ff > 52:54:00:49:82:40, ethertype 802.1Q (0x8100), length 118: vlan 100, p 0, ethertype IPv4, 66.0.0.5 > 66.0.0.6: ICMP echo request, id 24942, seq 28, length 80

                      15:18:30.218106  In 10:f3:11:1c:f8:ff > 52:54:00:49:82:40, ethertype 802.1Q (0x8100), length 118: vlan 100, p 0, ethertype IPv4, 66.0.0.5 > 66.0.0.6: ICMP echo request, id 24942, seq 29, length 80

                       

                      It looks like a bug in the driver.

                      • 8. Re: X710 strips incoming vlan tag with SRIOV
                        Intel Corporation
                        This message was posted on behalf of Intel Corporation

                        Hi Slitkowski,

                                Thank you for the update and sharing the information, I am glad the issue is fixed as well. Just to double check for the i40e driver,  where did you download the driver from? Thank you.

                        Regards,
                        Sharon T 

                        • 9. Re: X710 strips incoming vlan tag with SRIOV
                          slitkowski

                          I consider this as a workaround rather than a fix.

                          There is no reason to strip the VLAN by default as the "ip link set eno1 vf 1 vlan 0" normally sets back the vf to its default vlan config.

                           

                          The driver is the one provided by Redhat. I did not change it.

                          • 10. Re: X710 strips incoming vlan tag with SRIOV
                            Intel Corporation
                            This message was posted on behalf of Intel Corporation

                            Hi Slitkowski,

                             

                            Thank you for the information. Let me check in case there is information to share.

                             

                            Regards,
                            Sharon T

                            • 11. Re: X710 strips incoming vlan tag with SRIOV
                              Intel Corporation
                              This message was posted on behalf of Intel Corporation

                              Hi Slitkowski,

                                 I have forwarded your feedback to our engineering team. Just would like to double check
                              if there is switch between the router and the X710 NIC?

                              Regards

                              Sharon T 

                              • 12. Re: X710 strips incoming vlan tag with SRIOV
                                slitkowski

                                No there is no switch, my router that initiates the PING and the server which hosts the VM are connected using a back to back cable.

                                • 13. Re: X710 strips incoming vlan tag with SRIOV
                                  Intel Corporation
                                  This message was posted on behalf of Intel Corporation

                                  Hi Slitkowski,

                                   Thank you for the confirmation.

                                  Regards,
                                  Sharon T 

                                  • 14. Re: X710 strips incoming vlan tag with SRIOV
                                    Intel Corporation
                                    This message was posted on behalf of Intel Corporation

                                    Hi Slitkowski,

                                         Please be informed that we already share your information with our engineering team and they will determine if a driver update is necessary. 

                                    Regards,
                                    Sharon T