9 Replies Latest reply on Feb 4, 2017 6:23 PM by brodie7838 Branched to a new discussion.

    i219-V Stability Issues when Link Speed is negotiated above 10M-FDx

    brodie7838

      Greetings,

      A recently acquired motherboard has an on-board Intel i219-V NIC. Within a fresh Win7 Pro SP1 x64 install with latest Intel drivers (driver v12.15.23.1, PROSet v21.0.504.0), I am experiencing stability issues with this NIC when a specific combination of factors are met:

       

      1. NIC is set to any Link Speed above 10Mbps Full Duplex (100M-Full, 1G-Full, & Auto Negotiate)

       

      2. Steam (the application from Valve) is downloading application and/or game updates

       

      Basically once Steam starts its updates and reaches about 1MB/s transfer rate, the link drops and the whole computer loses its network connection (LAN and WAN) - all LAN ARP entries are flushed, cannot ping local gateway router, Windows shows the little yellow triangle over the network icon (link but no L2/L3, self-assigned 169 address)

       

      Once this issue occurs, it is resolvable in only two ways:

       

      - Reboot the computer

       

      - Change the Link Speed settings under the NIC's Device Properties page in Device Manager. It doesn't matter what I set it to, just the action of re-setting this parameter is enough to re-negotiate the link again, it seems.

       

      The only workaround that I've found so far is to manually set the NIC Link speed to 10Mbps Full - the NIC is stable (albeit much slower) in this configuration.

       

      Other Troubleshooting & Notes:

       

      - No other application or traffic type seems to result in this behavior. For example, speed tests, iperf tests, torrent, etc all max out my WAN (not LAN) with ease, without the NIC dropping the link

       

      - Replaced relevant CAT6 patch cables (all test good as well)

       

      - Tested different port on LAN switch (NIC is connected directly to an enterprise HP ProCurve switch)

       

      - Tested with switchport speed/duplex settings always matching NIC Link Speed settings (100M-Full, 10M-Full, etc)

       

      Edit: I tested with a USB3.0 Ethernet adapter, and Steam has no issues maxing out the ISP without the link dropping. All other equipment is the same and configured the same.

        • 1. Re: i219-V Stability Issues when Link Speed is negotiated above 10M-FDx
          Intel Corporation
          This message was posted on behalf of Intel Corporation

          Hi Brodie7838,

           

              Thank you for bringing this to our attention. As this is an onboard NIC, have you tried
          installing the driver provided by your board vendor. The reason we refer you to contact them is they might have driver customized by the board vendor which is more suitable for the onboard NIC. You may refer to URL below for support on onboard NIC
          http://www.intel.com/content/www/us/en/support/network-and-i-o/ethernet-products/000006628.html?
          wapkw=oem+network

           

          As this happened on you Steam application, have you also try check with them if there is certain requirement for the network connection?


          Thanks,
          wb

          • 2. Re: i219-V Stability Issues when Link Speed is negotiated above 10M-FDx
            brodie7838

            Hi, yes I originally started out with the latest available drivers as provided by the manufacturer (ASUS), and updated to the Intel generic drivers as part of troubleshooting. I'll be opening a ticket with Valve as well.

            • 3. Re: i219-V Stability Issues when Link Speed is negotiated above 10M-FDx
              Intel Corporation
              This message was posted on behalf of Intel Corporation

              Hi Brodie7838,


                  Good day. Just wondering if any assistance is needed ? Please feel free to update me. 

              Thanks
              wb
               

              • 4. Re: i219-V Stability Issues when Link Speed is negotiated above 10M-FDx
                brodie7838

                Hi wb,

                Thanks, but the issue still persists. I'm not sure that a resolution really exists at this point, so I will continue to use a USB 3.0 adapter. It may even just be isolated to my motherboard, but I have no way to prove that without getting a second board and NIC to test with.

                 

                If anyone has this NIC: download Steam and try to launch the installer and see what happens, then post here pretty please

                • 5. Re: i219-V Stability Issues when Link Speed is negotiated above 10M-FDx
                  Intel Corporation
                  This message was posted on behalf of Intel Corporation

                  Hi Brodie7838,

                  Thanks for the clarification.

                  rgds,
                  wb
                   

                  • 6. Re: i219-V Stability Issues when Link Speed is negotiated above 10M-FDx
                    brodie7838

                    As you can imagine, telling a software vendor like Valve that their application is causing my hardware NIC to crash went over like a lead balloon - because this is so clearly a hardware issue, there's really nothing for them to do. I have to agree with them too - it's not Valve's fault this Intel NIC can't handle the traffic (pretty sure it's just HTTP/HTTPS) that otherwise works on every other onboard, USB or PCIE NIC that I've tested; this is clearly a fault in Intel's code/hardware and nothing else.

                     

                    I find it frustrating that Intel is not taking a more active troubleshooting approach here - I have done most of the troubleshooting that would be required for someone else to test this scenario in a lab; considering Steam is free software there's no reason someone at Intel can't take a few minutes to simply try and re-create the reported issue. I guess "end users" don't count.

                    • 7. Re: i219-V Stability Issues when Link Speed is negotiated above 10M-FDx
                      ProspectTheorist

                      I believe I have a solution to your problem.  The symptoms you experienced are a subset of the ones I experienced.

                       

                      My Solution

                      • I solved it by allocating the maximum number of Receive and Transmit Buffers, which happens to be 2048 for both settings.
                      • I have not investigated what the lowest usable value for each setting is in my case, but I suspect it is not necessary to use the maximum.
                      • My hypothesis is that:
                          1. the default value of 256 Receive Buffers is too low
                          2. OR the default value of 512 Transmit Buffers is too low
                          3. OR both default values are too low.
                        • The above circumstances result in a buffer overflow of some sort, which causes the dropped connection at the local interface.

                       

                      For completeness, I have described my setup below:

                      My NIC is an onboard dual interface card on the ASRock Z170 Extreme7+ motherboard, which consists of one Intel I211, and one Intel I219-V.  I am using the "Intel_LAN(v21.1_v1)" drivers from the ASRock website, running fully updated Windows 7 SP1 Ultimate x64.

                       

                      The NIC interfaces are connected via fully tested and rated cat5e to a Ubiquiti US-24-250W Managed Switch, which is connected to a Ubiquiti USG‑PRO‑4 Gateway/Router.

                       

                      Whenever the link speed is set higher than 10Mbps Full Duplex on either of the interfaces, I have experienced the following:

                      1. Any steam download successfully receives a burst of 4.9 MB, upon which all connections are dropped on that interface, and no new connections can be made.
                        • If I throttle the steam download to 64 kB/s, the download can continue for a few minutes before experiencing the dropped connection.  I did not check if the same 4.9 MB threshold was reached before the drop, but I suspect that is the case.
                        • My suspicion about the 4.9 MB threshold above stems from my inspection of netmon 3.4 logs during the event, in which it appears the Steam download server sends 2-4 DATA packets before an ACK packet is sent by the Steam client.  The dropped connection always happens immediately following a stream of 10+ contiguous DATA packets from the Steam download server with no intervening ACKs from the Steam client.  The Steam client then begins sending requests for retransmission of the previous packets from the Steam download server, followed by a cease of all network activity.
                        • Something in the above behavior is causing our NIC to freak out and stop functioning.  I do not understand the fundamentals well enough to know why increasing Rx/Tx Buffers would solve the problem, when it seems like even the default buffer sizes would be sufficient for the amount of data being exchanged.
                      2. I experience the same dropped connection symptoms when attempting unthrottled SFTP transfers over the NIC.
                      3. After the connection is dropped, it takes Windows a minute or two to realize that there is no internet connection, and the only way to restore the connection is to disable and re-enable the NIC, reboot the computer, or tell the switch to reset the ports connected to the NIC.  The connection has not independently restored after a wait of 10 minutes.

                      I have tested and verified the above behavior with both interfaces individually connected to the switch, simultaneously connected to the switch, and teamed using 802.3ad LACP Dynamic Link Aggregation.

                       

                      Other Notes

                      • None of the switch, gateway/router, or remote link partners experience any connection issues during the given scenarios.
                      • I have provisioned both physical and virtual machines with the same software environment as my machine, and they do not experience these symptoms, and they do not use the same model of NIC.
                      • In combination with brodie7838's testing, particularly that of a USB3.0 Ethernet adapter, I think it is reasonable to conclude that the Intel NIC is the most likely culprit here.
                      • 8. Re: i219-V Stability Issues when Link Speed is negotiated above 10M-FDx
                        Intel Corporation
                        This message was posted on behalf of Intel Corporation

                        Hi ProspectTheorist,

                         Much thanks for sharing your solution and finding.

                        rgds,
                        wb
                         

                        • 9. Re: i219-V Stability Issues when Link Speed is negotiated above 10M-FDx
                          brodie7838

                          Hi ProspectTheorist,

                          I tested the buffer values you listed, and so far so good - I was able to download an entire Steam game at my ISP max, and the link remained up and stable. I'll keep an eye on it and update if I experience any issues.

                           

                          I'm running Driver 12.15.23.7,PROSet v21.1.30.0