7 Replies Latest reply on Sep 20, 2018 11:15 PM by zhoudan

    Is Intel's Thunderbolt 3 dual port controller's total data bandwidth 32gbps or 22gbps?? (JHL6540/DSL6540)

    mois

      Dear Intel,

       

      The following is to clarify the total PCIe data throughput supported by Intel's dual-port Thunderbolt 3 controllers, which have the product names JHL6540 and DSL6540.

       

      A JHL6540/DSL6540 controller has x4 PCIe lanes upstream right, so as a PCIe device it could do 32gbps data transfers to/from the CPU/PCH, both ways - 32gbps from CPU/PCH to controller and 32gbps from controller to CPU/PCH simultaneously, right.

       

      Meanwhile, each Thunderbolt 3 port can only do ~22gbps PCIe traffic.

       

      My question now to you is, how much *AGGREGATE* PCIe traffic does these two controllers support on BOTH ports in the same moment?

       

      Say to my JHL6540/DSL6540 would be connected to each TB3 port a TB3 devices that would pump 16gbps of PCIe data - this means my JHL6540/DSL6540 would be receiving a total of 32gbps of PCIe data.

       

      Would all those 32gbps go through the TB3 controller all to my CPU/PCH?

       

      Similarly, say my CPU pumps 16gbps PCIe data to each TB3 port (so 32gbps in total), would the TB3 controller handle that or would it cap the aggregate throughput to some lower value such as 22gbps TOTAL for both ports?

       

       

      https://thunderbolttechnology.net/sites/default/files/Thunderbolt3_TechBrief_FINAL.pdf is the only Intel document I find that discusses this and it does not say.

       

      I could imagine that there could be a constraint that JHL6540/DSL6540 would have a 22gbps *TOTAL* cap internally and so therefore to/from each port, so if one port does 10gbps the other can only do 12gbps, if one port does 15gbps the only one can only do 7gbps, etc. .

       

       

      Depending on how the JHL6540/DSL6540 behaves, I as a user will have more or less reason to use my laptop's second Thunderbolt 3 port at all - if indeed both Thunderbolt 3 ports share the same 22gbps bandwidth, then I can as well just daisy-chain devices instead.

       

       

      Please clarify ASAP, thanks!

        • 1. Re: Is Intel's Thunderbolt 3 dual port controller's total data bandwidth 32gbps or 22gbps?? (JHL6540/DSL6540)
          mois

          Question continued:

           

          If the JHL6540/DSL6540 has 32gbps total PCIe data bandwidth, then if I connect an eGPU to one Thunderbolt 3 port which consumes all 22gbps available on that port, then there will still be another 10gbps Thunderbolt 3 controller bandwidth capacity available, which is available on the other Thunderbolt 3 port only. This would be a reason for me to not daisy-chain on the first port but instead connect my extra Thunderbolt 3 peripherals to the laptop's second Thunderbolt 3 port.

          • 2. Re: Is Intel's Thunderbolt 3 dual port controller's total data bandwidth 32gbps or 22gbps?? (JHL6540/DSL6540)
            Intel Corporation
            This message was posted on behalf of Intel Corporation

            Hello mois

            Thank you for joining this Intel Community.

            I understand that you would like to know the total data bandwidth for Thunderbolt* 3. I would like to provide some information about this. 

            Thunderbolt*  is customized by each system manufacturer differently on their platforms. For this reason, our recommendation is that you can contact your system manufacturer to receive accurate information about their products the Thunderbolt* 3 implementation.

            I hope you find this information helpful. 

            Wanner G.  

            • 3. Re: Is Intel's Thunderbolt 3 dual port controller's total data bandwidth 32gbps or 22gbps?? (JHL6540/DSL6540)
              mois

              Hi Wanner,

               

              Thank you for responding.

               

              I mean to ask this question on a TECHNOLOGICALLY ADVANCED level. Presently I find NO INFORMATION WHATSOEVER available in publicly on this topic, and it really surprises me.

               

              I would like to learn from you/Intel, the MAXIMUM PCIe data bandwidth that one single JHL6540/DSL6540 controller has.

               

              I understand that laptop manufacturers implement your JHL6540/DSL6540 in DIFFERENT ways, which have performance impact.

               

              Specifically, laptop designs that would run your JHL6540/DSL6540 BELOW its full capacity that I am aware of, are

               

              • How many PCIe lanes that the JHL6540/DSL6540 is given - it's a x4 PCIe v3 lanes chip (giving 32gbps full-duplex bandwidth upstream), however can be operated also if given only x2 PCIe v3 lanes (giving 16gbps full-duplex bandwidth upstream),

              • For U series Intel CPU:s, the laptop's OPI configuration, specifically if it's set to 4GT/s (meaning optimized for performance) or 2GT/s (meaning optimized for battery) - I understand this is a CPU setting that's specified by a laptop manufacturer's BIOS and therefore manufacturer-specific, ref. https://egpu.io/ultrabook-buyers-guide-external-gpu/#opi , https://www.reddit.com/r/eGPU/comments/7vb0gg/u_series_chips_pcie_lanes_and_thunderbolt_3/ .

              • (For ULV processors where the x4 PCIe lanes go from the PCH, rather than are routed directly to the CPU as is the case in quad-core laptop processors, I would understand if the CPU's IO load to other PCH-connected devices such as the laptop's internal SSD or WIFI, could have a performance impact on Thunderbolt 3 throughput.)

               

              For this question however, I will presume that the laptop manufacturer has dedicated x4 PCIe v3 lanes to your JHL6540/DSL6540, and that the laptop is operating in 4GT/s mode.

               

              (If in the list above I missed any major factors impacting the possible JHL6540/DSL6540 real-world throughput, feel free to fill in the blanks. Anyhow I guess for this question, this list suffices.)

               

              And last now, going back to https://thunderbolttechnology.net/sites/default/files/Thunderbolt3_TechBrief_FINAL.pdf , I understand from this document that the JHL6540/DSL6540's total bandwidth for ONE Thunderbolt 3 port must be 22gbps full-duplex.

               

               

              Now, please explain to me, what is the JHL6540/DSL6540's total maximum PCIe data bandwidth, for BOTH Thunderbolt 3 ports?

               

               

              I have two working hypotheses for what it could be:

               

              1. The JHL6540/DSL6540 has an internal 22gbps full-duplex cap, which means you have a "pool" of 22gbps shared full-duplex bandwidth between both TB3 ports, e.g. if you do 15gbps PCIe data on one port, then the other port will have 7gbps.

              2. The JHL6540/DSL6540 has internal 32gbps full-duplex, as in its upper limit is defined by its upstream PCIe bandwidth, and both TB3 ports share these 32gbps in a pooled fashion with each TB3 port being able to eat at most 22gbps at a given time, meaning if you do 22gbps PCIe data on one TB3 port (that is, you saturate the whole link), then you will STILL be able to do another 10gbps PCIe data separately on the other TB3 port.

               

               

              The above is extremely valuable to know when buying Thunderbolt 3 gear:

               

              For instance, an eGPU should normally be given an own, dedicated Thunderbolt 3 port. If the JHL6540/DSL6540 has 32gbps total PCIe data bandwidth, then this means you have 10gbps potential extra bandwidth on the second Thunderbolt 3 port available, which means that you could run, for instance, a 10gbps Ethernet adapter, or an external PCIe NVMe SSD drive, and as long as that second TB3 port has less than 10gbps data in each direction it will *NOT* interfere with the eGPU:s operation, and, on the rare occasions that the data on the second TB3 port would be in the 10-22gbps interval, then this would cause no latency or slowdown in eGPU operation also, presuming that the eGPU is operating at <= 10gbps at that time.

               

              If on the other hand both TB3 ports share a total 22gbps bandwidth pool, then from a total bandwidth point of view which TB3 port eGPU, 10gbps ethernet and PCIe NVMe SSD are connected to, does not matter, and considerations would instead be made relating to any / the latency and bandwidth overhead incurred by daisy chaining, e.g. total number of devices connected in daisy chain on the TB3 port, latency and bandwidth impact from how many steps downstream a device is located in the daisy chain.

               

               

              I hope my examples and elaborations above clarified the intention and scope of the question. It's extremely important that there is a public figure explaining this.

               

              I could deduce this information by benchmarking also, however that requires ~1000-2000 worth of equipment and would take me a full day. Your company (Intel) has designed this technology, you know your design specs already, there is no reason for you to not share it publicly.

               

              Looking forward to your response ASAP, thanks!

              Mois

              • 4. Re: Is Intel's Thunderbolt 3 dual port controller's total data bandwidth 32gbps or 22gbps?? (JHL6540/DSL6540)
                Intel Corporation
                This message was posted on behalf of Intel Corporation

                Hello mois,

                 

                Thank you for your response.

                 

                The information that you are requesting is very specific, and I would like to add that there are other support websites where you can receive the information you are looking for. I would like to highlight that you keep providing examples that include how system manufacturers implement this technology on their products. For this reason, please review these support websites. Keep in mind that you may need to register or create an account.

                 

                Resource & Design Center
                https://www.intel.com/content/www/us/en/design/resource-design-center.html?_ga=2.238754115.255706475.1534181477-616312718.1532536837

                 

                Intel® Developer Zone
                https://software.intel.com/en-us/home

                 

                Hopefully, you will find relevant information about Intel® Thunderbolt*.

                 

                Wanner G.

                • 5. Re: Is Intel's Thunderbolt 3 dual port controller's total data bandwidth 32gbps or 22gbps?? (JHL6540/DSL6540)
                  mois

                  Hi Wanner,

                   

                  Thank you so much for your response.


                  I kindly ask you to let me know the JHL6540/DSL6540's total PCIe data bandwidth for both TB3 ports, given optimal working conditions and a maximally performance-optimized setup.

                   

                  How many gbps of PCIe data, on both TB3 ports cumulatively, is the JHL6540/DSL6540 design spec:ed to be able to process?

                   

                   

                  Thank you for pointing out that a particular laptop manufacturer may choose less than optimal working conditions that dump performance, so the figures you give me are for the JHL6540/DSL6540 chip's potential capacity only.

                   

                  Looking forward to your clarification, thanks!!
                  Mois

                  • 6. Re: Is Intel's Thunderbolt 3 dual port controller's total data bandwidth 32gbps or 22gbps?? (JHL6540/DSL6540)
                    mois

                    Like before, Akitio's staff stepped in and clarified, they wrote me this:

                     

                    "If you are using storage devices only (e.g. HDDs or SSDs), with one TB3 controller in your laptop, I would expect the performance to be almost identical no matter if you connect the second drive to the second port on your laptop or to the second port on the other drive.

                     

                    Either way, the maximum bandwidth for that single controller, which is around 2750 MB/s, will be shared among the two devices.

                     

                    For an eGPU, it's best to connect it directly to the computer and not connect any other devices in the daisy chain. If you connect a second device, it's best to connect it to the second port on your laptop, no matter if that is a high performance devices like a NVMe SSD or something else like a 1G network card.

                     

                    When using an eGPU on one port of your laptop and a network card on the other port, I would still expect fairly good performance for both devices. However, something like a NVMe SSD can potentialy use up a lot of the bandwidth, so the transfer rates of that SSD are likely to be throttled.

                    ..

                    Yes, the two ports share the same bandwidth. The 2750 MB/s is the actual transfer rate you can expect. For example, if you run a benchmark on a NVMe SSD that can go fast enough, it will max out around 2750 MB/s. If you test two at the same time, I would expect around 1375 MB/s per drive."

                     

                    Thanks Akitio !!

                     

                     

                    Intel, on your next revision of Thunderbolt 3 controller chips, please don't just share the bandwidth of ONE Thunderbolt 3 cable on TWO Thunderbolt 3 cables. At least deliver 32gbps = 32/22=145% of total bandwidth on the controller's two ports.

                     

                    Thunderbolt 3 ports are a scarce resource, so it is a real utility limitation of your whole technology - and hence the whole Thunderbolt 3 products industry - that your controller, as the enabling chip for the whole ecosystem, only has been designed for 50% capacity per Thunderbolt 3 port, when both ports are at full load.

                     

                    I understand PCIe lanes are a scarce resource too, but, the TB3 controller is presently only using 22/32=68% of the precious x4 lanes that a good laptop has afforded it.

                     

                    I guess laptop manufacturers could work around your design limitations by using one single TB3 controller per TB3 port, though. This would be viable on quad-core CPU:s, which have more PCIe lanes available to devote to Thunderbolt. I'll forward this feedback to Lenovo.

                    • 7. Re: Is Intel's Thunderbolt 3 dual port controller's total data bandwidth 32gbps or 22gbps?? (JHL6540/DSL6540)
                      zhoudan

                      Hey mois,

                       

                      In some implementation, the two thunderbolt ports might not be able to function simultaneously. I encountered such a situation that when I'm driving an eGPU with one thunderbolt 3 port, the other tb3 port will no work at all, no matter what device I plug into it. So I guess it might not be good practice to use both port.

                       

                      zhoudan