3 Replies Latest reply on Aug 13, 2018 11:39 AM by Intel Corporation

    With Intel's Thunderbolt 3 technology, what is the expectable overhead of using daisy-chaining to connect another device?

    mois

      Hi Intel,

       

      I can't find any recent clear statement about how much overhead daisy-chaining Thunderbolt 3 peripherals incurs.

       

      E.g., if I connect two or three devices to one Thunderbolt 3 port in a daisy-chained configuration, how much overhead or what limits will the first, second and third device likely have, expectably and very approximately only, compared to if the Thunderbolt 3 port only had one single device connected to it?

       

      I understand that different Thunderbolt 3 device controllers may behave differently, e.g. using your JHL6540 and DSL6540 as Thunderbolt 3 controller in the laptop and the TI83 controller in each device, however I also think you can provide a ballpark figure estimation based on how the Thunderbolt 3 standard has been specified and designed.

       

      Please let me know ASAP, thanks.

        • 1. Re: With Intel's Thunderbolt 3 technology, what is the expectable overhead of using daisy-chaining to connect another device?
          Intel Corporation
          This message was posted on behalf of Intel Corporation

          Hello mois, 

          Thank you for joining this Intel Community.

          Please refer to the other thread created about Thunderbolt 3. 

          https://communities.intel.com/thread/127679

          Wanner G. 

          • 2. Re: With Intel's Thunderbolt 3 technology, what is the expectable overhead of using daisy-chaining to connect another device?
            mois

            Hi Wanner,

             

            You do not yet make any mentioning whatsoever in the other thread about the impact of daisy chaining on Thunderbolt 3's performance. The question about total PCIe data bandwidth per dual-port TB3 controller in the other thread, is much more prioritary than this question however.

             

            Again please note that I like to ask this question on the level of an advanced user.

             

             

            My question specifically is, in a typical dual-port (JHL6540 and DSL6540) or single-port (JHL6340 and DSL6340) Thunderbolt 3 controller setup, roughly approximately,

             

            • What is the latency and bandwidth impact on the PCIe data on a device, of being located N hops (1 / 2 / 3 / 4 / 5) away from the laptop's Thunderbolt 3 port?

            • Also, how will latency and bandwidth impact on one TB3 device due to daisy chaining be correlated with the degree of PCIe data link saturation caused by the Thunderbolt 3 devices "upstream" (between the controller and the given device), e.g. say in the case of attempt by upstream devices to occupy 0%, 25%, 50%, 75%, 100% of the TB3 daisy chain's total 22gbps PCIe data bandwidth.

            • Same question as above but by "downstream" devices.

             

            I think this is relevant information for people to have, when making up their minds about what Thunderbolt 3 devices to buy, and, as it turns out I can't find one single mentioning on the Internet about daisy chain latency and bandwidth impact in TB3, all tests are from back in the Thunderbolt 1 and 2 days. Maybe Intel improved something?

             

             

            The tests from Thunderbolt 1 and 2 don't mention latency, only bandwidth. Latency is useful to know as it will determine networking latency for a 10gbps Ethernet TB3 controller, and disk access latency for database operations on a TB3-connected PCIe NVMe SSD.

             

            Not considering that constraint though, the old tests did a good job at benchmarking bandwidth loss in the daisy chain:

             

            • With Thunderbolt 1 (~2011, 7 years ago), in a daisy chain with THREE devices, if the first and second devices consumed NO bandwidth, then still the THIRD device would suffer a 35% bandwidth loss, reference https://www.anandtech.com/show/10248/thunderbolt-3-in-action-akitio-thunder3-duo-pro-das-review/5 . (No mentioning here of latency overhead.)

            • A test from 2013 is https://youtu.be/a6I5lerqbvE?t=147 , I guess this is Thunderbolt 1 still. Merely adding one downstream device had a 0.5% bandwidth impact on chain's first device. The second device in the chain had 15% slower data reception and 7% slower data pushing.

            • Another Thunderbolt 1 daisy chain overhead benchmark https://www.macworld.com/article/1163773/storage/thunderbolt-how-devices-affect-each-other-on-a-daisy-chain.html suggests connecting Thunderbolt displays means a big slowdown, but did not see any slowdown due to location in the daisy chain.

            • https://nofilmschool.com/2014/07/thunderbolt-daisy-chain-external-hard-drives-performance from 2014 concludes that "apparently, an external drive array's placement in a Thunderbolt daisy chain can degrade its throughput, even if devices in between aren't doing anything". They seemed to use a mix of Thunderbolt 2 and 1 devices. They got all kinds of numbers out of this test, apparently total measured throughput was 24gbps (3GB/sec) (per what - four TB2 ports?) and they on the one hand suggest peak performance of one drive was 271MB/sec=2.1gbps but on the other hand 709.8MB/sec. The maybe more relevant number they report is that, by moving a Thunderbolt 2 device from the beginning of the daisy chain to the end (6:th position), bandwidth dropped by 22% (from 709.8MB/sec to 584.7MB/sec).


            These tests would need to be re-made today in 2018 when TB3 gives 22gbps = 2.75GB/sec potential throughput and in particular SSD:s have gotten faster.

             

            Also, I don't find any benchmarks daisy chain impact on Thunderbolt 3 10+gbps ethernet controllers.

             

             

            Your rough estimation of latency and bandwidth impact now in the Thunderbolt 3 era would be much appreciated.

             

            Overall information about how Thunderbolt 3 can be expected to work, is extremely scarce. I hope this question serves as a spark to fill out some of the blanks.

             

            Mois

            • 3. Re: With Intel's Thunderbolt 3 technology, what is the expectable overhead of using daisy-chaining to connect another device?
              Intel Corporation
              This message was posted on behalf of Intel Corporation

              Hello mois,

              Thank you for your response. 

              The information that you are requesting is very specific, and I would like to add that there are other support websites where you can receive the information you are looking for. I would like to highlight that you keep providing examples that include how system manufacturers implement this technology on their products. For this reason, please review these support websites. Keep in mind that you may need to register or create an account.

              Resource & Design Center
              https://www.intel.com/content/www/us/en/design/resource-design-center.html?_ga=2.238754115.255706475.1534181477-616312718.1532536837

              Intel® Developer Zone
              https://software.intel.com/en-us/home

              Hopefully, you will find relevant information about Intel® Thunderbolt*.