2 Replies Latest reply on Feb 9, 2018 10:41 AM by cvare

    What are the conditions for OsNativePciEnumeration=True with Thunderbolt?


      Hello, community!


      In the release notes for Thunderbolt 3 Windows driver (https://downloadmirror.intel.com/26052/eng/Thunderbolt%20Release%20Notes.pdf), we see "Added support for Windows* 10 RS3 Native PCIe Enumeration ("Native Express" mode) and Selective Suspend (RTD3) state".


      If you run WMI Explorer (Releases · vinaypamnani/wmie2 · GitHub) in Windows with the Thunderbolt driver and service installed, you can find the Boolean field "OsNativePciEnumeration" as a property of a SdkTbtController instance.


      However, this is always False. I have tried JHL6340, JHL6540, DSL6540 controllers with the latest driver and up to date Windows 10 16299. I asked a friend with a 2016 Macbook Pro (2x JHL6540) to try it in Bootcamp because the Apple Thunderbolt implementation is known to be significantly different from normal computers. It is still False.


      So what will enable OsNativePciEnumeration=True? The JHL7540 Titan Ridge controller? NVM firmware updates? Full BIOS updates? Does Microsoft have to flick a switch? Will Linux support it, or can it just use the functionality of Linux which can already do enumeration?


      The problem is that Intel has announced the Titan Ridge family of controllers, but they are nowhere to be seen, so I cannot find out whether they will enable the functionality. Thanks, Intel, for announcing a phantom product, with no known availability date.


      That above is the question. The rest is just discussion:


      If anybody is interested, I may have figured out what makes a supported Thunderbolt-ready motherboard able to detect a Thunderbolt add-in card. The supported slot of the supported motherboard is configured as an acpiphp (ACPI PCI Hotplug Bridge) and has subordinate higher than the bus number. On a supported slot of a supported motherboard, the Thunderbolt 3 add-in card works for almost all functionality without the GPIO header attached at any point in time. The exception is PCI hotplugging - but if an eGPU is attached at boot time, it 100% works. Thunderbolt networking can hotplug happily without the header, though - it does not need to expose DMA, iirc. This means that a BIOS modification can make any motherboard whatsoever support Thunderbolt add-in cards. Also, a NVM firmware update and driver modifications should allow for the hotplug event signals to be sent through the PCI bus from the NHI, instead of the GPIO header. This might require changes to current PCI enumeration, by handing the job entirely over to the operating system kernel. Thunderbolt is not detected at all in Linux when passing acpi=off to the kernel, which means that even Linux cannot handle Thunderbolt without firmware assistance - despite its ability to throw away the BIOS allocation and re-do everything from scratch. ACPI has been compared to a trojan / backdoor by senior Linux developers, and needs to disappear. If Linux can support drivers for most of the world's hardware, then drivers can be added for each device / system to handle the ACPI functionality in a fashion that can be audited properly for bugs and security.


      The hints in the release notes suggest that Intel might be moving to free Thunderbolt from the BIOS / ACPI mess. The reason for the move to open source Thunderbolt technology is because of Apple forcing them - because Apple want to work with AMD also. I despise Apple, but they pushed DisplayPort before it was cool, they pushed Thunderbolt before it was cool, they are the first (and possibly only) ones to put NVMe in ARM (iPhone), and they are responsible for OpenCL. I applaud these achievements by them. But to know for sure what is happening is difficult, especially given that this "Native Express" mode of Windows RS3 appears to be completely undocumented, with the only mention being the Intel driver release notes. I hope for Thunderbolt to be completely released from Intel to the public domain, so that we may see it in all systems and architectures. It is amazing, but I would rather drop it than be stuck with Intel systems forever. Also, even with Intel systems, you cannot add as many ports as you want. There are no motherboards that support multiple add-in cards, and the best that exists is one old Gigabyte Z170 motherboard with 1x onboard port and add-in card support for a total of three ports. So the only way to get four ports is a Macbook Pro which has obvious massive and debilitating drawbacks.


      On another tangent, Thunderbolt networking is only one of many services that could be offered. It is not hard-coded or baked into the hardware. It just uses DMA rings to write data to allowed regions of the peer system. This means interesting things like using your Thunderbolt 3 desktop as an external graphics card for your laptop should be possible. This would be implemented by exposing the GPU to the Thunderbolt connection as if it were a VM running on the desktop (using the IOMMU).


      But for now, I pose five challenges for those familiar with kernel programming who are interested:


      1. Download Linux 4.15 kernel source and modify the Thunderbolt networking code to allow for higher than 10Gbps on Thunderbolt connections with enough bandwidth, and post iperf results and code. This will require two Linux Thunderbolt 3 machines, both with the driver modification. For an additional challenge, try to modify it such that it does not break compatibility with computers with unmodified Thunderbolt Networking drivers.

      2. Download Linux 4.15 kernel source. Can you remove the Thunderbolt bottleneck for PCIe which limits it to 22Gbps, instead of the almost 32Gbps provided by PCIe 3.0 x4? This is almost certainly because of allocation of DMA rings for DisplayPort. So is it possible for full PCIe bandwidth if DisplayPort is disabled in the drivers? Or would this require NVM firmware updates for the downsteam devices, or even the host controller?

      3. Download Linux 4.15 kernel source. Is it possible to make modifications to allow for a Thunderbolt controller to act as a DisplayPort capture card without NVM firmware modification? I am a firm believer that everything with a display should allow for that display to be used as a monitor for another device - making your laptop a USB-C monitor. Using Thunderbolt with driver modifications would require the computer to be powered on, but it would be the most likely way to allow for it without dedicated hardware from the USB-C port to the display (presumably a MUX to the display, and USB Port CC firmware updates).

      4. A computer knows when a Thunderbolt loopback (connecting two Thunderbolt ports together, either from the same controller or from different controllers) is connected, and ignores it. Can anybody provide evidence whether this is in controller hardware, NVM firmware, system BIOS firmware, or the operating system drivers? Can it be overridden to allow for Thunderbolt Networking loopback? This one is more of an exercise in curiosity, than anything practical.

      5. I don't expect anybody to be capable of turning a desktop into an eGPU on their own by passing the client through to the GPU as if it were a VM running on the desktop. But if you think you are up for the challenge, go ahead!


      I appreciate absolutely any answers or discussion this can generate. Even if you are not sure, feel free to speculate or throw ideas around!


      Thank you and happy computing!