8 Replies Latest reply on Jan 12, 2016 3:16 PM by Surendar.Chandra

    Intel 750 not showing up in ESXi 5.5

    Surendar.Chandra

      Judging from another question, I guess it is possible. I have 1.0e.2 installed:

       

      ~ # vmkload_mod -s intel-nvme

      vmkload_mod module information

      input file: /usr/lib/vmware/vmkmod/intel-nvme

      Version: 1.0e.2.0-1OEM.550.0.0.1391871

      License: BSD

      Required name-spaces:

        com.vmware.vmkapi#v2_2_0_0

      Parameters:

       

      On this box, I have two SATA SSDs, a Intel 3700 and this drive. The 3700 shows up fine.

       

      ~ # esxcli storage core device  list

      t10.NVMe____INTEL_SSDPEDMD400G4_CVFT4474004C400BGN__00000001

         Display Name: Local NVMe Disk (t10.NVMe____INTEL_SSDPEDMD400G4_CVFT4474004C400BGN__00000001)

         Has Settable Display Name: true

         Size: 381554

         Device Type: Direct-Access

         Multipath Plugin: NMP

         Devfs Path: /vmfs/devices/disks/t10.NVMe____INTEL_SSDPEDMD400G4_CVFT4474004C400BGN__00000001

         Vendor: NVMe   

         Model: INTEL SSDPEDMD40

         Revision: 0110

         SCSI Level: 6

         Is Pseudo: false

         Status: on

         Is RDM Capable: false

         Is Local: true

         Is Removable: false

         Is SSD: true

         Is Offline: false

         Is Perennially Reserved: false

         Queue Full Sample Size: 0

         Queue Full Threshold: 0

         Thin Provisioning Status: unknown

         Attached Filters:

         VAAI Status: unknown

         Other UIDs: vml.010000000043564654343437343030344334303042474e2020494e54454c20

         Is Local SAS Device: false

         Is USB: false

         Is Boot USB Device: false

         No of outstanding IOs with competing worlds: 32

       

      750 is there:

      \==+PCI Device :

                     |----Segment.........................................0x0000

                     |----Bus.............................................0x81

                     |----Slot............................................0x00

                     |----Function........................................0x00

                     |----Runtime Owner...................................vmkernel

                     |----Has Configured Owner............................false

                     |----Configured Owner................................

                     |----Vendor Id.......................................0x8086

                     |----Device Id.......................................0x0953

                     |----Sub-Vendor Id...................................0x8086

                     |----Sub-Device Id...................................0x370d

                     |----Vendor Name.....................................Intel Corporation

                     |----Device Name.....................................NVM-Express PCIe SSD

                     |----Device Class....................................264

                     |----Device Class Name...............................Non-Volatile memory controller

                     |----PIC Line........................................11

                     |----Old IRQ.........................................11

                     |----Vector..........................................57

                     |----PCI Pin.........................................0

                     |----Spawned Bus.....................................0

                     |----Flags...........................................513

       

       

      ~ #  esxcli storage core adapter list

      HBA Name  Driver      Link State  UID           Description                                          

      --------  ----------  ----------  ------------  ------------------------------------------------------

      vmhba38   ahci        link-n/a    sata.vmhba38  (0:0:31.2) Intel Corporation Wellsburg AHCI Controller

      vmhba39   ahci        link-n/a    sata.vmhba39  (0:0:31.2) Intel Corporation Wellsburg AHCI Controller

      vmhba0    ahci        link-n/a    sata.vmhba0   (0:0:17.4) Intel Corporation Wellsburg AHCI Controller

      vmhba1    ahci        link-n/a    sata.vmhba1   (0:0:31.2) Intel Corporation Wellsburg AHCI Controller

      vmhba2    intel-nvme  link-n/a    pscsi.vmhba2  (0:130:0.0) Intel Corporation DC P3700 AIC           

      vmhba32   ahci        link-n/a    sata.vmhba32  (0:0:17.4) Intel Corporation Wellsburg AHCI Controller

      vmhba33   ahci        link-n/a    sata.vmhba33  (0:0:17.4) Intel Corporation Wellsburg AHCI Controller

      vmhba34   ahci        link-n/a    sata.vmhba34  (0:0:17.4) Intel Corporation Wellsburg AHCI Controller

      vmhba35   ahci        link-n/a    sata.vmhba35  (0:0:31.2) Intel Corporation Wellsburg AHCI Controller

      vmhba36   ahci        link-n/a    sata.vmhba36  (0:0:31.2) Intel Corporation Wellsburg AHCI Controller

      vmhba37   ahci        link-n/a    sata.vmhba37  (0:0:31.2) Intel Corporation Wellsburg AHCI Controller

       

      Can you please help. Thanks

        • 2. Re: Intel 750 not showing up in ESXi 5.5
          warren_intel

          Or now I see you have an Intel SSD DC P3700 series showing fine. This is what we know The Skinny on NVM Express and ESXi

          Which driver are you using...

          • 3. Re: Intel 750 not showing up in ESXi 5.5
            Surendar.Chandra

            Thanks for that link. I tried with both Intel drivers '1.0e.2.0-1OEM.550.0.0.1391871' and VMware driver '1.0e.0.30-1vmw.550.0.0.1391871'. Both see the DC P3700 and both do not see the 750. I saw another commenter ask about delete support for 750 under ESXi and was hoping that meant these worked with ESXi. It is not in the HCL list though I've seen many drives not on that list work just fine. Thanks Warren

            • 4. Re: Intel 750 not showing up in ESXi 5.5
              Surendar.Chandra

              Tried moving the 750 into its own box without any other Intel NVMe drives; still no dice.

               

              000:081:00.0                                         

                 Address: 000:081:00.0                             

                 Segment: 0x0000                                   

                 Bus: 0x81                                         

                 Slot: 0x00                                        

                 Function: 0x00                                    

                 VMkernel Name: vmhba2                             

                 Vendor Name: Intel Corporation                    

                 Device Name: <class> Non-Volatile memory controller

                 Configured Owner: Unknown                       

                 Current Owner: VMkernel                         

                 Vendor ID: 0x8086                               

                 Device ID: 0x0953                               

                 SubVendor ID: 0x8086                            

                 SubDevice ID: 0x370d                            

                 Device Class: 0x0108                            

                 Device Class Name: Non-Volatile memory controller

                 Programming Interface: 0x02                       

                 Revision ID: 0x01                                 

                 Interrupt Line: 0x0b              

                 IRQ: 11                           

                 Interrupt Vector: 0x39            

                 PCI Pin: 0xd0                     

                 Spawned Bus: 0x00                 

                 Flags: 0x0201                     

                 Module ID: 18                                   

                 Module Name: nvme                               

                 Chassis: 0                        

                 Physical Slot: 1                  

                 Slot Description: Chassis slot 1.00

                 Passthru Capable: true          

                 Parent Device: PCI 0:128:2:0    

                 Dependent Device: PCI 0:129:0:0 

                 Reset Method: Function reset    

                 FPT Sharable: true

               

              Tried with Intel as well as VMware drivers. Not sure how it worked for the other person :-(

              • 5. Re: Intel 750 not showing up in ESXi 5.5
                Surendar.Chandra

                Doesn't work with ESX 6.0 either both with the builtin driver and Intel

                 

                ~ # vmkload_mod -s intel-nvme

                vmkload_mod module information

                input file: /usr/lib/vmware/vmkmod/intel-nvme

                Version: 1.0e.2.0-1OEM.550.0.0.1391871

                License: BSD

                Required name-spaces:

                  com.vmware.vmkapi#v2_2_0_0

                Parameters:

                 

                Built in:

                 

                ~ #  vmkload_mod -s nvme

                vmkload_mod module information

                input file: /usr/lib/vmware/vmkmod/nvme

                Version: 1.0e.0.35-1vmw.600.1.17.3029758

                Build Type: release

                License: BSD

                Required name-spaces:

                  com.vmware.vmkapi#v2_3_0_0

                Parameters:

                  nvme_compl_worlds_num: int

                    Total number of NVMe completion worlds/queues.

                  max_namespaces: int

                    Maximum number of namespaces supported.

                  max_io_request: int

                    IO Queue high water marker throttle

                  transfer_size: int

                    Maximum IO request Xfer size in 1K bytes maximum 1024

                  io_command_id_size: int

                    Number of command IDs per submission queue

                  io_cpl_queue_size: int

                    NVMe number of IO completion queue entries

                  io_sub_queue_size: int

                    NVMe number of IO submission queue entries

                  admin_cpl_queue_size: int

                    NVMe number of Admin completion queue entries

                  admin_sub_queue_size: int

                    NVMe number of Admin submission queue entries.

                  max_prp_list: int

                    Maximum number of PRP lists supported.

                  nvme_force_intx: int

                    Force using INTX mode instead of MSIx.

                  nvme_log_level: int

                    Log level.

                  1 - error

                  2 - warning

                  3 - info (default)

                  4 - verbose

                 

                This is with the latest ESX (6.0.0 #1 SMP Release build-3073146 Sep 16 2015)

                • 6. Re: Intel 750 not showing up in ESXi 5.5
                  warren_intel

                  I think I need to understand again what you have in your system. The SSD 750 should be visible as a data drive. Does it show in system BIOS?

                  You said "On this box, I have two SATA SSDs, a Intel 3700 and this drive. The 3700 shows up fine."

                   

                  The SSD 750 is not SATA and the correct hardware and bios settings are required to support it. If the drive looks like a SATA then you have the 2.5" version and will need a proper HBA or U.2 cable to connect it to.  If a drive bay is used the system will need an Intel(R) Xeon(tm) E5 "v3" processor to have the PCIe lanes routed in the drive bays.

                   

                  To confirm can you provide the SSD model# and the motherboard or system model you are using.

                  • 7. Re: Intel 750 not showing up in ESXi 5.5
                    Surendar.Chandra

                    Sorry I mis-wrote. It is a PCIe drive (and not the 2.5" version) - please globally replace SATA with PCIe in my message.

                     

                    It is showing up as NVM-Express PCIe SSD, Device ID: 953, Sub Device ID: 370D, Class ID: 108, Vendor ID: 8086

                     

                    Its a Super Micro SYS-2028TP-DTTR

                     

                    Thanks Warren

                    • 8. Re: Intel 750 not showing up in ESXi 5.5
                      Surendar.Chandra

                      ~ # vmkvsitools lspci | grep SSD

                      0000:81:00.0 Mass storage controller: Intel Corporation NVM-Express PCIe SSD [vmhba2]

                       

                      Thouogh hba2 does not show up in storage list

                       

                      ~ # esxcli storage core adapter list

                      HBA Name  Driver  Link State  UID           Description                                          

                      --------  ------  ----------  ------------  ------------------------------------------------------

                      vmhba38   ahci    link-n/a    sata.vmhba38  (0:0:31.2) Intel Corporation Wellsburg AHCI Controller

                      vmhba39   ahci    link-n/a    sata.vmhba39  (0:0:31.2) Intel Corporation Wellsburg AHCI Controller

                      vmhba0    ahci    link-n/a    sata.vmhba0   (0:0:17.4) Intel Corporation Wellsburg AHCI Controller

                      vmhba1    ahci    link-n/a    sata.vmhba1   (0:0:31.2) Intel Corporation Wellsburg AHCI Controller

                      vmhba32   ahci    link-n/a    sata.vmhba32  (0:0:17.4) Intel Corporation Wellsburg AHCI Controller

                      vmhba33   ahci    link-n/a    sata.vmhba33  (0:0:17.4) Intel Corporation Wellsburg AHCI Controller

                      vmhba34   ahci    link-n/a    sata.vmhba34  (0:0:17.4) Intel Corporation Wellsburg AHCI Controller

                      vmhba35   ahci    link-n/a    sata.vmhba35  (0:0:31.2) Intel Corporation Wellsburg AHCI Controller

                      vmhba36   ahci    link-n/a    sata.vmhba36  (0:0:31.2) Intel Corporation Wellsburg AHCI Controller

                      vmhba37   ahci    link-n/a    sata.vmhba37  (0:0:31.2) Intel Corporation Wellsburg AHCI Controller

                       

                      Rescan did not help either

                      # esxcli storage core adapter rescan --all