2 Replies Latest reply on Mar 23, 2017 3:05 PM by Intel Corporation

    Can't get information about Omnipath HFI on RHEL 7.3 hosts

    wilshire461

      Recently we got some new KNL nodes and decided to try RHEL 7.3 on these hosts running the 3.10.0-514.10.2.el7.x86_64 kernel.

       

      After installing the IntelOPA-Basic software and upgrading the firmware on the HFI and rebooting the nodes, we still can't get anything other than the following from opainfo

       

      [root@sknl0701 ~]# opainfo

      oib_utils ERROR: [7534] open_verbs_ctx: failed to find verbs device

      opainfo: Unable to open hfi:port 0:1

       

      Even though the software and firmware never complains about any errors we can still see that even after forcing dracut to recreate the system image the hfi1 driver will not load.

       

      [root@shas0101 ~]# lsmod | grep hfi1

      hfi1                  633634  1

      rdmavt                 57992  1 hfi1

      ib_mad                 47817  5 hfi1,ib_cm,ib_sa,rdmavt,ib_umad

      ib_core                98787  14 hfi1,rdma_cm,ib_cm,ib_sa,iw_cm,xprtrdma,ib_mad,ib_ucm,rdmavt,ib_iser,ib_umad,ib_uverbs,ib_ipoib,ib_isert

      i2c_algo_bit           13413  2 hfi1,mgag200

      i2c_core               40582  6 drm,hfi1,ipmi_ssif,drm_kms_helper,mgag200,i2c_algo_bit

       

       

      [root@sknl0701 ~]# modprobe -v hfi1

      [root@sknl0701 ~]# lsmod | grep hfi1

      hfi1                  697628  0

      rdmavt                 63294  1 hfi1

      ib_core               210381  13 hfi1,rdma_cm,ib_cm,iw_cm,rpcrdma,ib_ucm,rdmavt,ib_iser,ib_umad,ib_uverbs,rdma_ucm,ib_ipoib,ib_isert

      i2c_algo_bit           13413  3 igb,hfi1,mgag200

      i2c_core               40756  7 drm,igb,hfi1,ipmi_ssif,drm_kms_helper,mgag200,i2c_algo_bit

       

       

      [root@sknl0701 ~]# yum info libibmad

      Loaded plugins: product-id, search-disabled-repos, subscription-manager

      Installed Packages

      Name        : libibmad

      Arch        : x86_64

      Version     : 1.3.12

      Release     : 1.el7

      Size        : 132 k

      Repo        : installed

      From repo   : curc

      Summary     : OpenFabrics Alliance InfiniBand MAD library

      URL         : http://openfabrics.org/

      License     : GPLv2 or BSD

      Description : libibmad provides low layer IB functions for use by the IB diagnostic

                  : and management programs. These include MAD, SA, SMP, and other basic

                  : IB functions.

       

       

       

      [root@sknl0701 ~]# yum info libibmad-devel

      Loaded plugins: product-id, search-disabled-repos, subscription-manager

      Installed Packages

      Name        : libibmad-devel

      Arch        : x86_64

      Version     : 1.3.12

      Release     : 1.el7

      Size        : 50 k

      Repo        : installed

      From repo   : curc

      Summary     : Development files for the libibmad library

      URL         : http://openfabrics.org/

      License     : GPLv2 or BSD

      Description : Development files for the libibmad library.

       

      libibmad was and has been installed on the new node as well, so I am out of ideas at the moment.  Any help would be appreciated!