2 Replies Latest reply: Mar 29, 2012 3:05 AM by emilec RSS

    SuSE Linux Enterprise 11 SP1 under Modular Server MFSYS25V2: (Dual-) SCM Problem / MultiPath

    bic_admin

      We are trying to install SLES 11 SP1 x64 on a MFS5520VIBR Compute-Modul  and are facing a

      critical problem regarding  MultiPath.

       

       

      uname -a

      Linux LXTEST 2.6.32.12-0.7-default #1 SMP 2010-05-20 11:14:20 +0200 x86_64 x86_64 x86_64 GNU/Linux


      Installation Medium:

      SLES-11-SP1-DVD-x86_64-GM-DVD1

       

       

      Our Installation Procedure:

      - Remove SCM #2

      - Create Virtual Drive

      - Basic Installation of SLES 11 SP1 (apply almost all default settings)

      - Install rpm package: (from Package SLES11SP1_MPIO_Setup20110712)

               rpm -ivh --force scsi_dh_alua_sles11sp1-2-1.x86_64.rpm

      - Copy MultiPath-Config (without editing) (from Package SLES11SP1_MPIO_Setup20110712)

                cp multipath.conf /etc/multipath.conf

      - Activate MultiPathD

                multipathd  on

      - Ensure that MultiPathD is running

                multipathd  on

                     ===> Output: multipathd  on

      - Plug-In the SCM #2

      - Reboot Compute-Modul

       

      _____________________________________________________________

       

      Output of multipath -ll:

           <EMPTY>

       

      Output of multipath -v4:

      Feb 29 10:54:40 | Discover device /sys/block/sdb
      Feb 29 10:54:40 | sdb: not found in pathvec
      Feb 29 10:54:40 | sdb: mask = 0x1f
      Feb 29 10:54:40 | sdb: dev_t = 8:16
      Feb 29 10:54:40 | sdb: size = 83886080
      Feb 29 10:54:40 | sdb: subsystem = scsi
      Feb 29 10:54:40 | sdb: vendor = Intel
      Feb 29 10:54:40 | sdb: product = Multi-Flex
      Feb 29 10:54:40 | sdb: rev = 0308
      Feb 29 10:54:40 | sdb: h:b:t:l = 0:0:1:0
      Feb 29 10:54:40 | sdb: serial = 4C20202000000000000000008714B4EE666632D3
      Feb 29 10:54:40 | sdb: get_state
      Feb 29 10:54:40 | loading /lib64/multipath/libchecktur.so checker
      Feb 29 10:54:40 | sdb: path checker = tur (controller setting)
      Feb 29 10:54:40 | sdb: state = running
      Feb 29 10:54:40 | sdb: state = up
      Feb 29 10:54:40 | sdb: getuid = /lib/udev/scsi_id -g -u /dev/%n (controller setting)
      Feb 29 10:54:40 | sdb: uid = 222910001555ba713 (callout)
      Feb 29 10:54:40 | sdb: state = running
      Feb 29 10:54:40 | sdb: prio = alua (controller setting)
      Feb 29 10:54:40 | reported target port group is 1
      Feb 29 10:54:40 | aas = 80 [active/optimized] [preferred]
      Feb 29 10:54:40 | sdb: alua prio = 150
      Feb 29 10:54:40 | Discover device /sys/block/sda
      Feb 29 10:54:40 | sda: not found in pathvec
      Feb 29 10:54:40 | sda: mask = 0x1f
      Feb 29 10:54:40 | sda: dev_t = 8:0
      Feb 29 10:54:40 | sda: size = 83886080
      Feb 29 10:54:40 | sda: subsystem = scsi
      Feb 29 10:54:40 | sda: vendor = Intel
      Feb 29 10:54:40 | sda: product = Multi-Flex
      Feb 29 10:54:40 | sda: rev = 0308
      Feb 29 10:54:40 | sda: h:b:t:l = 0:0:0:0
      Feb 29 10:54:40 | sda: serial = 4C20202000000000000000008714B4EE666632D3
      Feb 29 10:54:40 | sda: get_state
      Feb 29 10:54:40 | sda: path checker = tur (controller setting)
      Feb 29 10:54:40 | sda: state = running
      Feb 29 10:54:40 | sda: state = up
      Feb 29 10:54:40 | sda: getuid = /lib/udev/scsi_id -g -u /dev/%n (controller setting)
      Feb 29 10:54:40 | sda: uid = 222910001555ba713 (callout)
      Feb 29 10:54:40 | sda: state = running
      Feb 29 10:54:40 | sda: prio = alua (controller setting)
      Feb 29 10:54:40 | reported target port group is 0
      Feb 29 10:54:40 | aas = 02 [standby]
      Feb 29 10:54:40 | sda: alua prio = 1
      Feb 29 10:54:40 | Discover device /sys/block/sdc
      Feb 29 10:54:40 | sdc: not found in pathvec
      Feb 29 10:54:40 | sdc: mask = 0x1f
      Feb 29 10:54:40 | sdc: dev_t = 8:32
      Feb 29 10:54:40 | sdc: size = 0
      Feb 29 10:54:40 | sdc: subsystem = scsi
      Feb 29 10:54:40 | sdc: vendor = Virtual
      Feb 29 10:54:40 | sdc: product = Floppy
      Feb 29 10:54:40 | sdc: rev = 1.00
      Feb 29 10:54:40 | sdc: h:b:t:l = 1:0:0:1
      Feb 29 10:54:40 | Couldn't open node for sdc: No medium found
      Feb 29 10:54:40 | Discover device /sys/block/sr0
      Feb 29 10:54:40 | sr0: device node name blacklisted
      Feb 29 10:54:40 | Discover device /sys/block/loop0
      Feb 29 10:54:40 | loop0: device node name blacklisted
      Feb 29 10:54:40 | Discover device /sys/block/loop1
      Feb 29 10:54:40 | loop1: device node name blacklisted
      Feb 29 10:54:40 | Discover device /sys/block/loop2
      Feb 29 10:54:40 | loop2: device node name blacklisted
      Feb 29 10:54:40 | Discover device /sys/block/loop3
      Feb 29 10:54:40 | loop3: device node name blacklisted
      Feb 29 10:54:40 | Discover device /sys/block/loop4
      Feb 29 10:54:40 | loop4: device node name blacklisted
      Feb 29 10:54:40 | Discover device /sys/block/loop5
      Feb 29 10:54:40 | loop5: device node name blacklisted
      Feb 29 10:54:40 | Discover device /sys/block/loop6
      Feb 29 10:54:40 | loop6: device node name blacklisted
      Feb 29 10:54:40 | Discover device /sys/block/loop7
      Feb 29 10:54:40 | loop7: device node name blacklisted
      Feb 29 10:54:40 | Discovery status 0
      ===== paths list =====
      uuid              hcil    dev dev_t pri dm_st chk_st vend/prod/rev    dev_st
      222910001555ba713 0:0:1:0 sdb 8:16  150 undef ready  Intel,Multi-Flex running
      222910001555ba713 0:0:0:0 sda 8:0   1   undef ready  Intel,Multi-Flex running
                        1:0:0:1 sdc 8:32  -1  undef faulty Virtual,Floppy   running
      Feb 29 10:54:40 | sdb: ownership set to 222910001555ba713
      Feb 29 10:54:40 | sdb: not found in pathvec
      Feb 29 10:54:40 | sdb: mask = 0xc
      Feb 29 10:54:40 | sdb: get_state
      Feb 29 10:54:40 | sdb: state = running
      Feb 29 10:54:40 | sdb: state = up
      Feb 29 10:54:40 | sdb: state = running
      Feb 29 10:54:40 | reported target port group is 1
      Feb 29 10:54:40 | aas = 80 [active/optimized] [preferred]
      Feb 29 10:54:40 | sdb: alua prio = 150
      Feb 29 10:54:40 | sda: ownership set to 222910001555ba713
      Feb 29 10:54:40 | sda: not found in pathvec
      Feb 29 10:54:40 | sda: mask = 0xc
      Feb 29 10:54:40 | sda: get_state
      Feb 29 10:54:40 | sda: state = running
      Feb 29 10:54:40 | sda: state = up
      Feb 29 10:54:40 | sda: state = running
      Feb 29 10:54:40 | reported target port group is 0
      Feb 29 10:54:40 | aas = 02 [standby]
      Feb 29 10:54:40 | sda: alua prio = 1
      Feb 29 10:54:40 | 222910001555ba713: features = 1 queue_if_no_path (controller setting)
      Feb 29 10:54:40 | 222910001555ba713: no_path_retry = -2 (controller setting)
      Feb 29 10:54:40 | 222910001555ba713: update dev_loss_tmo to 2147483647

       

      Feb 29 10:54:40 | sdb: checking rport for /devices/pci0000:00/0000:00:07.0/0000:04:00.0/host0/port-0:1/expander-0:1/port-0:1:1/end_device-0:1:1/target0:0:1/0:0:1:0/block/sdb
      Feb 29 10:54:40 | sdb: Not a FC device
      Feb 29 10:54:40 | 222910001555ba713: verified path sdb dev_t 8:16
      Feb 29 10:54:40 | 222910001555ba713: update dev_loss_tmo to 2147483647

       

      Feb 29 10:54:40 | sda: checking rport for /devices/pci0000:00/0000:00:07.0/0000:04:00.0/host0/port-0:0/expander-0:0/port-0:0:1/end_device-0:0:1/target0:0:0/0:0:0:0/block/sda
      Feb 29 10:54:40 | sda: Not a FC device
      Feb 29 10:54:40 | 222910001555ba713: verified path sda dev_t 8:0
      Feb 29 10:54:40 | 222910001555ba713: pgfailback = -2 (controller setting)
      Feb 29 10:54:40 | 222910001555ba713: pgpolicy = group_by_prio (controller setting)
      Feb 29 10:54:40 | 222910001555ba713: selector = round-robin 0 (controller setting)
      Feb 29 10:54:40 | 222910001555ba713: features = 1 queue_if_no_path (controller setting)
      Feb 29 10:54:40 | 222910001555ba713: hwhandler = 1 alua (controller setting)
      Feb 29 10:54:40 | 222910001555ba713: rr_weight = 1 (internal default)
      Feb 29 10:54:40 | 222910001555ba713: minio = 100 (controller setting)
      Feb 29 10:54:40 | 222910001555ba713: no_path_retry = -2 (controller setting)
      Feb 29 10:54:40 | 222910001555ba713: pg_timeout = NONE (internal default)
      Feb 29 10:54:40 | 222910001555ba713: assembled map [1 queue_if_no_path 1 alua 2 1 round-robin 0 1 1 8:16 100 round-robin 0 1 1 8:0 100]

       

      Feb 29 10:54:40 | 222910001555ba713: set ACT_CREATE (map does not exist)
      Feb 29 10:54:40 | 222910001555ba713: addmap [0 83886080 multipath 1 queue_if_no_path 1 alua 2 1 round-robin 0 1 1 8:16 100 round-robin 0 1 1 8:0 100]

       

      Feb 29 10:54:40 | libdevmapper: ioctl/libdm-iface.c(1702): device-mapper: reload ioctl failed: Device or resource busy
      Feb 29 10:54:40 | 222910001555ba713: addmap [0 83886080 multipath 1 queue_if_no_path 1 alua 2 1 round-robin 0 1 1 8:16 100 round-robin 0 1 1 8:0 100]

       

      Feb 29 10:54:41 | libdevmapper: ioctl/libdm-iface.c(1702): device-mapper: reload ioctl failed: Device or resource busy
      Feb 29 10:54:41 | 222910001555ba713: domap (0) failure for create/reload map
      Feb 29 10:54:41 | 222910001555ba713: remove multipath map
      Feb 29 10:54:41 | sdb: orphaned
      Feb 29 10:54:41 | tur checker refcount 2
      Feb 29 10:54:41 | sda: orphaned
      Feb 29 10:54:41 | tur checker refcount 1
      Feb 29 10:54:41 | sdb: ownership set to 222910001555ba713
      Feb 29 10:54:41 | sdb: not found in pathvec
      Feb 29 10:54:41 | sdb: mask = 0xc
      Feb 29 10:54:41 | sdb: get_state
      Feb 29 10:54:41 | sdb: path checker = tur (controller setting)
      Feb 29 10:54:41 | sdb: state = running
      Feb 29 10:54:41 | sdb: state = up
      Feb 29 10:54:41 | sdb: state = running
      Feb 29 10:54:41 | sdb: prio = alua (controller setting)
      Feb 29 10:54:41 | reported target port group is 1
      Feb 29 10:54:41 | aas = 80 [active/optimized] [preferred]
      Feb 29 10:54:41 | sdb: alua prio = 150
      Feb 29 10:54:41 | sda: ownership set to 222910001555ba713
      Feb 29 10:54:41 | sda: not found in pathvec
      Feb 29 10:54:41 | sda: mask = 0xc
      Feb 29 10:54:41 | sda: get_state
      Feb 29 10:54:41 | sda: path checker = tur (controller setting)
      Feb 29 10:54:41 | sda: state = running
      Feb 29 10:54:41 | sda: state = up
      Feb 29 10:54:41 | sda: state = running
      Feb 29 10:54:41 | sda: prio = alua (controller setting)
      Feb 29 10:54:41 | reported target port group is 0
      Feb 29 10:54:41 | aas = 02 [standby]
      Feb 29 10:54:41 | sda: alua prio = 1
      Feb 29 10:54:41 | 222910001555ba713: features = 1 queue_if_no_path (controller setting)
      Feb 29 10:54:41 | 222910001555ba713: no_path_retry = -2 (controller setting)
      Feb 29 10:54:41 | 222910001555ba713: update dev_loss_tmo to 2147483647

       

      Feb 29 10:54:41 | sdb: checking rport for /devices/pci0000:00/0000:00:07.0/0000:04:00.0/host0/port-0:1/expander-0:1/port-0:1:1/end_device-0:1:1/target0:0:1/0:0:1:0/block/sdb
      Feb 29 10:54:41 | sdb: Not a FC device
      Feb 29 10:54:41 | 222910001555ba713: verified path sdb dev_t 8:16
      Feb 29 10:54:41 | 222910001555ba713: update dev_loss_tmo to 2147483647

       

      Feb 29 10:54:41 | sda: checking rport for /devices/pci0000:00/0000:00:07.0/0000:04:00.0/host0/port-0:0/expander-0:0/port-0:0:1/end_device-0:0:1/target0:0:0/0:0:0:0/block/sda
      Feb 29 10:54:41 | sda: Not a FC device
      Feb 29 10:54:41 | 222910001555ba713: verified path sda dev_t 8:0
      Feb 29 10:54:41 | 222910001555ba713: pgfailback = -2 (controller setting)
      Feb 29 10:54:41 | 222910001555ba713: pgpolicy = group_by_prio (controller setting)
      Feb 29 10:54:41 | 222910001555ba713: selector = round-robin 0 (controller setting)
      Feb 29 10:54:41 | 222910001555ba713: features = 1 queue_if_no_path (controller setting)
      Feb 29 10:54:41 | 222910001555ba713: hwhandler = 1 alua (controller setting)
      Feb 29 10:54:41 | 222910001555ba713: rr_weight = 1 (internal default)
      Feb 29 10:54:41 | 222910001555ba713: minio = 100 (controller setting)
      Feb 29 10:54:41 | 222910001555ba713: no_path_retry = -2 (controller setting)
      Feb 29 10:54:41 | 222910001555ba713: pg_timeout = NONE (internal default)
      Feb 29 10:54:41 | 222910001555ba713: assembled map [1 queue_if_no_path 1 alua 2 1 round-robin 0 1 1 8:16 100 round-robin 0 1 1 8:0 100]

       

      Feb 29 10:54:41 | 222910001555ba713: set ACT_CREATE (map does not exist)
      Feb 29 10:54:41 | 222910001555ba713: addmap [0 83886080 multipath 1 queue_if_no_path 1 alua 2 1 round-robin 0 1 1 8:16 100 round-robin 0 1 1 8:0 100]

       

      Feb 29 10:54:41 | libdevmapper: ioctl/libdm-iface.c(1702): device-mapper: reload ioctl failed: Device or resource busy
      Feb 29 10:54:41 | 222910001555ba713: addmap [0 83886080 multipath 1 queue_if_no_path 1 alua 2 1 round-robin 0 1 1 8:16 100 round-robin 0 1 1 8:0 100]

       

      Feb 29 10:54:41 | libdevmapper: ioctl/libdm-iface.c(1702): device-mapper: reload ioctl failed: Device or resource busy
      Feb 29 10:54:42 | 222910001555ba713: domap (0) failure for create/reload map
      Feb 29 10:54:42 | 222910001555ba713: remove multipath map
      Feb 29 10:54:42 | sdb: orphaned
      Feb 29 10:54:42 | tur checker refcount 2
      Feb 29 10:54:42 | sda: orphaned
      Feb 29 10:54:42 | tur checker refcount 1
      Feb 29 10:54:42 | unloading const prioritizer
      Feb 29 10:54:42 | unloading alua prioritizer
      Feb 29 10:54:42 | unloading tur checker
      Feb 29 10:54:42 | unloading directio checker

       

       

      When we change the affinity in the IMS Web-Console from SCM #1 to SCM #2, the File-System is mounted READ-ONLY.

      So the failover does NOT work and our ERP-System would Crash.

       

      Output of "multipath -v2 -d"

          create: 222910001555ba713 undef Intel,Multi-Flex
      size=40G features='1 queue_if_no_path' hwhandler='1 alua' wp=undef
      |-+- policy='round-robin 0' prio=150 status=undef
      | `- 0:0:1:0 sdb 8:16 undef ready  running
      `-+- policy='round-robin 0' prio=1 status=undef
        `- 0:0:0:0 sda 8:0  undef ready  running

       

      Output of "cat /proc/scsi/scsi":

      Attached devices:
      Host: scsi0 Channel: 00 Id: 00 Lun: 00
        Vendor: Intel    Model: Multi-Flex       Rev: 0308
        Type:   Direct-Access                    ANSI  SCSI revision: 05
      Host: scsi0 Channel: 00 Id: 01 Lun: 00
        Vendor: Intel    Model: Multi-Flex       Rev: 0308
        Type:   Direct-Access                    ANSI  SCSI revision: 05
      Host: scsi1 Channel: 00 Id: 00 Lun: 00
        Vendor: Virtual  Model: CDROM            Rev: 1.00
        Type:   CD-ROM                           ANSI  SCSI revision: 00
      Host: scsi1 Channel: 00 Id: 00 Lun: 01
        Vendor: Virtual  Model: Floppy           Rev: 1.00
        Type:   Direct-Access                    ANSI  SCSI revision: 00

       

      When affinity is configured to SCM #1:

      sginfo -a /dev/sda gives detailed output

      sginfo -a /dev/sdb gives small output

       

       

      When affinity is configured to SCM #2:

      sginfo -a /dev/sda gives small output

      sginfo -a /dev/sdb gives detailed output

       

       

      What else has to be done to have a working multipath-configuration?