4 Replies Latest reply on Oct 18, 2017 8:06 AM by PavlosP

    X710 10GbE SFP card, i40e and NIC Link is Down due to DCB init failed and tx_timeout

    PavlosP

      Hi,

       

      We run CentOS 7.4 with kernel 4.9.x on HP hardware and noticed that few server got their network interfaces marked down by the kernel. In the logs we saw a lot of

      reports for DCB init failed -53, disabled, TX driver issue detected, PF reset issued and eth0: tx_timeout: VSI_seid followed by marking the link down.

       

      Here is the full log:

      2017-10-04T15:50:29.908202+02:00kernel: i40e 0000:04:00.1 eth0: tx_timeout recovery level 1, hung_queue 11

      2017-10-04T15:50:30.061686+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:50:30.061693+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:50:36.085291+02:00kernel: i40e 0000:04:00.1 eth0: tx_timeout: VSI_seid: 388, Q 2, NTC: 0x20, HWB: 0x20, NTU: 0x100, TAIL: 0x100, INT: 0x0

      2017-10-04T15:50:36.085295+02:00kernel: i40e 0000:04:00.1 eth0: tx_timeout recovery level 2, hung_queue 2

      2017-10-04T15:50:39.328928+02:00kernel: i40e 0000:04:00.0: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:50:39.328936+02:00kernel: i40e 0000:04:00.0: DCB init failed -53, disabled

      2017-10-04T15:50:39.637232+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:50:39.637237+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:50:40.111808+02:00kernel: i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-04T15:50:40.788697+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:50:40.788702+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:50:46.839994+02:00kernel: i40e 0000:04:00.1 eth0: tx_timeout: VSI_seid: 388, Q 11, NTC: 0x54, HWB: 0x54, NTU: 0xed, TAIL: 0xed, INT: 0x1

      2017-10-04T15:50:46.839998+02:00kernel: i40e 0000:04:00.1 eth0: tx_timeout recovery level 3, hung_queue 11

      2017-10-04T15:50:50.119447+02:00kernel: i40e 0000:04:00.0: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:50:50.119455+02:00kernel: i40e 0000:04:00.0: DCB init failed -53, disabled

      2017-10-04T15:50:50.301798+02:00kernel: i40e 0000:04:00.0 eth1: NIC Link is Down

      2017-10-04T15:50:50.423744+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:50:50.423752+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:50:50.600812+02:00kernel: i40e 0000:04:00.1 eth0: NIC Link is Down

      2017-10-04T15:50:50.764799+02:00kernel: i40e 0000:04:00.1 eth0: NIC Link is Up 10 Gbps Full Duplex, Flow Control: None

      2017-10-04T15:50:53.234804+02:00kernel: i40e 0000:04:00.0 eth1: NIC Link is Up 10 Gbps Full Duplex, Flow Control: None

      2017-10-04T15:51:17.201808+02:00kernel: i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-04T15:51:17.783439+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:51:17.783447+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:51:18.392805+02:00kernel: i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-04T15:51:18.814970+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:51:18.814978+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:51:19.436807+02:00kernel: i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-04T15:51:19.767258+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:51:19.767265+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:51:20.440800+02:00kernel: i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-04T15:51:20.793083+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:51:20.793091+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:51:21.471805+02:00kernel: i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-04T15:51:21.810807+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:51:21.810811+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:51:22.468707+02:00kernel: i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-04T15:51:22.772829+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:51:22.772833+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:51:23.411802+02:00kernel: i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-04T15:51:23.796867+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:51:23.796872+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:51:24.440800+02:00kernel: i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-04T15:51:24.758945+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:51:24.758950+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:51:25.411806+02:00kernel: i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-04T15:51:25.782778+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:51:25.782781+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:51:26.417804+02:00kernel: i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-04T15:51:26.804559+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:51:26.804568+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:51:27.448800+02:00kernel: i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-04T15:51:27.765882+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:51:27.765889+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:51:33.187800+02:00kernel: i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-04T15:51:33.784824+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:51:33.784827+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:51:34.340383+02:00kernel: i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-04T15:51:34.810411+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:51:34.810415+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:51:35.350800+02:00kernel: i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-04T15:51:35.769594+02:00kernel: i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-04T15:51:35.769600+02:00kernel: i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-04T15:51:36.404803+02:00kernel: i40e 0000:04:00.1: TX driver issue detected, PF reset issued

       

      The firmware  5.60 0x800033b1 1.1752.0.

       

      Because this issue occurred many times we decided to downgrade the firmware to 5.60 0x80002dac 1.1618.0.

      We have been running this firmware for 20hours and so far the issue hasn't reoccurred. But, we still see reports for DCB init failed, here is a part of the log:

      2017-10-06T07:36:04.508245+02:00 kernel: [60714.891133] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:04.508253+02:00 kernel: [60714.941154] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:07.485087+02:00 kernel: [60717.910685] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:08.544822+02:00 kernel: [60718.922177] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:08.544826+02:00 kernel: [60718.976268] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:09.662086+02:00 kernel: [60720.087544] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:10.526650+02:00 kernel: [60720.906953] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:10.526657+02:00 kernel: [60720.957855] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:12.127091+02:00 kernel: [60722.553258] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:12.509188+02:00 kernel: [60722.891523] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:12.509193+02:00 kernel: [60722.941451] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:14.542083+02:00 kernel: [60724.968613] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:15.515736+02:00 kernel: [60725.898114] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:15.515742+02:00 kernel: [60725.948320] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:17.054084+02:00 kernel: [60727.482217] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:17.499895+02:00 kernel: [60727.881722] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:17.499899+02:00 kernel: [60727.931928] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:23.073089+02:00 kernel: [60733.498949] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:23.519433+02:00 kernel: [60733.898115] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:23.519440+02:00 kernel: [60733.950592] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:23.750083+02:00 kernel: [60734.175558] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:24.542845+02:00 kernel: [60734.922501] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:24.542851+02:00 kernel: [60734.973989] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:25.804083+02:00 kernel: [60736.229381] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:26.527743+02:00 kernel: [60736.906173] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:26.527750+02:00 kernel: [60736.958286] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:26.761082+02:00 kernel: [60737.185843] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:27.549959+02:00 kernel: [60737.930406] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:27.549965+02:00 kernel: [60737.981294] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:28.730084+02:00 kernel: [60739.156229] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:29.127699+02:00 kernel: [60739.509889] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:29.127706+02:00 kernel: [60739.559362] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:30.195079+02:00 kernel: [60740.620866] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:30.494200+02:00 kernel: [60740.874560] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:30.494206+02:00 kernel: [60740.924833] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:32.206081+02:00 kernel: [60742.632196] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:32.542615+02:00 kernel: [60742.922219] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:32.542621+02:00 kernel: [60742.974168] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:33.418078+02:00 kernel: [60743.842578] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:34.523784+02:00 kernel: [60744.905169] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:34.523792+02:00 kernel: [60744.955104] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:35.379092+02:00 kernel: [60745.805674] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:35.546831+02:00 kernel: [60745.928123] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:35.546837+02:00 kernel: [60745.978287] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:37.173085+02:00 kernel: [60747.597806] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:37.505649+02:00 kernel: [60747.884064] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:37.505655+02:00 kernel: [60747.935578] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:38.903089+02:00 kernel: [60749.323330] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:39.518842+02:00 kernel: [60749.897537] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:39.518849+02:00 kernel: [60749.949198] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:40.001094+02:00 kernel: [60750.425300] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:40.294976+02:00 kernel: [60750.672969] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:40.294982+02:00 kernel: [60750.725401] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:41.763084+02:00 kernel: [60752.187674] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:42.522862+02:00 kernel: [60752.903896] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:42.522869+02:00 kernel: [60752.953498] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:43.326092+02:00 kernel: [60753.751729] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:43.548946+02:00 kernel: [60753.929795] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:43.548952+02:00 kernel: [60753.980056] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:45.499095+02:00 kernel: [60755.925140] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:45.750231+02:00 kernel: [60756.131802] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:45.750238+02:00 kernel: [60756.181360] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:47.697084+02:00 kernel: [60758.122317] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:48.538715+02:00 kernel: [60758.919704] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:48.538721+02:00 kernel: [60758.969756] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-06T07:36:48.774079+02:00 kernel: [60759.199913] i40e 0000:04:00.1: TX driver issue detected, PF reset issued

      2017-10-06T07:36:49.499160+02:00 kernel: [60759.880666] i40e 0000:04:00.1: Query for DCB configuration failed, err I40E_ERR_ADMIN_QUEUE_ERROR aq_err I40E_AQ_RC_EPERM

      2017-10-06T07:36:49.499168+02:00 kernel: [60759.930263] i40e 0000:04:00.1: DCB init failed -53, disabled

       

      Furthermore, we noticed a kernel crash, see below:

      2017-10-05T17:54:26.526390+02:00 kernel: [11418.560560] i40e 0000:04:00.1: DCB init failed -53, disabled

      2017-10-05T17:54:32.481310+02:00 kernel: [11424.483864] ------------[ cut here ]------------

      2017-10-05T17:54:32.481314+02:00 kernel: [11424.504705] WARNING: CPU: 3 PID: 0 at net/sched/sch_generic.c:316 dev_watchdog+0x232/0x240

      2017-10-05T17:54:32.481315+02:00 kernel: [11424.541886] NETDEV WATCHDOG: north (i40e): transmit queue 11 timed out

      2017-10-05T17:54:33.507030+02:00 kernel: [11424.571387] Modules linked in: sctp_diag sctp dccp_diag dccp unix_diag udp_diag tcp_diag inet_diag 8021q garp mrp xfs libcrc32c loop vfat fat sb_edac edac_core x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd iTCO_wdt intel_cstate iTCO_vendor_support intel_rapl_perf i2c_i801 lpc_ich pcspkr mfd_core hpwdt hpilo i2c_smbus fjes sg wmi ipmi_si acpi_power_meter ipmi_msghandler ioatdma shpchp ip_tables ext4 jbd2 mbcache sd_mod mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ixgbe ttm mdio i40e dca tg3 ptp crc32c_intel drm pps_core hpsa scsi_transport_sas dm_mirror dm_region_hash dm_log dm_mod

      2017-10-05T17:54:33.507035+02:00 kernel: [11424.874760] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.9.52-1.booking.el7.x86_64 #1

      2017-10-05T17:54:33.507036+02:00 kernel: [11424.910445] Hardware name: HP ProLiant DL380 Gen9/ProLiant DL380 Gen9, BIOS P89 04/25/2017

      2017-10-05T17:54:33.507038+02:00 kernel: [11424.947905]  ffff880c4fcc3db0 ffffffff81363cdc ffff880c4fcc3e00 0000000000000000

      2017-10-05T17:54:33.507043+02:00 kernel: [11424.981294]  ffff880c4fcc3df0 ffffffff81082441 0000013c00000246 000000000000000b

      2017-10-05T17:54:33.507045+02:00 kernel: [11425.014606]  ffff880c48743000 0000000000000040 ffff88184623cf40 0000000000000003

      2017-10-05T17:54:33.507045+02:00 kernel: [11425.047951] Call Trace:

      2017-10-05T17:54:33.507047+02:00 kernel: [11425.059172]  <IRQ>

      2017-10-05T17:54:33.507047+02:00 kernel: [11425.067975]  [<ffffffff81363cdc>] dump_stack+0x63/0x87

      2017-10-05T17:54:33.507048+02:00 kernel: [11425.091645]  [<ffffffff81082441>] __warn+0xd1/0xf0

      2017-10-05T17:54:33.507050+02:00 kernel: [11425.113566]  [<ffffffff810824bf>] warn_slowpath_fmt+0x5f/0x80

      2017-10-05T17:54:33.507051+02:00 kernel: [11425.139919]  [<ffffffff81657882>] dev_watchdog+0x232/0x240

      2017-10-05T17:54:33.507051+02:00 kernel: [11425.165073]  [<ffffffff81657650>] ? dev_deactivate_queue.constprop.27+0x60/0x60

      2017-10-05T17:54:33.507052+02:00 kernel: [11425.197971]  [<ffffffff810f4d45>] call_timer_fn+0x35/0x120

      2017-10-05T17:54:33.507052+02:00 kernel: [11425.222648]  [<ffffffff810f59d6>] run_timer_softirq+0x1f6/0x4b0

      2017-10-05T17:54:33.507053+02:00 kernel: [11425.249315]  [<ffffffff810fd8eb>] ? ktime_get+0x3b/0xb0

      2017-10-05T17:54:33.507053+02:00 kernel: [11425.272507]  [<ffffffff81053006>] ? lapic_next_deadline+0x26/0x30

      2017-10-05T17:54:33.507055+02:00 kernel: [11425.299879]  [<ffffffff817620a9>] __do_softirq+0xc9/0x26d

      2017-10-05T17:54:33.507056+02:00 kernel: [11425.325941]  [<ffffffff81088929>] irq_exit+0xd9/0xf0

      2017-10-05T17:54:33.507056+02:00 kernel: [11425.348778]  [<ffffffff81761ef2>] smp_apic_timer_interrupt+0x42/0x50

      2017-10-05T17:54:33.507057+02:00 kernel: [11425.377940]  [<ffffffff817610ac>] apic_timer_interrupt+0x8c/0xa0

      2017-10-05T17:54:33.507057+02:00 kernel: [11425.405474]  <EOI>

      2017-10-05T17:54:33.507058+02:00 kernel: [11425.414086]  [<ffffffff8175ee61>] ? poll_idle+0x31/0x5d

      2017-10-05T17:54:33.507060+02:00 kernel: [11425.437584]  [<ffffffff815d9b2d>] cpuidle_enter_state+0x9d/0x260

      2017-10-05T17:54:33.507061+02:00 kernel: [11425.464852]  [<ffffffff815d9d27>] cpuidle_enter+0x17/0x20

      2017-10-05T17:54:33.507061+02:00 kernel: [11425.489937]  [<ffffffff810c9ab3>] call_cpuidle+0x23/0x40

      2017-10-05T17:54:33.507062+02:00 kernel: [11425.514326]  [<ffffffff810c9d29>] cpu_startup_entry+0x159/0x250

      2017-10-05T17:54:33.507062+02:00 kernel: [11425.541202]  [<ffffffff81051a04>] start_secondary+0x154/0x190

      2017-10-05T17:54:33.507063+02:00 kernel: [11425.567550] ---[ end trace 1afc42121276ab06 ]---

       

      I need to mention that we have disabled lldp support in the kernel as we run lldpd daemon on our servers.

      Do you know if the above issue and those errors are fixed in the latest firmware?

       

      Cheers,

      Pavlos