Multicast not working Intel X710-DA4

[jhuyb]

We have a couple of Intel X710-DA4 quadport 10Gbit cards but it seems multicasting doesn't work from or to them.

I installed the latest firmware and the 1.2.38 driver for Linux on CentOS 6/7 but whatever I try it only sends out unicast according to tcpdump or omping.

I also tried disabling the rp filters in Linux and manually adding the multicast route to the interface.

On the same switch I also have an NC550 card in a server and this one works fine with multicast.

[SYeo3]

Hi jimmy1987,

Thank you for posting.

Please provide more information regarding your environment:

1. Server model:

2. Type of Connection for X710 and NC550 - Is it DA, SR or LR?

Looking forward to your reply.

Sincerely,

Sandy

[jhuyb]

Hi Sandy,

The server model is HP DL360P G8.

As for the NIC connectivity, they are all connected by SR fibers.

[SYeo3]

Hi jimmy1987,

Thanks for providing the details.

We would like to check your network configuration. Please send us the output of -

1. ethtool -i eth(x)

2. ifconfig eth(x)

Hope to hear from you again. Sorry for wasn't able to include in the my request last time. Thank you for your patience and understanding.

Sincerely,

Sandy

[rguo6]

Hi,

seeing same issue on my server. X710-DA4, cannot receive the multicast packets.

server

| |

switch

two connections between server and switch, one port sends mcast-packets, another port cannot receive the packets. broadcast is fine.

[root@test ~]# ethtool -i eth12

driver: i40e

version: 1.2.38

firmware-version: f4.22.27454 a1.2 n4.26 e152d

bus-info: 0000:84:00.0

[root@test ~]# ethtool -i eth13

driver: i40e

version: 1.2.38

firmware-version: f4.22.27454 a1.2 n4.26 e152d

bus-info: 0000:84:00.1

[root@test ~]# ifconfig eth12

eth12 Link encap:Ethernet HWaddr 68:05:CA:32:53:30

inet6 addr: fe80::6a05:caff:fe32:5330/64 Scope:Link

UP BROADCAST MULTICAST MTU:9000 Metric:1

RX packets:308 errors:0 dropped:0 overruns:0 frame:0

TX packets:763 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:18690 (18.2 KiB) TX bytes:32511 (31.7 KiB)

[root@test ~]# ifconfig eth13

eth13 Link encap:Ethernet HWaddr 68:05:CA:32:53:31

inet6 addr: fe80::6a05:caff:fe32:5331/64 Scope:Link

UP BROADCAST MULTICAST MTU:9000 Metric:1

RX packets:310 errors:0 dropped:0 overruns:0 frame:0

TX packets:39 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:18868 (18.4 KiB) TX bytes:3820 (3.7 KiB)

[SYeo3]

Hi RuiGuo,

Thanks for providing the info. We'll check on this.

Sincerely,

Sandy

[rguo6]

Hi Sandy,

Really appreciate if you can resolve it ASAP. I have lots of businesses are standing by.

Regards,

Rui

[jhuyb]

Hi Sandy,

My appologies for not responding earlier.

[root@172.17.0.20 ~]# ethtool -i app1

driver: i40e

version: 1.2.38

firmware-version: f4.33.31377 a1.2 n4.42 e191d

bus-info: 0000:07:00.0

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: yes

[root@172.17.0.20 ~]# ethtool -i app2

driver: i40e

version: 1.2.38

firmware-version: f4.33.31377 a1.2 n4.42 e191d

bus-info: 0000:07:00.2

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: yes

[root@172.17.0.20 ~]# ethtool -i eth5

driver: i40e

version: 1.2.38

firmware-version: f4.33.31377 a1.2 n4.42 e191d

bus-info: 0000:07:00.1

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: yes

[root@172.17.0.20 ~]# ethtool -i eth7

driver: i40e

version: 1.2.38

firmware-version: f4.33.31377 a1.2 n4.42 e191d

bus-info: 0000:07:00.3

supports-statistics: yes

supports-test: yes

supports-eeprom-access: yes

supports-register-dump: yes

supports-priv-flags: yes

app1 Link encap:Ethernet HWaddr 68:05:CA:2F:7C:E0 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:3 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:230 (230.0 b)app2 Link encap:Ethernet HWaddr 68:05:CA:2F:7C:E0 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:3 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 b) TX bytes:238 (238.0 b)eth5 Link encap:Ethernet HWaddr 68:05:CA:2F:7C:E1 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:1350839 errors:0 dropped:0 overruns:0 frame:0 TX packets:860761 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:276842903 (264.0 MiB) TX bytes:167083402 (159.3 MiB)eth7 Link encap:Ethernet HWaddr 68:05:CA:2F:7C:E1 UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1 RX packets:1350492 errors:0 dropped:0 overruns:0 frame:0 TX packets:860762 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:276657923 (263.8 MiB) TX bytes:167104193 (159.3 MiB)

For us counts the same as for RuiGuo, we have businesses on standby because of this issue currently.

[SYeo3]

Hi all,

Thank you for providing the details.

Upon checking the report, it appears that the adapter firmware is not on the latest version. Please update the adapter's firmware. You may download the firmware update package here - https://downloadcenter.intel.com/download/24769/NVM-Update-Utility-for-Intel-Ethernet-Converged-Network-Adapter-XL710-X710-Series NVM Update Utility

In case you still encounter the same issue after updating the firmware, please provide us the model of the module that you are using.

Looking forward to your test results.

Sincerely,

Sandy

[rguo6]

Hi Sandy,

After firmware update, mine is the same as jimmy's

[root@test ~]# ethtool -i eth12

driver: i40e

version: 1.2.38

firmware-version: f4.33.31377 a1.2 n4.42 e191d

bus-info: 0000:84:00.0

I am downloading it from your providing URL.

See all the info i have from the NIC:

x710da4fhblk

Regards

Rui

[jhuyb]

Hi Sandy,

We already had the latest version, not sure how I can find that number?

The card itself is an Intel X710DA4

[Brian_J_Intel1]

jimmy1987, I have run in to some similar behavior when running CentOS 7 and RHEL 7 and found that the issue was the firewalld was blocking multicast and other traffic. I had a VXLAN configuration using netperf to test throughput. I could ping the physical network port but netperf traffic would not work and the VXLAN interfaces could not ping or pass netperf traffic either. What I finally found was that the port was set to Public which blocks specific ports but would allow pings between the servers. Since I am in a lab environment I stopped the firewalld service and immediately was able to receive VXLAN and netperf traffic. I stopped the service with 'systemctl stop firewalld' command. One other place to look is iptables to see if it is filtering the traffic as well. One other thing to test is to run tcpdump on both servers and on the switch port to see if the Tx traffic to the other server is making it out of the server and through the switch ports. Just some thoughts to try, if you have not already, to try to get passed the issue.

[rguo6]

Hi thehevy,

It's not the firewalld blocking. Why my other NICs are good to receive multicast pkts?

Can you work around with my test?

server

| |

switch

left connection, server is good to send out multicast pkts.

switches are good to receive multicast pkts and flood the pkts out.

but the second connections on the server side, doesn't see any incoming multicast pks.

broadcast, ping are good.

other NICs on my server are good to receive multicast pkts.

[SYeo3]

Hi all,

Thanks for your updates. I'll check on this.

Sincerely,

Sandy

[jhuyb]

Unfortunately that was not it as we tested before with centos 6 and 7 and with and without firewall / iptables.

It seems it can transmit them but not receive them.

[rguo6]

Hi Sandy,

Any update on this? Thanks.

Regards,

Rui

[SYeo3]

Hi RuiGuo, thanks for following up, we are checking on this.

Hi jimmy1987, I understand you are using SR module, please confirm if you are using this model E10GSFPSR.

Sincerely,

Sandy

[rguo6]

Hi Sandy,

Any update on this? I really have lot servers are standing by.

Please don't stop me to order more this kind of NICs.

Regards,

Rui

[Brian_J_Intel1]

I am not sure if this will help but you can try running the following commend on each port:

bridge link set dev em1 hwmode vepa

Replace em1 with the specific device ids on each port.

After you run this command for each port you can verify that all ports are now in VEPA mode by use the following command:

dmesg | grep i40e

you should see all the ports reporting some like the following:

[1355998.519860] i40e 0000:04:00.1: enabling bridge mode: VEPA

This setting will change the HW L2 switch on each port from VEB mode which enables traffic from the VSIs to be able to loopback to other VSI on the port to VEPA mode which disables the loopback function. The loopback function is not used in most OpenStack deployments since the OVS does the loopback to other VMs running on the same host. The primary use of VEB mode is when using SR-IOV with VFs directly assigned to the VMs so a VM can communicate with other VMs on the same host. Putting the HW L2 switch in VEPA mode does not require an external VEPA enabled switch unless using SR-IOV with VFs.

BTW, in the next version of the driver we will set the default HW L2 switch to non-loopback (VEPA mode) by default since loopback in not required for most deployments.

[jhuyb]

Hi thehevy,

I understand you also work for intel? I will try your suggestion tomorrow if I can find a bit of time. In case it works do we need to set it at every reboot or will it stick?

@sandy That is correct, that is the type of optic we use.

[Brian_J_Intel1]

Yes, I work at Intel in the Network Division as a Solutions Architect. I have been focusing on Network Virtualization Overlays (NVO) and Network Functions Virtualization (NFV) using our 10GbE, 40GbE and our products that are still in development. I try to monitor these post and provide input / suggestions based on work that I am doing in my lab.