Transmit Packet Steering (XPS) issue
by amitbern2 from LinuxQuestions.org on (#5QTEQ)
Hi,
I have a network driver (driver interface eth1) with 12 TX queues and 12 CPU cores
I am using Amazon Linux 2: 4.14.243-185.433.amzn2.x86_64
I am trying to use XPS to steer TX packets for each queue to a single core
To config the XPS, I run the following:
echo 1 > /sys/class/net/eth1/queues/tx-0/xps_cpus
echo 2 > /sys/class/net/eth1/queues/tx-1/xps_cpus
echo 4 > /sys/class/net/eth1/queues/tx-2/xps_cpus
echo 8 > /sys/class/net/eth1/queues/tx-3/xps_cpus
echo 10 > /sys/class/net/eth1/queues/tx-4/xps_cpus
echo 20 > /sys/class/net/eth1/queues/tx-5/xps_cpus
echo 40 > /sys/class/net/eth1/queues/tx-6/xps_cpus
echo 80 > /sys/class/net/eth1/queues/tx-7/xps_cpus
echo 100 > /sys/class/net/eth1/queues/tx-8/xps_cpus
echo 200 > /sys/class/net/eth1/queues/tx-9/xps_cpus
echo 400 > /sys/class/net/eth1/queues/tx-10/xps_cpus
echo 800 > /sys/class/net/eth1/queues/tx-11/xps_cpus
1. using iperf2, when sending single flow (single queue) it doesn't seem to use the xps configuration above, every run, a different CPU will start and transmit the flow even if source/dest IP and ports stay the same.
it also doesn't seem to choose the same queue every time, any reason why?
command:
iperf -c 1.2.3.4 -p 5001 -B 5.6.7.8:5002
2. Tried also to bind the iperf to specific core and it still didn't work:
numactl --physcpubind=0 iperf -c 1.2.3.4 -p 5001 -B 5.6.7.8:5002
The driver is calling __netdev_pick_tx to choose the queue index:
https://elixir.bootlin.com/linux/v4....re/dev.c#L3196
3. Any other thing needs to be done to enable XPS? how to check if CONFIG_XPS is enabled?
Any help will be appreciated.
Thanks
I have a network driver (driver interface eth1) with 12 TX queues and 12 CPU cores
I am using Amazon Linux 2: 4.14.243-185.433.amzn2.x86_64
I am trying to use XPS to steer TX packets for each queue to a single core
To config the XPS, I run the following:
echo 1 > /sys/class/net/eth1/queues/tx-0/xps_cpus
echo 2 > /sys/class/net/eth1/queues/tx-1/xps_cpus
echo 4 > /sys/class/net/eth1/queues/tx-2/xps_cpus
echo 8 > /sys/class/net/eth1/queues/tx-3/xps_cpus
echo 10 > /sys/class/net/eth1/queues/tx-4/xps_cpus
echo 20 > /sys/class/net/eth1/queues/tx-5/xps_cpus
echo 40 > /sys/class/net/eth1/queues/tx-6/xps_cpus
echo 80 > /sys/class/net/eth1/queues/tx-7/xps_cpus
echo 100 > /sys/class/net/eth1/queues/tx-8/xps_cpus
echo 200 > /sys/class/net/eth1/queues/tx-9/xps_cpus
echo 400 > /sys/class/net/eth1/queues/tx-10/xps_cpus
echo 800 > /sys/class/net/eth1/queues/tx-11/xps_cpus
1. using iperf2, when sending single flow (single queue) it doesn't seem to use the xps configuration above, every run, a different CPU will start and transmit the flow even if source/dest IP and ports stay the same.
it also doesn't seem to choose the same queue every time, any reason why?
command:
iperf -c 1.2.3.4 -p 5001 -B 5.6.7.8:5002
2. Tried also to bind the iperf to specific core and it still didn't work:
numactl --physcpubind=0 iperf -c 1.2.3.4 -p 5001 -B 5.6.7.8:5002
The driver is calling __netdev_pick_tx to choose the queue index:
https://elixir.bootlin.com/linux/v4....re/dev.c#L3196
3. Any other thing needs to be done to enable XPS? how to check if CONFIG_XPS is enabled?
Any help will be appreciated.
Thanks