NFS performance deterioration going from 4.18 kernel to 5.7 by 4x
by perf from LinuxQuestions.org on (#5DD2A)
Setup:
NVMe SSD formatted with ext4 filesystem.
The ext4 mount point is exported and mounted remotely on another node.
Both the NFS server and client were initially on 4.18 kernel and later upgraded to 5.7.12, with no other changes to the setup.
NFS version: v3
CPU:
32 cores
model name: Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz
Memory: 256G
Mount options:
rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys ,mountaddr=x.x.x.x,mountvers=3,mountport=41611,mountproto=udp,local_lock=none,addr=x.x.x.x
FIO command:
fio --ioengine=libaio --blocksize=16k -directory=/mnt/nfs_nvme --randrepeat=1 --gtod_reduce=0 --name=tm --buffer_compress_percentage=0 --group_reporting=1 --numjobs=1 --readwrite=randwrite --size=50G --direct=1 --iodepth=128
Result:
Kernel ---> IOPS
4.18 -----> 51800.00
5.7.12 ---> 10658.38
Based on disk stats I observed the write amplification is 3x.
Has anybody seen this issue?
NVMe SSD formatted with ext4 filesystem.
The ext4 mount point is exported and mounted remotely on another node.
Both the NFS server and client were initially on 4.18 kernel and later upgraded to 5.7.12, with no other changes to the setup.
NFS version: v3
CPU:
32 cores
model name: Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz
Memory: 256G
Mount options:
rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys ,mountaddr=x.x.x.x,mountvers=3,mountport=41611,mountproto=udp,local_lock=none,addr=x.x.x.x
FIO command:
fio --ioengine=libaio --blocksize=16k -directory=/mnt/nfs_nvme --randrepeat=1 --gtod_reduce=0 --name=tm --buffer_compress_percentage=0 --group_reporting=1 --numjobs=1 --readwrite=randwrite --size=50G --direct=1 --iodepth=128
Result:
Kernel ---> IOPS
4.18 -----> 51800.00
5.7.12 ---> 10658.38
Based on disk stats I observed the write amplification is 3x.
Has anybody seen this issue?