Direct IO fails when writting from CMA allocated buffer
by markbeta from LinuxQuestions.org on (#5JT5G)
Hello, I'm using kernel 5.4.0 and running it on ARM64 CPU.
I am trying to optimize writing to an NVMe disk by opening a file with the O_DIRECT flag. The source buffer is allocated within a kernel module (u-dma-buf) using dma_alloc_coherent() and mmaped to userspace program where it is used in a write() call.
Underneath the dma_alloc_coherent() API is a CMA (Contiguous Memory Allocator) that allocates contiguous locations in physical memory that are aligned to 4096 boundary (I confirmed that by printing the address) and I also made sure that the pointer that the buffer is mmaped to is also aligned to 4096 boundary in virtual memory of the process.
However, the write() call returns a value less than 0, and when printing errno message it says "Bad address". Why would this be the case?
What is more strange is that I only can see this error with ext4 and xfs filesystems. If I perform the same write on a disk formatted using btrfs filesystem, everything works as expected.
Also, I'm performing writes with a size multiple of block size.
Thank you in advance.
I am trying to optimize writing to an NVMe disk by opening a file with the O_DIRECT flag. The source buffer is allocated within a kernel module (u-dma-buf) using dma_alloc_coherent() and mmaped to userspace program where it is used in a write() call.
Underneath the dma_alloc_coherent() API is a CMA (Contiguous Memory Allocator) that allocates contiguous locations in physical memory that are aligned to 4096 boundary (I confirmed that by printing the address) and I also made sure that the pointer that the buffer is mmaped to is also aligned to 4096 boundary in virtual memory of the process.
However, the write() call returns a value less than 0, and when printing errno message it says "Bad address". Why would this be the case?
What is more strange is that I only can see this error with ext4 and xfs filesystems. If I perform the same write on a disk formatted using btrfs filesystem, everything works as expected.
Also, I'm performing writes with a size multiple of block size.
Thank you in advance.