LFS for Alt Target Architectures

I'm wanting a second opinion on this before I start dedicating my time on this. Anyways, I'm wanting to go through the LFS book however instead of compiling for x86_64 / amd64 I would be compiling for aarch64 but using a x86_64 host. I would also like to do PowerPC, RISC-V, etc. but I want to prove out I can do it for aarch64. So I've figured out how to chroot once I get past ch. 5 (Basically using qemu-system / binfmt to chroot into a foreign filesystem.). So the only question I have, is it possible to build ch. 5 for aarch64? I would assume all I would have to do is set LFS_TGT to "aarch64-lfs-linux-gnu". (Plus some small deviations from the book.)

I know I will have to deviate a bit in chapter 5. due to when you compile gcc you will have to change some flags around to get gcc compiled for aarch64.

Note I've already ventured into cross compiling but it got to the point where it feels janky and there's a worry I might contaminate the sysroot / chroot with host libraries.