Running Local LLMs on Aggregated Physical Resources by Chaining Apple Mac Silicons
arslan writes:
Anyone that has ran local LLMs would know that the size of the model is limited by your physical resource and typically the smaller the LLM the less productive it is. The prevailing option, without doing some work parallelisation, is to run larger LLMs locally is to increase resources, especially upgrading the GPU with bigger memory or if you're on a Mac with Unified Memory upgrade/swap for a bigger one.
Now, with Mac silicon, one can cluster a bunch of Macs with Thunderbolt 5 (TB5) and use their proprietary protocol RDMA over it, instead of IP over Thunderbolt, and get blazing speed, enough to treat the cluster like a cluster of Nvidia GPUs.
Check out the experiment done by this bloke where he setup a four-machine cluster to run trillion-parameter models locally at $50,000 versus $780,000+ for equivalent Nvidia H100 infrastructure.: https://www.sean-weldon.com/blog/2026-06-16-clustering-mac-studios-for-local-ai-apple-rdma-over-thunderbolt-5
Read more of this story at SoylentNews.