Karpathy's LLM OS, what are your thoughts?
by haghiri75 from LinuxQuestions.org on (#6QEGS)
Andrej Karpathy talked about an operating system fully controlled by a large language model, and he provided some good examples and a chart explaining what he has in mind visually. If you haven't watched his video, you can find it here. What he suggests about an LLM OS, is pretty similar to the concept of the most modern operating systems (at least from my perspective, as a person with experience in distribution development) and let's be honest, it seems really cool.
But I have some challenges in my mind for this particular type of OS/distro. I just put a list of my mind's challenges here, and if you have any ideas/solutions or even questions, I am glad to hear and discuss.
First challenge in my mind is the popularity of the new operating systems amongst people. An LLM OS is usually something targeting average desktop users and not server/enterprise users. In this market, even Apple has around 20 percent share and most of that share is just taken over by Microsoft. This challenge in my opinion can be solved by making this LLM OS for handheld devices such as tablets or phones, but again it may face the popularity problem.
Second, it's the resource/hardware problem. Not all people have high-end NVIDIA GPU's on their own computers. I tried a simple model (Zephyr 7B) on a 2080 Ti and also my personal computer (no nvidia gpu) and I can say that even with my personal laptop with an i7 processor and 16 gbs of ram, it was pretty slow. It may have a solution, but in my personal opinion, it is not very welcome by GNU/Linux community which will be the next challenge of mine!
Third, if we provide an API and the interaction between LLM and users become online, there will be a lot of rightful privacy concerns. I guess for solving this challenge, we need to ask users to be sure what we're sending over the network to the API. It also makes the operating system highly dependent on the internet.
I had these challenges in my mind, otherwise I think I personally would like to have an operating system like this on my computer!
But I have some challenges in my mind for this particular type of OS/distro. I just put a list of my mind's challenges here, and if you have any ideas/solutions or even questions, I am glad to hear and discuss.
First challenge in my mind is the popularity of the new operating systems amongst people. An LLM OS is usually something targeting average desktop users and not server/enterprise users. In this market, even Apple has around 20 percent share and most of that share is just taken over by Microsoft. This challenge in my opinion can be solved by making this LLM OS for handheld devices such as tablets or phones, but again it may face the popularity problem.
Second, it's the resource/hardware problem. Not all people have high-end NVIDIA GPU's on their own computers. I tried a simple model (Zephyr 7B) on a 2080 Ti and also my personal computer (no nvidia gpu) and I can say that even with my personal laptop with an i7 processor and 16 gbs of ram, it was pretty slow. It may have a solution, but in my personal opinion, it is not very welcome by GNU/Linux community which will be the next challenge of mine!
Third, if we provide an API and the interaction between LLM and users become online, there will be a lot of rightful privacy concerns. I guess for solving this challenge, we need to ask users to be sure what we're sending over the network to the API. It also makes the operating system highly dependent on the internet.
I had these challenges in my mind, otherwise I think I personally would like to have an operating system like this on my computer!