How do we find how LLM organized; break up due to VRAM
by rico001 from LinuxQuestions.org on (#6DBFE)
An LLM has parts for helping with programming. Couldn't we seperate out. Everyday speech doesn't use programming or that data... So could we seperate out the parts of the LLM out like a human brain sections (medula... cerebrum) using software such as h2o studio software?
How do we make it more usable by low end computers? looking over orca free willy 2 papers might give give ideas. I think it might help to use multiple computers across networks to run a single LLM. Our human lanuage uses a whole system, not just text. It uses sight, smell, hearing, and checks the feedback it hears or receives from itself (example). opinion: TV with Nelson Ratings is like a quantum computer system or has some quantum properties, perhaps? Thanks for any help.
Example: want to fit 12Gb model into 8G vram? Example llm: Falcon 7 Billion parameters. Apache 2 license, blog with companies info... https://huggingface.co/blog/falcon Thanks for feedback.
How do we make it more usable by low end computers? looking over orca free willy 2 papers might give give ideas. I think it might help to use multiple computers across networks to run a single LLM. Our human lanuage uses a whole system, not just text. It uses sight, smell, hearing, and checks the feedback it hears or receives from itself (example). opinion: TV with Nelson Ratings is like a quantum computer system or has some quantum properties, perhaps? Thanks for any help.
Example: want to fit 12Gb model into 8G vram? Example llm: Falcon 7 Billion parameters. Apache 2 license, blog with companies info... https://huggingface.co/blog/falcon Thanks for feedback.