Optimise AI
Large Language Models. LLMs, Generative AI, whatever you want to call it. The trend is disrupting economies and changing the tech industry with trillions(?) in investor funding.
Imperfect has echoed many times that there are good uses of LLms. Additionally he has combated many rants in reply posts with that sentiment. Something I have gathered from a lot of his writing is that even though there are bad uses of a tool that doesn't mean we cant use it in a good way?
Think of AI like a lockpick. There are people using it to steal things and get to places they arent meant to be. And then there are people that lost their keys or are in an emergency and need to get through a locked door or gate.
I have also been reading carl newport's digital minimalism recently. In the first 2 chapters the book explains the idea of optimising tools to make them the best way to accomplish their task.
So, very clearly, a data center burning through capital and water and electricity is not an efficient way to accomplish the task of a natural language prediction mechanism.
But I want to take another step back and abstract the concept of an LLM even more. Before we look at the how, lets analyse the what.
I dont know why it is called a large language model. But when I see that phrase I wonder if there is such thing as a small language model? It seems yes as evident by the first result to apear when i search it on google. An SLM is just a small version of an LLM. Small must mean more efficient right?
After some research; SLM does not have a definitinitive size where it differentiates LLM and SLM. But generally from what I have gathered. Even relatively small 7-8B parameter models(which do not run fast on my laptop) are considered LLMs. and its not until the sub 1-2B paremeters where they start being called that.
Additionally the little experience I have had while writing this with SLMs have informed me that they are absolutely stupid and you are better off waiting the extra 30 seconds for a larger model to load.
I mostly use LLMs to ramble into and get them to analyse my blog posts which I find helps me think about what I have written from another perspetive. To do this... We probably dont need the several hundred billion parameters that chatgpt or claude have.
So, my limited testing has told me than google/gemma @ 1B parameters is good enough to show me a perspective on my writing. Which is what I needed it to do.
It is nice feeling like you have optimised something. And I have in several senses. Now instead of burning electricity being paid for buy investors overseas(I am not american) I can burn electricity being paid for by me. But I am using a lot less.
That is not the only benefit, of locally hosted AI. Of course everyone knows there is privacy and control over what happens. But also I think a dumber model might actually give me more of what I want; A summary and brief analayse compared to breaking down every point I have like chatgpt does when it doesnt have the context of the audience or purpose of this blog.