New Show Hacker News story: Show HN: I trained a 65B LLM on my texts to talk to myself (details inside)
Show HN: I trained a 65B LLM on my texts to talk to myself (details inside)
4 by muttled | 1 comments on Hacker News.
I trained the 65b model on my texts so I can talk to myself. It's pretty useless as an assistant, and will only do stuff you convince it to, but I guess it's technically uncensored? I'll leave it up for a bit if you want to chat with it. I posted this to Reddit and had several hundred people talking to it. Salient points from that discussion: LLAMA 1 65b Rank 128 5 epochs Batch size 1, 256 cutoff Trained in the Oobabooga suite using bitsandbytes 4-bit quantization for the lora Loss around 1.5 seems to give the most coherent results Trained on raw text dumps that is then parsed by a crappy Blazor Server app I threw together in a few hours. Text format is just "Sender:The Message\n" Trained on 2x 3090 Training took about 16 hours at a 90% power cap on the 3090's Trained on ~30k texts (I talk a lot, that was just 2 years) There's nothing telling it that it's a robot, though it sometimes seems to know It was largely inspired by the Unreal Engine lora tutorial I generated a list of fake names and addresses, pulled a list of my contacts, and then scripted out swapping the names and addresses for fictitious PII. I don't really send other sensitive data through text and my account is so thoroughly associated with my real name/location that the data leakage risk is manageable for the short period of time I'll have this available. It tends to halucinate fake PII as well which I think is partially a side effect of the data scrubbing. You'll notice it says things like that I live at 420 Ligma. I'll need to mix in some actual assistant tasks to the dataset before it will actually be useful as an assistant. Right now it's largely just for idle conversation. It's pretty ADHD and will randomly go off on its own tangents. I don't think it's the model. I think I just talk like that. Let me know if you have any questions or comments. I built it for myself, but figured I'll let the communities that have taught and entertained me so much play with it a little, too. Note: it says some pretty unhinged stuff. There's absolutely no guardrails. It also tends to talk like you're already friends with history.
4 by muttled | 1 comments on Hacker News.
I trained the 65b model on my texts so I can talk to myself. It's pretty useless as an assistant, and will only do stuff you convince it to, but I guess it's technically uncensored? I'll leave it up for a bit if you want to chat with it. I posted this to Reddit and had several hundred people talking to it. Salient points from that discussion: LLAMA 1 65b Rank 128 5 epochs Batch size 1, 256 cutoff Trained in the Oobabooga suite using bitsandbytes 4-bit quantization for the lora Loss around 1.5 seems to give the most coherent results Trained on raw text dumps that is then parsed by a crappy Blazor Server app I threw together in a few hours. Text format is just "Sender:The Message\n" Trained on 2x 3090 Training took about 16 hours at a 90% power cap on the 3090's Trained on ~30k texts (I talk a lot, that was just 2 years) There's nothing telling it that it's a robot, though it sometimes seems to know It was largely inspired by the Unreal Engine lora tutorial I generated a list of fake names and addresses, pulled a list of my contacts, and then scripted out swapping the names and addresses for fictitious PII. I don't really send other sensitive data through text and my account is so thoroughly associated with my real name/location that the data leakage risk is manageable for the short period of time I'll have this available. It tends to halucinate fake PII as well which I think is partially a side effect of the data scrubbing. You'll notice it says things like that I live at 420 Ligma. I'll need to mix in some actual assistant tasks to the dataset before it will actually be useful as an assistant. Right now it's largely just for idle conversation. It's pretty ADHD and will randomly go off on its own tangents. I don't think it's the model. I think I just talk like that. Let me know if you have any questions or comments. I built it for myself, but figured I'll let the communities that have taught and entertained me so much play with it a little, too. Note: it says some pretty unhinged stuff. There's absolutely no guardrails. It also tends to talk like you're already friends with history.
Comments
Post a Comment