all writing notes

  • If you can’t afford the latest, most expensive hardware, local AI is all about …

    If you can’t afford the latest, most expensive hardware, local AI is all about trade-offs. And some trade-offs are more dangerous than others. Using the best model generally recommended for my era…

    If you can’t afford the latest, most expensive hardware, local AI is all about trade-offs. And some trade-offs are more dangerous than others.

    Using the best model generally recommended for my era of hardware (Qwen3.8 27B 8-bit on a 64GB M1 Max, Hermes Agent, 64K context, thinking off), it functioned, but slowly.

    Almost 3 minutes to respond to a simple “GM” when loaded into Hermes. Much of that time was spent bootstrapping Hermes’s system prompt before it wrote a word, but the harness is what makes the model useful.

    Once it had loaded its environment, things started to speed up, but it still took approximately five minutes for a basic readme text file. Nowhere near functional enough to get in the *flow*.

    I also tried switching from a standard LM Studio server to oMLX. Same model, different engine. Token generation was about 12.5% faster, but that didn’t translate into faster work through Hermes. The default engine actually finished about 9% quicker overall in my limited tests, not the breakthrough I was looking for.

    So I started hunting speed with lesser models and more quantised versions (6-bit, then 4-bit). I made modest incremental gains with each step down. Tests still passed, but needed more course correction, and it was still just slow.

    To make any real gains meant switching to mixture-of-experts models (Ornith and Qwen3.6, both around 4x faster on the tasks they completed), but at that point things started getting dangerous…

    Ornith got file paths wrong, created files in the wrong place, then attempted a recursive deletion to clean up its own mess. Hermes blocked the deletion, but that’s not something you can leave to chance with an agentic AI.

    The uncomfortable truth is if you want to get into a flow state with AI, you’re probably going to need to max out on modern hardware or simply accept that you need to use cloud inference.

  • If you want a real scare, check your AI logs for all of the random websites it …

    If you want a real scare, check your AI logs for all of the random websites it has visited in the course of “serving you” Happy Halloween 🎃

    If you want a real scare, check your AI logs for all of the random websites it has visited in the course of “serving you”

    Happy Halloween 🎃

  • Eurovision for AI. Who is building this?

    Eurovision for AI. Who is building this? Slopcore has broken through, and this earworm has been stuck in my head all week. https://twitter.com/Extraltodeus/status/2109726045553488131

    Eurovision for AI. Who is building this?

    Slopcore has broken through, and this earworm has been stuck in my head all week.

  • While Grok Bot is good and has considerably upped my Grok usage, coding in Curs…

    While Grok Bot is good and has considerably upped my Grok usage, coding in Cursor without Claude API included is a definite downgrade. For anyone still on the fence of the X…

    While Grok Bot is good and has considerably upped my Grok usage, coding in Cursor without Claude API included is a definite downgrade.

    For anyone still on the fence of the X subscription…

    I did hear there is a Grok model update around the corner though.

  • Time my agents started pulling their weight a bit more.

    Time my agents started pulling their weight a bit more. That NZ$45,099 512GB Mac Studio isn’t going to buy itself!

    Time my agents started pulling their weight a bit more.

    That NZ$45,099 512GB Mac Studio isn’t going to buy itself!

  • Claude: These keywords haven’t converted in a week. Recommend removing.

    Claude: These keywords haven’t converted in a week. Recommend removing. Me: That’s literally what we sell.

    Claude: These keywords haven’t converted in a week. Recommend removing.

    Me: That’s literally what we sell.

  • Never press the update button when its time to be productive ⏱️

    Never press the update button when its time to be productive ⏱️

    Never press the update button when its time to be productive ⏱️

  • Tried swapping Hermes over to Qwen 3.6 27B for a fully local agent on my 64GB M…

    Tried swapping Hermes over to Qwen 3.6 27B for a fully local agent on my 64GB Macbook. The model chats fine in LM Studio, but running it as a multi-turn agent with…

    Tried swapping Hermes over to Qwen 3.6 27B for a fully local agent on my 64GB Macbook.

    The model chats fine in LM Studio, but running it as a multi-turn agent with recursive tool calls turned my Mac into an expensive heater.

    Turns out 64GB is plenty for Q&A. Not so much for the full agentic experience 😬

  • Anthropic S1 filing and Open Weights China FUD dropping on the same day. Intere…

    Anthropic S1 filing and Open Weights China FUD dropping on the same day. Interesting 🤔 https://x.com/AnthropicAI/status/2114705042817110293 https://x.com/cohere/status/2114752002370544004

    Anthropic S1 filing and Open Weights China FUD dropping on the same day. Interesting 🤔

  • We need more utopian sci-fi, so that advanced AI has some good role models to l…

    We need more utopian sci-fi, so that advanced AI has some good role models to look up to.

    We need more utopian sci-fi, so that advanced AI has some good role models to look up to.