Weekly notes

• Separating signal from noise in coding evaluations: Clanker benchmarks are dodgy dodgy. Never forget this!
• Experiences with local models for coding: Need lots of memory (64gb+) for mediocre results currently. Lots of gratitude for people who are doing this experimentation. This does seem to be improving but probably there is a minimum amount of resources required to work in a long conversation or a medium to large code base.
• Limit line lengths to increase readability: Today I learned there is a character unit which is the width of the ‘0’ character in the current font.

article {
  max-width: 60ch;
}

• I’m the “unscientific testing” guy. Here’s my thoughts on the steam machine after 9 days: The steam machine is pretty great. The reviews on youtube talking about cost and performance are not speaking to anything I care about …

• Mastodon, The Only Good Choice: I agree with this sentiment but some people I follow are only using Bluesky. I never use twitter anymore.

• None Yet. What does incident management look like on day 1: Brilliant. Small, simple process that makes for a good starting point for a company.
  • Declare
    • Call the incident
    • Call it when you need more eyes (not severity)
    • Waiting means you might start working the problem later than you could have
  • Stop regular work
    • People working an incident need to focus on recovery
  • Who
    • One person at a time for however long the team agrees
  • Comms
    • One person makes the company aware of what is known, what we’re doing and when to expect another update
  • After
    • Learning
      • What happened
      • What was learned
      • What will change to avoid or help recover from this kind of incident
    • Write down action items and add to the backlog
  • Reporting
    • Spreadsheet
      • Date
      • Duration
      • What broke
      • What fixed it
  • Later
    • Severity
    • Commander
    • Blameless
• Rewriting Bun in Rust: Big port of bun from zig to rust implementation languages at incredible expense by Anthropic. Makes for an exciting case study, but many questions remain: how many bugs are in the new millions of lines codebase? What’s the quality of the resulting rust code? What about safety? It doesn’t seem like this kind of usage is going to be accessible for people not-Anthropic or with massive amounts of money to burn on an experiment like this. $160,000 in api use. In other news, open weight models are apparently 6-12 months behind frontier models and also improving, so there will be alternatives to openai and anthropic in the future for this kind of tool it seems like …
• Transforming How We Run Kafka at Honeycomb: Considerations for a large migration of kafka from ec2 to eks in aws. Considerations:
  • Practiced migration plan
  • Collaborative activity across teams
  • Informed by recent outages
  • Start by understanding historical context like “how we got here”
  • Migration plan is a living document that is updated after every phase