Notes from p99conf
This is an online only conference where speakers prepare and record talks offline before they are shared with people in the conference and I guess there must be a community discord or slack server somewhere where people can talk about what they’re seeing.
I’ve watched a couple videos so far. Both excellent!
- Bryan Cantrill, Sharpening the Axe: Proposes the idea that toolmaking is not always seen as a valuable investment to be made as a company / team. It’s an “investment in the future that takes away time and money from the now”. He makes the point that we are where we are today in terms of tools like CI/CD, operating systems, debuggers, languages, etc because of people trying to scratch an itch that was hyper local to them and they were allowed to do so.
- “Making tools is fundamentally human” We build tools that help us build better artifacts. This was a really great quote / idea!
- Overcoming Variable Payloads to Optimize for Performance: Talk by original author of flask about event processing pipeline @ sentry.io. Architecture includes kafka, rabbitmq, redis, an app written in python that is a monolith, and a few components written in rust. Rabbit is in there to allow for concurrent processing of events (Apparently there are concurrency limitations in python that are hard constraints to overcome). Sentry is great. My company uses it for crash reporting.
- 100k rps at the edge
- 150k rps at an inner service layer (I guess the edge takes aggregate events and splits them creating a larger number of inner events)
- event payloads are written to redis to avoid copying data between processing steps