Weekly Notes 36: Neat conversation between Simon Willison and an LLM about sql schema and queries for an acl system, and a broad overview of a recent interest of mine - data engineering
- Transformer Explainer: Neat visualization of how large language models work. :)
- Noisy Neighbor Detection with eBPF: eBPF use at netflix to understand when containers on a host are being starved for cpu cycles by others.
- What is Data Engineering?: Excellent article about data engineering from the Pragmatic Engineer blog.
- Incident affecting Cloud Build, Cloud Developer Tools, Google Cloud Dataflow, Google Cloud Deploy, Google Cloud SQL, Google Compute Engine, Google Kubernetes Engine: Software update to a critical service caused a failure. And then a secondary component failure which isn’t described prevented the initial service from failing over to 2 healthy azs.
- Advancing Our Chef Infrastructure: Evolution of chef use at slack. Feels familiar. Simplest thing you can do is start with one shared server. Slack grew out of the number of vms a single chef server could manage and needed to sharded. Also they had to evolve processes to handle phased rollouts to different environments. There’s more here that was a fun read me having used and enjoyed working with chef in a past life.
- A CoPE’s Duty: Indexing on Prod: Developing a production sense across the org and the idea that testing in pre-prod can never be similar to production in important ways like diversity of users / traffic in the system and often volume.
- SQLite Schema for Access Control Lists: Conversation with llm about how an acl system in a software application works that focuses on the database layer