- AWS Cost Explorer Tips and Tricks: Sample queries that surface specific info about services, ec2 instance types, data transfer types, and a few other interesting views. Cost explorer is a nice, easy to grok tool and it shows in that these queries are useful and simple.
- Things they didn’t teach you about Software Engineering: Lots of truth in this about what being a developer actually entails. The quest to write / find the most beautiful code for our systems at the expense of time spent understanding requirements, documentation, thinking about debugging, or delivering value is often energy that is misplaced.
- Asynchronous computing @Facebook: Driving efficiency and developer productivity at Facebook scale: Describes the problems faced by Fb’s queuing system as it scaled up. It started out simply as a mysql table of jobs, a job dispatcher, and a pool of works. This worked for awhile and was easy to reason about. (For many companies it’s fine to stop here.) At some point the async service became unreliable. Jobs would be delayed, priority might not be honoured (inversion type issues), worker capacity would be underutilized or overloaded. They came up with several techniques to spread load around the point when a job enters the system.
- Added the idea of delay tolerance to priority
- Shift work forward or backwards
- Bucket work into normal processing, special event (super bowl), incident (something bad happening in the world people are posting status updates for in high volume) -> more queues
- Rate limits, throttling
- More queues! :) They had a head of line blocking issue. Hard jobs or jobs that created lots of individual sub-jobs would gum up queues.
- Related to more queues is step batching. Jobs would be designed to batch steps together into a single task rather than split into individual ones.
