- Monitorama PDX 2024 - Pugs, Poe’s and pipelines; An engineering perspective on big-data streams: Cost of a data pipeline relates to how much data you push through it. If you can reduce the amount of data persisted at the point you ingest, you can save a lot of complexity and cash. (Filtering + aggregating) Point of enrichment says for each data item, get meta info needed to understand what you’re looking at and filter ruthlessly to what you need to run the system. Point of aggregation says for each data item, keep a window of similar data and merge new information into it. (eg Minutely latency data vs a raw stream) You can also merge data later in the pipeline but the cost goes way up. (Talked about summary merge table in clickhouse where the PUG is transparent to users of ch)
- Neat talk. Data pipeline for monitoring data:
- Collect
- Ingest
- Enrich / transform
- Store
- Analyze / decide
- Alert
- Reasons to use your shell’s job control: Primer on how to use your terminal’s commands to background (bg), kill, foreground (fg) and stop commands it runs.
- Two traps in iostat: %util and svctm: %Util for ssds tells you how often a drive is doing at least something not how well you are making use of total capacity for work. SSDs do parallel work all the time and iostat doesn’t take that into account. Iostat was made when magnetic spinning disks (HDD) were more common. HDDs do work serially because physical constraints. (Single arm, read head, position of data on disk, etc)
- Quick takes on Rogers Network outage executive summary: The crtc published a report about the big rogers outage from a couple years ago. Implicated was a network migration, inability to access network equipment, inability to communicate effectively initially, a sense of risk decreasing during a migration, “cleanup” work that was done at the end of a large multi-phase migration. Thoughtful comments after reviewing a somewhat candid report.
- “Out of band” network management is not trivial: Not trivial indeed.
