- The bubble has popped for unprofitable software companies: Software companies don’t always make enough money from customers to sustain themselves without big infusions of cash from investors and how this can sometimes be problematic especially during economic downturns.
- Deployment in distributed systems:
- Twitter thread :
- I’m growing convinced that one of the most challenging software control systems to get right is the deployment mechanism for a distributed system. It comes with lots of nuanced failure modes, many of which don’t lend themselves to a centralized solution. 1/
- There are table stakes like velocity controls and reliable rollbacks. But there are also more nuanced considerations like traffic draining (how and how long does the system wait for the traffic to drain before it deploys to a host) 2/
- Making before breaking (how does it verify a newly deployed host is healthy before moving on to the next one), safety invariants (how does it make sure deployments leave the service in a consistent state, even if some of them fail), 3/
- …and ability to overwrite these controls if you need to perform an emergent deployment or rollback. Most systems tune these details through years of hard won operational lessons. That’s one reason having a forum to share operational learnings across teams is so important. 4/
- Blue/green deployments are great because they bypass many of these concerns by not performing a deployment on a live service. But blue/green deployments are not always practical (e.g. stateful services) and require infrastructure that supports them. 5/5
- Distributed systems should tolerate failures. Done well, deployments are just a form of staggered failure injection.
- I like that way of thinking about it. I think most distributed systems are built with the expectation that failures are uncorrelated. But deployments are a form of a highly correlated activity, and building systems resilient to correlated failures is hard.
- Author : Joe Magerramov
- Vanilla Rails is plenty: You can get far with the organizational tools of controllers, domain objects plus “additional systems of objects” which are plain ruby objects and tasteful abstractions. They don’t generally reach for services. Services as an organizing concept can be overused filling up applications with procedurally written scripts.