Weekly notes

• LLM Serving Fairness: No more noisy neighbours: Several controls employed for fairness - rate limits, admission control, tiering of tenants based on contract value, 1-queue per tenant, round robin across tenants in a tier, token-bucket style rate limiting.
• North Mini Code: Agentic Coding Model for Developers | Cohere: Llm trained by a company based in Canada. Part of Sovereign ai movement.