Sunday, 19 March 2023

An example of resource planning a 100 person technology team

Here's an example of how an engineering leader can go about managing headcount and resource planning to deliver on both the technology roadmap as well as business goals. For context, this is a resource plan for a 100-person size engineering organisation who had end-to-end responsibility for building and managing a live-streaming and video-on-demand online video platform "Delta". 

I was the CTO responsible for managing everything, pretty much a CIO in my own right, overseeing: 1\ Physical Infrastructure services: Datacentres, Networking, Compute, Storage; 2\ Internet backbone transit links; 3\ Cloud integrations and peering; 4\ Telecoms integrations; 5\ Software development & testing; 6\ Video broadcast & streaming infrastructure; 7\ CDN management; 8\ Personalisation & Content Recommender systems development; 9\ Agile Program Office; 10\ Technical Operations Command Centre offering 24/7/365 first/second/3rd line support; 11\ Enterprise & Solutions Architecture; 12\ Security, Anti-piracy, Risk & Governance; 13\ Platform Intelligence Dashboards & Analytics; 14\ Vendors.

Whilst I had my own technology strategy to deliver for the underlying platform to serve the scaling needs of the business, as we were aggressively targeting a growth phase - my teams had to also continue to build out product features and enhancements for all devices we supported (sometimes the apps varying by user experience depending on the device itself, e.g. Smart TV versus iOS vs Web app), we also had to deliver new business product offerings and deliver across group-wide projects and initiatives as shared goals.

With these challenges, an engineering leader must have a firm handle on the "resource" allocation. Sorry, agile folks - I myself cringe at the mere mention of "resources" - but heck, since joining Amazon, I was amazed of how natural the term "resources" is used by management. There are two camps, people who belong to "I am NOT a resource" and "I am a resource"!!

Anyway, I digress. As a leader, you have to keep an eye on your workforce. Based on on your existing headcount allocated to you, how do you best organise your people and plan around the various delivery demands placed on your team? How do you show your stakeholders where your resources are allocated? How do you motivate for additional headcount or escalate a change of priorities if you don't have the data that accompanies your story? How do you show that although you might have 100 people under your org, the capacity for doing project work is only 80 or less, because you need to account for management, team leads, shared service work and other internal projects? See below pivot summary table:

As tedious as it might be, an engineering leader has to do some "management" work - no matter how boring it might feel. This stuff is important. The finance folks need this information. The business owners need to have a view of what impact their projects have on your teams. 

A resource allocation plan can be a powerful tool - my advice - use it often, maintain it and let it become a key mechanism for you to have productive, collaborative conversations with your stakeholders.

Yes, I maintained my own list, accounting for each person in the org. 
No, I don't always support the notion that people are fungible resources, can be chopped and changed or "allocated" using numbers on a spreadsheet. I also don't believe the fractional numbers help either, but they are meant to be rough allocations to help with accounting - which I'm afraid is unavoidable - part of any software managers job really.

Below picture shows what goes into headcount planning / forecasting for the year, trying to keep the headcount to 100 people due to the hiring freeze. Just imagine, a 100-person size team responsible for building the full infrastructure and consumer applications for a streaming video platform that served the whole African continent (50+ countries)! Amazon has the Frugality leadership principle - we were frugal on people but wise on partnerships, using another of Amazon's LP - Invent & Simplify.



As an experienced engineering leader, my advice to software managers: If you're not maintaining a resource allocation plan, you're missing a vital tool of software management. Be careful!!

Saturday, 18 March 2023

Visualising a technology roadmap and year plan on just one page

Here's something that other engineering leaders might find useful: Showing all the work under one's ownership, a technology roadmap on a single page that nicely prints to A3 paper.

In this example, I capture everything that was important to me as the leader responsible for what was at the time, one of Africa's largest online video streaming platform (Live TV + VOD). I was responsible for the full platform end-to-end:
  • Physical Platform Infrastructure and Networking - Data Centres (Compute, Storage, Networking)
  • Cloud Services commercials & workloads
  • Software Engineering  - multi-platform, multi-device software development & testing
  • Enterprise & Solution Architects
  • Video Streaming Hardware & Software infrastructure integration & management
  • Recommendations & Content Discovery Engines - AI/ML scientists and engineering
  • Agile Program Management Office - Agile Specialists
  • Technical Operations & Integrations - Command Center & Mission Control 24/7/365 support
  • Security, Anti-piracy, Risk & Governance streams
How do I show on one page, the focus areas for the above teams in terms of our year plan, showcasing:
  • The themes that categorise the work so non-technical customers & stakeholders can understand
  • The cadence of releases for product feature delivery for product & marketing teams
  • The key KPIs our work drives - growth targets for monthly active users
  • The key events happening during the year that would put a strain on the platform load/stability
  • Show what the tech team will be producing month-on-month
  • The owners and points-of-contact for each work stream
So if you're a CTO or Head of Engineering responsible for an overarching technical platform, and you're not visualising your work in your own roadmap view, then IMHO, you're missing a trick. I used Excel to craft my roadmap because I was being scrappy - but it is possible to create such views if you invest in a solid product/program planning tool.

Click to enlarge: An example of a technology leader's roadmap / year plan

Friday, 17 March 2023

An example of a CTO's operating year's business plan

I'm continuing my series of posts sharing my work portfolio from my past as a technology executive (CTO) when I was directly responsible for a large engineering team (150+ people excluding partners). I approached this role as the next big challenge in my career aspirations. I gave myself a timebox of 3 years, after that, I'd intended venturing on to something else outside of the video world. I'm grateful for the experience as I left with learning all types of executive-management skills for managing business and technology operations. I also accomplished praiseworthy results in what would seem a short amount of time, in just 3 years. This shows how much intensity and density can be packed into a 3 year work experience. I know I would probably not see that much excitement and high-stakes challenges again for some time, even as I write this post in 2023, five years later, whilst working with Amazon's AWS...

Every CTO/Engineering Leader must write an Operating Plan

If you're a technology executive and you're not owning your very own technology operations business plan, in writing and not through high-level powerpoint slide decks, then you're missing a trick and you're missing out on a powerful asset and artefact, that IMHO, is essential for navigating your technology journey. Even before my time with Amazon, I used to write a lot, despite my employer not being a writing-documents oriented company, especially at executive leadership level. Personally, I needed a way to make sense of things, apply critical thinking and process my thoughts into a credible strategy for executing. I also needed to take my senior managers on my journey, lead by setting an example of thought-leadership through writing (because writing documents was unfamiliar to some managers). I also wanted to reach the software engineers on the floor, doing the day-to-day work, help them see the big picture and grasp how the work they do matters to the end goal. 

Another benefit of sharing your technology strategy in written form is to promote alignment and information sharing across many groups of stakeholders - customers, internal partners and external third-party primary partners. As a CTO for a specific business segment (online video), I had my own company's leaders as my primary stakeholders (CEOs, fellow C-Suite top team peers like Customer Acquisition, Retention, Marketing, Sales, Ops, Product, etc.), I also had to support other business' shared goals. In addition, I was part of a wider group structure of the parent group company, inheriting goals from the Group-CTO, Group-CIO, Group-CDO and Group-CISO domains. So the ever familiar problem of having multiple demands placed on a technology organisation. 

As a leader then, you need a mechanism for alignment on your execution plan for the year - this is usually an organisation's yearly operating plan. The aim is to show how you, as the engineering leader are accounting for all the priority goals you're taking on from your various customers and show how the work you plan to deliver, relates to achieving those goals and objectives. Additionally, the specific technology initiatives that you are taking accountability for yourself (like tech platform modernisation, addressing technical debt, etc.) remain nevertheless important for your org to deliver results and meet stakeholder expectations.

Below is one version of a year plan that I authored and used as a constant reminder for checking in and reflections on progress. Let me know if it helped you :-)

[Disclaimer: I write about my past work experiences, this post dating back to 2017-2020 referring to entities that no long exist today (in 2023). Previous mentions of such entities are widely in the public domain through news media outlets, press briefings, launch announcements, etc. I take time to ensure that nothing I share exposes commercially sensitive material. My intent is to share my learnings and experiences to teach and mentor people on a similar path to me; with the side benefit of showcasing my professional work portfolio to current and future prospective employers & head hunters, through my writing].

Thursday, 16 March 2023

How I led the turnaround of a tech platform from 1X to ±9X in just 3 years

In a previous post, I shared how I ended up transitioning from a program manager to being the leader of a large engineering organisation as CTO. I then followed up another post sharing my context in this leadership position that was beset with highly volatile, uncertain, complex and unambiguous challenges - the world of online video streaming. A dynamic, tough, competitive space, especially as Africa was only then waking up to the streaming video wars that started in 2015.

[Disclaimer: I write about my past work experiences, this post dating back to 2017-2020 referring to entities that no long exist today (in 2023). Previous mentions of such entities are widely in the public domain through news media outlets, press briefings, launch announcements, etc. I take time to ensure that nothing I share exposes commercially sensitive material. My intent is to share my learnings and experiences to teach and mentor people on a similar path to me; with the side benefit of showcasing my professional work portfolio to current and future prospective employers & head hunters, through my writing].

In my 3.5 years as CTO, I would see the business change executive leadership multiple times - I think by 2020, we'd been through at least six changes in executive leadership (CEO/Exco-heads). This meant that almost every year, I had a new boss, who's strategy shifted to suit their new vision as they aimed to leave their mark on the business goals of an an aggressive growth strategy.

From a CTO perspective, I had to navigate different expectations of these new leaders, who didn't grasp the nature of technology and engineering challenges. Quite often, my team would be compared with the big players like Netflix, Amazon, Hulu - citing the incredible numbers of concurrent users (millions), comparing against our fledgling platform built in Africa, by Africans, for Africans - a platform that emerged not natively online, but an extension of the existing PayTV broadcast ecosystem. Despite whatever truths be valid or not from the engineering perspective, my job as CTO was to manage expectations upwards as best as I could whilst working with my engineering team on the necessary upgrades to scale the platform accordingly.

Once an app is live and already being used daily, with a steady increase in monthly active users - despite technical debt and platform instability - customers expect to receive feature updates and when it comes to consuming video, a stable viewing experience. Add an aggressive marketing and acquisition strategy, my technology team didn't have the luxury to pause, stop and fix things. Business goes and and IT tech folks just have to get on and deliver, smartly.

One of the first things I did with my team, in addition to resetting their mental model - was to create a simple graph of our state - not so dissimilar to Amazon's infamous flywheel. Based on the business growth strategy, the key drivers impacting my technology team pivoted around two fundamental missions: 1\ Increase the number of active users (growth) and 2\ Increase Engagement. So I created this map, as our North Star - that would remain, un-erased on the whiteboard for 3 years since creating. This picture would become my constant reference during my management meetings (reminding my directs and technical leads why we exist, what areas are important) as well as my all-hands updates:

Technology Strategy Map - North Star / Key Focus Areas

A video streaming business is about reaching as many customers as possible, providing engaging content that captivates their users to return and consume more and more content, staying longer on the app, remaining more engaged. The app user experience needs to be seamless, easy, intuitive and on-par with other well known players (ala Netflix). The tech platform then becomes core in providing the experience. What the picture highlights are the key pillars, which became the driving force for our improvement goals. I focused our attention to execute on these and only these: 1\ Product development (software delivery of app features across as many devices and platforms as possible); 2\ Streaming & Networking infrastructure - CDN & cost optimisation; 2\ Platform Scaling challenges (big area of technical debt - transition to AWS cloud); 3\ Platform Intelligence (don't fly blind); 4\ Content Discovery (AI/ML recommenders); 5\ Telco partnerships (we didn't own the full network). Controlling all of this, was technical operations excellence, something I called "Mission Control" - active, preventative and proactive monitoring of the live platform (technical alerting and social media monitoring).

A large part of this challenge was managing cost. Based on the business growth targets, the cost to serve the technology platform per customer had to remain low. In parallel with me turning around not only a distressed platform whilst keeping the integrity of the engineering teams together (because of layoffs and re-org), I had to work on delivering cost-savings whilst armed with my own sizeable budget. We could no longer be operating on a shoestring budget anymore, and at the same time, we couldn't afford to be wasteful - frugality was still top-of-mind. My budget over my tenure would amount to R1.5 billion (±100 million USD) covering people, capital expenditure, and operations expenditure (video streaming businesses is a costly game), aligned to a Buy / Partner strategy instead of Build-your-Own.

As a result of this strategy, my team were able to execute year-on-year, exceeding expectations and turning around what was once a platform on the brink of extinction, rebooting and refreshing in 2017 to deliver a nett ±9X improvement on active users and availability of the platform:

Platform growth over 3 years under my leadership


As a result of this experience, I wrote a white paper that dived quite deep into the technical challenges a CTO in the online video streaming business must not overlook - check it out here.

My engagement as a CTO was timeboxed from the start - I wasn't expecting to work longer than 3 years as I'd set myself the challenge of learning and setting up a leadership succession plan that would continue to lead the team and deliver on the technology transformation strategy I created. At the end of that experience in 2020, just around Covid times, it was clear I needed to reset myself, and after 20+ years career in video technology, I sought a change...and decided to leave the TV world behind.

Looking back, more than two years later, I somewhat miss the thrill of high-stakes leadership, and the intensity of the experience. More so, leading a large group of people (150+), rallying them to raise the bar and turnaround their tech platform (that I had no prior experience in building, nor was I familiar with the code) and also themselves, in trusting me to lead them - and get past their stigma of not trusting a business program manager... 

My next posts will dive deeper into some of the streams on how we got to 9X, covering some hits and also some big misses too!!

Wednesday, 15 March 2023

How I communicated a CTO's 3 year plan through writing

In my previous post, I shared my method for applying visualisation in simplifying complex technology migration programs, communicating effectively by way of one-piece-of-paper. I also shared how I communicated progress updates for a plan with multiple programs using a simple slide show. Before getting to the stage of summarising complex topics through the powerful, simplifying visual means, one must do the deep work of critical thinking, reflection & reviews. This process usually takes the form of written word which serve as inputs into the visualisation and storytelling stage.

[Disclaimer: I write about my past work experiences, this post dating back to 2017-2020 referring to entities that no long exist today (in 2023). Previous mentions of such entities are widely in the public domain through news media outlets, press briefings, launch announcements, etc. I take time to ensure that nothing I share exposes commercially sensitive material. My intent is to share my learnings and experiences to teach and mentor people on a similar path to me; with the side benefit of showcasing my professional work portfolio to current and future prospective employers, through my writing].

How does deep work happen then? Well, through writing narratives, of course! I love writing. Even before joining Amazon, I was rather Amazonian! As I reviewed the content I produced over my past work experiences, it became clear that what I was doing way back then in 2016-2020, was actually preparing for my future work at Amazon!! Amazon is infamous for its writing culture. Powerpoints be damned! If you aren't able to articulate your thoughts and ideas through writing at Amazon, then you're not going to get much traction!

In this post, I share what I wrote back in 2018, basically reflections that I'd written for an audience consisting of ExCo (Group CTO & CIO members), my senior engineering direct reports as well as some non-technical board members who expressed some curiosity on the learnings from our journey to date, since the massive re-organisation of the business and resetting 2017 strategy (as I shared previously in my experience as CTO in  my first 100 days). 

The situation in 2018 at high level, executive board level: 
In Q1 2017, I was tasked to build the future online video platform (Delta) for the group. By the end of 2017, the group inherited another business unit, bringing its own video platform (Sierra) into the mix. The parent company created a new business unit as a result, consolidating a new operating model, bringing all entities servicing online video customer segments into a single operating business unit. With two different product portfolios, serving different market segments and powered by two different technology stacks, Sierra & Delta. Sierra was borne in the digital world from day-one cloud-native, with Delta having evolved from the traditional broadcast PaytTV world, pre-cloud. So you now have two independent CTOs (separated by continents apart) responsible for two different tech platforms, talk about ambiguity! From a high-level cost accounting perspective, it looks like there's much duplication going on surely!? Simply put: both organisations build apps that consume video, why don't you guys merge into a single platform? :-) 
I thrive in thinking big and looking around corners. I also enjoy solving what seems at first, to be intractable problems. I also love to cooperate and collaborate - always seeking out the best interest for the business in the end, even at the expense of my own career (I was never one for empire building). Setting all personal differences and healthy tech competition aside, I chose to focus on building bridges, looking for reasonable compromises. I also set out to describe the situation in terms that both technical and business stakeholders could understand. So I created this doc, to share my thinking into a 3 year technical strategy would look like, taking into account the learnings from the conversations to date. 

I'm sharing this with my readers as an example of what is expected from a senior engineering leader when it comes to strategic thinking and planning longer term - you have to do the deep work, confront the brutal facts of the current reality and set clear goals and aspirations for a future state. 

Incidentally, the contents of the doc is not so dissimilar to the content one would find in the Amazon Operating Plans (OP1/OP2):