A product roadmap visual depicting a single tech platform journey

I spent the first decade of my software engineering life building technology stacks for digital TV businesses for the likes of DirecTV, Sky, Multichoice DStv, Liberty, US Cable, etc - working at that time, for what was the world's leading Digital TV Technology Services company, NDS. We were in the business of selling full-stack embedded software (like Android / iOS SDK) tailored for set-top boxes (STBs), along with the backend infrastructure services needed for digitising, encrypting and transmitting TV signals over-the-[air/cable/internet] to these consumer device STBs, so people can basically watch TV. We would sell the tech stack along with a suite of TV applications that could essentially be tailored for any type of customer need, including changing different look-and-feel frontend/user experiences, configurable features like live recording or basic watching without recording, on-demand or internet streaming -- all without having to run multiple versions of software codebase per customer. In modern software parlance, we built multi-tenanted technology stacks in a way.

In 2010, we embarked on a vision to harmonize in creating the Nextgen version of the platform - taking the best of all customer engineering projects and core platform enhancements, and creating the next-generation stack, to scale to as many video entertainment providers across the world. We called this initiative Project Sunrise, symbolizing a new dawn for the next-generation experience built on Fusion Mediahighway Advanced offering a fully customizable Snowflake Unity UI experience. 

Here's a short clip from 2011 on Snowflake:

Back then in 2010, I'd just come off delivering what was the biggest migration program in the history of the company, launching a new service for Sky - and thereafter landed another client build for UPC Horizon Gateway STB. In addition, we had 5+ other customers all lined up for new tenants! It was going to be a busy next few years indeed. 

Our approach to building this technology stack included foresight from the very beginning. We were intentional about using a single stack, configurable architecture end-to-end, including customizable applications for custom experiences - to avoid rework and duplication, but most importantly quick delivery turnaround times. Our customers also benefited from leveraging features and capabilities they didn't have to pay for, because some other customers would have already funded the development anyway :-) 

Technical Program & Product Management - Visualising the Roadmap

As our company was primarily a technology engineering company and a high-growth start-up, resources were constrained such that people took on multiple roles. I led the Sunrise project covering technical product and program management. I was responsible for creating the roadmap, backlog and overall sequencing, coordinating with multiple customer-delivery streams, along with the main core platform engineering deliveries - building the next-generation stack. The engineering activities were a mix of software integrations and application customizations through configuration, building out the default flagship application, that would come "out-of-the-box" for selling to prospective customers. The sales team would close the sale by signing off on the profiling customizations and configurations - and the Sunrise factory would eventually produce a release for the customer. This pattern of template-driven, profile-based software configuration approach was not new to us, but the technologies we used had changed over time (see the slide deck at the end). Project Sunrise was never a fully funded initiative though, so we had to partner with customer project teams and core platform engineering, being scrappy and inventive - but still ensuring we have a reference stack available, at all times, for new sales.

How did I communicate the Roadmap then?

I had a ton of detail to manage, spanning multiple customer requirements backlogs - working with teams across the globe, managing a unified backlog, understanding the features and gaps, prioritizing features for the base profile & then owning a delivery plan (which I'll expand upon in future posts). The one mechanism that earned the trust of senior leadership was a visualization I produced, that showed on a single piece of paper how all the streams fit together. Once the executives saw the roadmap, they then had an easy mental model to understand the complex pieces and stages of convergence - that went into building out the NextGen Sunrise platform. 

I decided to write this blog piece today, 13+ years later because, it so happens, I find myself now again responsible for building a Nextgen product (V2), with V1 (single tenant) that is currently supporting existing customers with an active roadmap - and V2 targeting multiple-tenants onboarding new customers with their own specific configs/capabilities - and my team are considering ways of communicating the plan!

Check this out:

The above roadmap accomplishes the goal of showing the interplay between Customer engineering deliveries (above the Sunrise line) and simultaneously showing the core platform engineering feature deliveries (below the Sunrise line) - all contributing to the holistic platform called Sunrise. So customers mutually benefit from the platform core and they themselves benefit from the internal platform development. All of these deliveries are contained within a timeline that serves as the roadmap. 

As I revisited this picture, thirteen years later, I can still appreciate the value of a picture like this - a powerful visualization that will beat any detailed text narrative IMHO.

And here's our scrappy Project Kick-Off Charter

Oh, and here is the initial MVP we demoed on Sunrise:

 

Why I never ran a program without a Project Charter

Lessons on large-scale delivery program management ...

I continue to dig into my past artefacts to showcase my work portfolio. I'm using a multi-pronged approach here: 1\ Showcase my work to prospective employers; 2\ Openly share my work so that others (people I coach, my colleagues and boss, etc.) can benefit; and 3\ Act as my own living knowledge repository.  

I spent a decade climbing up the project management ladder, in the same way I climbed up the software engineering ladder (from junior engineer to principal engineer) - I first started project managing small software product development (2-4 teams with 10 services), then scaled up to large middleware services (20+ teams, 50+ services) as lead delivery owner, then up to full stack systems integration (full stack of all major components: kernel, middleware, integration layers, applications), then program managed a full go-to-market product launch scaling out to including Tech, Business (Finance, Marketing, Supply Chain) & Operations (Customer Care, Retention, Content, Legal, Regulatory) - as senior program manager. I also owned the full plan of starting a business from scratch to launch (a video streaming company) in 8 months. I did a stint in management consulting, running the top 5 business projects for a $3 billion run-rate business, which some companies might call Tier 0/1 initiatives - where I co-ordinated these large-scale programs, as Chief Program Director - delivering through multiple business lines, multiple project management offices and multiple product and engineering teams. In a sense, I served as the CxO program manager, advisor and delivery owner.  

It is with this experience and knowledge, that I dare to share about my work experiences - and I'm not making these things up - you can check my LinkedIn recommendations page for proof.

During my tenure as the lead program director mentioned above, I often found myself picking up and repairing distressed programs - and along the way, I'd help improve team processes and coach the management teams as well. I also ran new business & technology initiatives from scratch, start-to-finish-then-handover. So with this diverse experience, I developed a simple method that helped me navigate both types of program scenarios: either resetting or starting from scratch, the simple, powerful mechanism of a Project Charter document. To this day, I'm surprised to see many program & project managers failing to use the Project Charter in the way it was meant to be used (clarifying the essence), and often find less-experienced, newly minted PMP/Prince2 certified professionals, doing it "by the book". My approach to project charters went much deeper than that...

So what do I mean by using the Project Charter in "clarifying the essence" then?

A seasoned, experienced project leader, chief program director, end-to-end project manager, senior technical program manager, etc - call the roles what you like - in my view, is not about just putting a plan together, working backwards from a deadline or target completion date. No, I believe as senior program leaders must apply their minds to appreciate the bigger picture and create a program structure that becomes the north star in guiding and leading multiple delivery teams. I never started a program without first establishing my project charter, which at the top level, focuses on the following:

• Start by understanding the why. Why is this program needed? Why is it important?
• Move on to understanding the who. Who are the sponsors, stakeholders and teams impacted? Who will be working on the program? "First who, then what"
• A program manager must be sufficiently well-versed with all the roles expected from the program, and work hard to secure the roles needed. Yes, this means the program manager must escalate to get the people needed for the program (on the bus, as well as off the bus). A responsible program manager would raise all these risks & concerns up-front, before officially kicking off the program.
• Clarify the what, including calling out what's missing - Set up the mental model for the program. What is this program about? What is it not about? What's in scope? What's not in scope? What workstreams make up the program? How do all pieces come together?
• Agree, Align, Action - The 3 As of project execution involve agreement on the deliverable, alignment of all parties involved which includes acceptance of their workstreams and ultimately agreeing on the action plan to execute.

Project charters don't necessarily have to be communicated in a written document, a slide deck is more than adequate to communicate the essence. Depending on the business environment and culture (for example, some business cultures prefer slide decks over detailed documents to save reading time, whilst others like Amazon, insist on detailed text narratives). So a seasoned project leader must adapt their style to suit the particular business need & culture of the teams.

In this post, I'm sharing a version of a Project Charter as a slide deck. In a future post, I will share a detailed 50+ page document project charter that involved the launch of a consumer electronics device, the program covered a mix of engineering, business and operations workstreams.

Example Program: Transform Digital Self-Service of a $3 billion run-rate business

I was called in to help reset and kickstart an overarching cross-cutting program to improve a selection of key metrics that would result in increased usage of digital self-service channels, improving customer satisfaction and overall reducing operational costs. This program covered the full value chain delivering the service: 3rd party technology vendors developing phone "mobi" apps using USSD, iOS/Android self-service app, Website, Payments, a hardware kiosk station, set-top box interactive application, integration with internal & 3rd party CRM/Billing systems - and resulting business workflows: finance, customer care & banking channels. Technology teams were spread between the CTO/CIO lines (3 IT pillars), and business teams reported separately to the CEO. The program also served the needs of Group Strategy, Risk & Regulatory. Bringing all these things together requires a steady hand, a tactful negotiator, a strategic and business mindset as well as a strong technology leader. This is why I enjoyed such challenges as these programs were never boring, limited to only tech/engineering.

Enough said, let the slides talk and let me know what you think in the comments!

Pearls of Umar ibn Al-Khattab (3)

“Go easy on yourself, for the outcome of all affairs is determined by Allah’s decree. If something is meant to go elsewhere, it will never come your way, but if it is yours by destiny, from you it cannot flee.”

“Sit with those who have sinned and repented for they have the softest of hearts.” 

“Learn dignity and tranquility.”

“No amount of guilt can change the past and no amount of worrying can change the future.”

“Sometimes the people with the worst past, create the best future."

-- Umar ibn Al-Khattab

ChatGPT - Tetris

How to play:

• Arrow Up: Rotate Tetromino
• Arrow Down: Move Tetromino down
• Arrow Left: Move Tetromino left
• Arrow Right: Move Tetromino right

The game will end after 5 minutes.

This game was developed using ChatGPT as my co-pilot.
Thank you, ChatGPT 23/05/08

How I scaled engineering ops excellence to ±10X with Mission Control

Continuing with me sharing my experiences as CTO, in this post I share the actions I took to help improve an engineering organisation's operational health in our journey of scaling an online video streaming platform from 1X to 10X, from May 2017 to October 2020. To get to 10X improvement takes a journey, which I achieved in under 3 years, and after reaching the goal, I decided I'd learnt enough of the CTO experience and exited, after having set up a strong succession leadership pipeline in place. 

To get an idea of some of the major themes that I tackled during this time, as a leader I had to lead from the front, back, left, right setting the direction of my managers to follow (as all of these interventions were new to them), whilst doing my best to respect what came before:

• Establishing the team despite constant re-orgs going on at parent company - getting the right people in the right roles at the right time
• Transforming a rag-tag undisciplined team to a disciplined, clear-headed, focused organised unit 
• Introducing laser focus on product engineering by unbundling non-core video apps to other businesses
• Being critical on the technology platform by establishing a baseline of the architecture, using third party auditors to rate the scalability of the platform
  
• Improving physical infrastructure: networking, compute, storage and data centres. Move away from self-hosted and self managed data centres to partnering, shutting down data centres as needed.
• Build an industrial grade networking stack and leveraging modern peering facilities and overhauling the server infrastructure
• Setting the roadmap for cloud by transitioning first from single region data centres, to multiple data centre deployments, to running multiple stacks simultaneously, introducing containers and microservices then finally getting ready for cloud and leaping first into serverless paradigms
• Embracing cloud partnerships with big players: Akamai, Microsoft, AWS, etc.
• Improving product and engineering delivery by revamping and overhauling the agile work processes and backlog management.
• Introducing communications mechanisms that helped remove doubt and earned trust across the many different business units and teams (we were known as the online pirates doing their own thing)
• Improving risk, governance and security - bringing it to the top, raising awareness
• Creating strategic partnerships internally and externally to leverage skills and expertise I couldn't get in-house or afford to build or manage ourselves
• 
  Introduced technical operations controls - Mission Control, more active management of operations daily, 24/7 with increased focus, planning and prep for peak times, like weekends and major events planning.
• Aggressively reducing costs on key platform components whilst capitalising on gains through economy of scale

In this post, I'm going to share some of the early context and interventions I introduced in my first 3 months on the job, that remain effective to this day, more than five years later: Mission Control Ops.

The dreaded 403 We're sorry, something went wrong

I took over a team that weren't prepared for the intense discipline needed to run and operate a highly available 24/7/365 platform. There were many reasons for this which I might touch on in another most. I recall coining the term "Bloody May" as the month of so many outages, that I wondered: 1\ What on earth have I taken on? 2\ Is my life going to be consumed by work from now on? 3\ Is there any hope for this platform? 4\ How am I going to turn this platform around? 5\ How much is the job worth to me?

It turned out there was going to be many more "Bloody Mays"in 2017 whilst my team set about improving stability. In 2017, the platform experienced outages that racked up about 20 days of downtime in one year. This equates to ± 95% availability, which is unacceptable for a video streaming platform. When I left the team in Oct 2020, we had turned around the platform to reaching 99.5% availability trending higher. Today, 5 years on, I'm told the availability is much higher but their usage profile has drastically changed (reduced the number of concurrent streams to one device only, also reduced their devices supported, moved most of the services to AWS).