Providing clarity of job expectations and career pathing for engineers

I cut my teeth during my first decade of my working life experiencing the software engineering job family, starting as a junior engineer in 2000 and working my way up to principal engineer by 2010 (and then spent eight years from PE to CTO, then switching down to engineering manager in 2021). In between senior engineer and making principal engineer, I switched from coding to software engineering management and project delivery. Just before going back to technical, as a principal engineer, I was performing at the level of a senior technical program manager and software delivery owner for a very large software stack. 

I was fortunate to have worked for pure software product companies in that time, focused on consumer applications as well as infrastructure systems. The companies were led by people with an engineering background, CEOs/VPs/Directors with PhDs in the sciences, who had practiced as engineers before getting into leadership positions. As such, the culture of those companies were very engineering focused - leaders took the time needed to develop clear guidelines for career planning and development. The job family framework NDS created was super helpful in level setting expectations with engineers and managers alike. People had a map to reference, removing the ambiguity and suspicion that often comes with promotions. A clear job family framework, in my opinion, is a must-have for any engineering organisation because it provides people with direction to chart their career path. It is an extremely powerful mechanism especially when the company is diverse and allows a spread of career options. Having this map allowed me to experiment with different roles myself, switching between business units and trying roles out, until I figured I'd explored enough or reached my aspirations.

I'm talking about years 2003 to 2011, working with what was at the time, the world's leader in TV software and encryption systems, with offices world-wide (NDS/Cisco/Synamedia). I left that company with solid engineering management experiences and upon returning to South Africa, was well positioned to help raise the bar of engineering excellence. Surprisingly, more than ten years later, working in Amazon AWS, I marvel at NDS's high bar in the early 2000s - pretty much working now for Amazon AWS, I'm  pretty much picking up from where I left behind in NDS from an engineering excellence perspective, I'm glad I schooled with NDS.

When I led one of the largest consumer device initiatives in the history of that company, in 2011-2013 - I was quite surprised that the engineering organisation was missing a vital mechanism for its engineering workforce: The Engineering Job Family guidelines Career Ladder. As a program manager without much direct hierarchical influence, all I could do at the time was provide feedback to engineering leaders about the quality of their engineers and managers, as not meeting the high bar expected from the industry. I influenced quite a bit of change indirectly, helping overhaul the entire technology division's structure and engineering practices that was needed to get the program delivered. However, I couldn't do much about the HR guidelines when it came to job specifications. Instead, I wrote a blog post about my experiences and shared this with the HR/Tech executives in 2013.

Here's some proof of the indirect / referent influence I had as a program manager overhauling the engineering org, that if we didn't do would've cause the project to fail:

When I branched out on my own doing management consulting, I was invited by the business HR & Tech executives to talk more about the technical career ladder framework, which I picked up from NDS days. 

[P.S. I'm sharing material I created as a management consultant. If you'd like a copy of the frameworks, please reach out to me via my contact page.

Disclaimer: I write about my past work experiences to showcase to prospective employers, future clients if I decide to consult again. I'm publishing my work portfolio so that recruiters / headhunters or hiring managers have easy access to my work products, saving you time. If you like what you see, let's talk!]

I created this slide deck to talk about the concepts:

Sense making, apples v oranges, finding a path forward from multiple options by asking searching questions

I've been writing about my experiences as technology executive when I was placed in the midst of uncertainty and high ambiguity that impacted both my personal and professional aspirations in a big way. Making the decision to leave my experiment into boutique management consulting behind, after building a solid reputation as a high-level program manager, switching to a deeply technology role into a business unit that was going through disruption due to both external and internal forces -- was not an easy, straightforward transition to make. I experienced classic imposter syndrome (see this post). Nevertheless, looking back now, more than five years on, those experiences helped shape me to becoming more well-rounded, what some would call a diverse Business, Technology and Operations (BTO) executive - or - as Amazon calls it, a Strong General Athlete (SGA).

My writing this month is on communication methods, mechanisms and tools: 

• How should CTOs (engineering leaders / technology executives) communicate to all groups of stakeholders?
• What tools of writing and visualisations to use?
• How to use critical thinking and the art of reflection to deep dive on the technology strategy - calling out the good, the bad and the ugly?
• How to dive deep to sense make by asking searching questions, that force upwards stakeholder management to engage in guiding the teams on strategy?
• How to find a common ground and build bridges between two (perceived) competing technology organisations?

Questions & Answers Tree - Seeking Clarity from Executives

Let's recap the situation:

In 2017, I took on the role of CTO for an online video streaming technology platform. The business unit was part of a traditional satellite PayTV company, that created an online companion application to supplement its existing TV subscribers to watch TV on the go, initially through web & mobile applications ("Delta" platform) - by investing in digital media division. Not long after this value added service was created, about two years later, the parent investment company, started up a new video streaming business ("Sierra" platform borne in the cloud, no attachments to traditional PayTV like Netflix), completely independent from the existing PayTV business. The two businesses hardly interacted or shared common product, marketing or technology elements for the first two years. When I joined in 2017, there was talk about potential synergies and closer partnerships - which directed my three year turnaround strategy - to modernise Delta closing the gap on Sierra, thus creating comparable modern video consumer experience (Netflix was the bar). A year later, additional complexity and uncertainty came in when the parent investment company, decided to unbundle its independent video businesses to allow itself to focus solely on e-commerce ventures. What happened? Naturally, Sierra business was folded into Delta - create a new business with two product & engineering organisations running in parallel: 2 CPOs, 2 CTOs - tasked to figure out what the future world could look like in creating a Delta 2.0 strategy.

As part of the interactions, still being the management consultant (at the time, I was regarded as independent without any affiliations to taking any sides - since I worked with all businesses before and had existing relationships with all), I helped the executives tackle their options.

The first one - let's understand the assumptions and questions that challenge assumptions. Can executives be clear about their end game? What is the vision? Why are you so caught up about the apparent duplication in tech platforms?

Here's the tree:

Comparing Apples to Oranges: The decision table view

When two engineering teams are challenged about their platforms doing essentially the same thing, especially when the ask comes from non-technical executives, very often engineering leaders become defensive and say "Ah, you can't compare apples with oranges, you must compare apples with apples". Whilst this might be technically accurate, this is not the way to manage communications with stakeholders. Part of technology leader's job is to simplify technical and product capabilities, meeting your customer and stakeholder needs, where they're at. Even if you feel a visual oversimplifies, you still need to tell a story, like the one I used to gain approval that cemented Delta 2.0 roadmap:

Who knew that four years later, I would be digging out the same mechanisms to help AWS executives, (in my first month barely completed onboarding by the way) to decide on a technology stack that my engineering team would need to build/deploy/operate - for providing calls / chats support as their technical call centre platform, servicing one of their highly-regulated, strictly controlled, private cloud partitions? See below decision table, similar oranges to apples story:

How to sell a technology strategy to senior executives

I've started writing about my time as a CTO when I was responsible for turning around and transforming an online video platform to scale 10X as the business doubled down on its growth strategy by investing more heavily into marketing, acquisition and retention drives. As a result, the impact on the engineering team was very high, much higher than when the team built the first version of its platform "Delta 1.0" - and now, under new leadership, I was tasked to take this platform that was lacking in several areas and transform the stack to a future 2.0 vision. It all became more complicated and ambiguous a year later, when another business was merged and we ended up with having two video platforms, offering different product experiences, for different segments of customers.

Despite the technology ambiguity and business uncertainty, the ask from the top was for the Delta product & engineering teams to come up with a plan for a unified stack, showcasing the 2.0 experience. 

I previously shared the technical narratives to help communicate my technology plan. Alas, the docs were too lengthy and detailed for high-level business executives. I needed a way to sell my vision and at the same time secure funding to be allowed to multiple tracks of work running.

How do I show to non-technical business executives, the current state and our future desired state?

What words do I use to show we need to leap from A to B?

How do I convey a story that creates excitement?

How do I gain credibility that my planning is sound?

How do I show complex technology stack layers incl. infra to reset mental model of a "simple app"?

If you've been reading my other posts, you'll learn that I like visualisations, a lot. I also like story-telling - and even way back then, in 2017 - I was familiar with Amazon's future press release concept - and decided to use the future press release headline summaries as a way for me to communicate the release plan. Who knew, that five years later, I myself would end up working for Amazon!?

Here's the anonymised deck that shares concepts you could use, if you find yourself owning a technology platform modernisation challenge, moving your platform from V1.0 to a Nextgen 2.0 state, and sell the strategy to non-technical business executives. By the way, I put this strategy together during my first 100 days into my assignment, in the midst of many other operational responsibilities:

Sharing my writing example exercise from Amazon's interview process

Here is what I submitted to Amazon, as part of their interview process for an L7 Senior Engineering Manager role, in 2020. Depending on the role you're interviewing for, you will get a writing exercise - the one I chose was on Innovation - here's the ask:

Innovation

What is the most inventive or innovative thing you've done? It doesn't have to be something that's patented. It could be a process change, product idea, a new metric or customer facing interface – something that was your idea. It cannot be anything your current or previous employer would deem confidential information. Please provide us with context to understand the invention/innovation. What problem were you seeking to solve? Why was it important? What was the result? Why or how did it make a difference and change things?

Writing Guidelines

1. Write in the style you would use to write a business whitepaper or essay and do not use bullet points, graphics, tables, charts or flow charts.
2. Do not include any confidential or proprietary information from current/past employers.
3. Remember as you write that the reader may not be familiar with specific technical terminology, corporate cultures, and scenarios.  Use language and descriptions in your response that enable readers to fully understand the situation.
4. Please limit your response to 1-2 pages (no more than 8000 characters).

So since I was experimenting going back to being close to technical engineering, I decided to go deep into my past as an engineer, when I invented the Talking EPG:

More than ten years later, a talking interface finally made it to general availability:

Here's the document: Two pages in length, keeping to the written guidelines.

How I managed a technology budget ($100m) with 5+ CFOs, 3+ CEOs

In a previous post, I shared how I took on a challenging role as CTO for an online video streaming platform, with a business at the time that was going through rapid changes in leadership every year. In my 3.5 years stint, I would see my direct manager (CEO) change at least 4 times, their own ExCo board members change at least 2 times, and at least another 5 changes in CFOs and senior finance managers. Whilst these changes happened, my job was to still keep the technology platform running and my engineering team delivering on the product roadmap. 

Another spanner in the works was the ambiguity around having two engineering teams, led by two separate CTOs, building online video platforms, Delta and Sierra - a change that happened a year into my tenure after just landing the full-time CTO role (I'd stopped consulting, only to find myself in further uncertainty with a business merger and potentially more rounds of org-design changes). Delta was under my ownership, serving traditional TV customers with value-added internet services to access Live TV & Video on Demand (VOD) streaming. Sierra, was a pure internet, deep catalogue subscription video on demand service (like Netflix), built and run by a parallel engineering team, under different CTO, in another country, outside of South Africa.

So, with these many changes in executive leadership, at the top-level financial review - executives would naturally inspect why the business seems to be duplicating engineering efforts at building online video platforms, even though the two products serve a different set of customers, emerged at different timelines, using different set of technologies - but they both appear to "stream video" at the high level. So why is there duplication? Where are the costs going? How do we streamline engineering costs?

Enter the Financial Model & Technology Budget Commentary Document

The first order of business for me, being tasked with a turnaround challenge - was to review the existing cost components, identify the main cost drivers and derive a perpetual financial model based on the core business driver of growing monthly active users.

Starting in 2017, neither the technology team nor the finance managers could tell with a high level of confidence what the true costs of running the technology platform were, and ultimately what the cost of running a technology platform translated to a cost per user value, and ultimately how much from the profit margin this cost was eating into. My platform provided value-added-services for free, existing users were homed to a primary TV subscription package (satellite TV using a dish and decoder / Set Top Box (STB)) as main profile. Additional viewing profiles for online consumption through mobile phones, smart TVs, game consoles and web browsers, were offered at zero cost. Additionally, my platform served internet connected STBs with larger video catalogues for streaming on demand, this too, at zero cost to the subscriber.

Financial Model (brilliant, if I say so myself!)

So I created a detailed financial model that was able to show how to forecast a largely fixed cost investment, to operate in a variable cost domain. The largest cost components for video streaming is the infrastructure cost for the networking pipe (internet transit) and content delivery network (CDN) data costs. Since video consumes large amounts of data, these costs are covered by the streaming operator. CFOs prefer fixed cost accounting than variable costs because it helps manage their cashflows and forecasts better. In an on-demand video world, where customers come and go, or viewing behaviour varies by seasonality or during peak events, the experience is more bursty than constant. This is why video providers focus heavily on content personalisation and recommendation systems to drive stickiness and increase engagement. The more users watching video, the more data is consumed, the costs of data increases. The longer users remain on the platform, the more users are engaged and the more video data is consumed. All these point to increasing costs - but - at some point, the costs per user eventually decreases because the number of users increase, the cost per user eventually decreases. I created a financial model that showed how we could manage variable costs through a fixed cost model, by buying data wholesale upfront, using a minimum commit model with overflow-at-zero-cost into next year for unused data, and also negotiated year-on-year cost savings, since cost of data decreases year on year.

Using my financial model, I was able to show significant cost savings to the tune of R80m ($5-7m) in two years. I also, single-handedly negotiated significant costs savings with our primary transit links and CDN providers, as a result of my financial model forecasting. In year one, I negotiated a 1233% increase in CDN bandwidth and secured 71% decrease in out-of-band data costs, 50% decrease for inband per GB data costs - keeping my overall costs relatively flat to track a fixed expense. In year two, I negotiated additional cuts, reducing in-band data costs by a further 63%, out-of-band by 48%. In year three, I negotiated further deal reductions because of increased competition in both the transit-and-cdn space, landing on a further 20% reduction on inband and out-of-band costs. Over this period, I secured enough guaranteed capacity in the network that exceeded future growth targets, but kept most of the projected fixed costs inline. As a result, when I decided to leave the company, in my 3 years tenure, I drove internet costs down by 84% on a cost per user metric and 96%reduction on costs per GB per user metric.  Additionally, working with the same transit and CDN partners, I secured more than 7X improvement on overall internet throughput, reaching close to 1 terabits/sec on network capacity, which for the African continent, is not bad going indeed! In terms of storage, from 2017, I increased the data commitment by 2000% cumulatively year-on-year.