What makes OTT Streaming platforms different to DTH/DTT STB Broadcast?

In the early days of this blog, I shared in deep detail the technical and project landscape of PayTV systems for the traditional broadcast headend and set-top-box (STB) software platforms and projects as subject matter expert in this area.

In this post, I will provide a high level overview of the core differences and challenges between broadcast DTH/SetTopBox systems versus OTT platforms. OTT has become a hot topic, with the nascent rise of OTT (Over-the-Top) internet streaming platforms that many a traditional PayTV operator is now transitioning to, in light of the growing competition for eyeballs & cord-cutters driven by the likes of primarily SVOD (Subscription Video On Demand) players like Netflix, Amazon & the like. The challenge though, is that these PayTV operators tend to trip up by bringing PayTV/DTH thinking into the OTT-world, which is why I felt the need to write about this, because I have, and still am currently experiencing these challenges. Suffice to say, it becomes very tricky to disagree can call out a high level broadcast TV executive's assertions (expecting like-for-like DTH & OTT experience) as not completely accurate when it comes to OTT engineering...

The main aim of this post is thus targeted at senior executives who come from a broadcast world, and may not fully grasp the nuances of an online streaming world, specifically pointing out the spectrum of control of the end-to-end platform that PayTV operators generally enjoyed much comfort in that world, in contrast to the new world of online, where it becomes impractical to expect the level of control previously enjoyed - basically you can't control all the components of the internet, unless you invest in owning a large part of the network, which some operators have attempted & haven't really succeeded!! Simply put, the internet was not designed for Live TV streaming unlike traditional broadcast TV - the internet is somewhat chaotic & messy to say the least. In addition to this level of infrastructure control & lack thereof, an OTT platform is largely software systems designed for internet-scale (i.e. massively distributed server-side backend components) which requires a different skillset and experiences traditionally enjoyed in the closed ecosystem of broadcast & set top box device software development teams.

Another point of this post responds to the standpoint from traditional PayTV executives who may have spent a large part of their work in building broadcast systems with an engineering mindset anchored to a past era that is no longer relevant in the OTT world. Expressing opinions such as "engineering systems either work or they don't, it's either a 1 or 0, no in between states..with an expectation of 100% always-on availability..." in my view is somewhat misinformed & borders on belittling the engineering challenges faced in the online world. Whist every responsible engineer understands such demands and expectations of any system, a key point to remember is that broadcast TV has benefited from many years of refinement, standardisation, systems & processes specialised for the domain of TV - and when it comes to the internet - well, it was never really designed for video consumption, the speed of innovation has outpaced the development of standards & so is a much more messy world compared to traditional broadcast TV (satellite/terrestrial).

Disclaimer: This post is not about downplaying the broadcast engineering world, in fact, a lot of OTT operations highly depend on the upstream capabilities from PayTV operator's existing broadcast engineering infrastructure. Broadcast engineering systems are a marvel of engineering in its own respects, systems are highly complicated & superior engineering. OTT systems, and the point of my post, is to show the extension of the value chain, highlighting the new software & systems engineering challenges that present itself in the OTT world, that will be new concepts for existing broadcast engineering teams to learn & appreciate.

Key points for TL;DR folks (especially the PayTV Exec CXOs)

• It's not a simple thing to compare OTT platforms wearing hat of a traditional DTH broadcast guy, it  is not simply a like-for-like comparison.
• Don't be naive and pass value-judgements just because you were an engineer in the broadcast TV world (Headend or Set Top Box) does not necessarily make you qualified to understand OTT engineering domain. You can't bring broadcast/STB-thinking-mindset to the world of OTT.
• PayTV operators have historically enjoyed a high level of end-to-end control, not so in OTT world. The internet is outside the PayTV operator's control.
• There are far more components in the value chain of OTT than broadcast (backend, frontend and device software stack is more complicated in OTT than STB software).
• High level of device & platform fragmentation in OTT that a traditional PayTV operator must consider.
• The internet was never designed for TV consumption & has innovated at a much faster pace than traditional broadcast TV, the internet is a messy place, broadcast TV is somewhat cleaner and more determinstic.
• OTT engineering teams have different challenges compared to traditional broadcast & set top box engineering.
• PayTV operators need to plan carefully the convergence (DTH & OTT) of engineering teams and be prepared for a "clash of worlds".
• Set Top Box software systems enjoy far more security control & protection than its OTT sibling.
• Peak events & load challenges are unique to OTT platforms compared to broadcast DTH STB (scaling challenges don't really exist in DTH/STB).
• Redundancy, Fast Rollbacks, Failover & Disaster Recovery are easier to achieve in OTT than in broadcast DTH/STB systems.

Open letter to Business & Tech Leaders

Dear Business & Technology Owners...

Have you ever stopped to reflect on why is there always tension and conflict between business stakeholders and technology providers? Instead of there being a "healthy tension" between the two partners (assuming you take the terms "partnership" seriously and not just paying lip service to the term), the relationship is often clouded with doubt, suspicion & mistrust on all sides?

I've been in this technology software business for all of my professional life (~20 years), spending the first half working on technology product & platform development (company run by engineers & not accountants), and the latter half on the other (dark?) side - the corporate business - with in-house technology / IT "service" divisions, where business (accountants mostly non-technical) people run the show - an environment where there is often just a complete lack of alignment, understanding & appreciation for what goes on in both worlds: technology-and-business-worlds alike.

[Bias disclaimer: Yes, I do I have a bias - I believe if a company thinks itself to be a technology-driven business, then I expect the CEO to at least have an engineering/science background, or the CTO/CIO must have written code & built solid (industrial-strength) products in a previous lifetime. If a CEO is not technical then at least develop a keen appreciation for the engineering challenges and then trust your technology owners to do the responsible thing & you move out the way....]

At a recent ExCo of which I'm just an observer - this lack of alignment reared it's ugly head again when "the business" could not understand how a six-month's projects wish-list from their side, translated into a two-and-a-half-year implementation estimate from the technology team, making a poor attempt at explaining, the classic "IT demand & capacity planning" challenges (what I still find quite surprising in today's time, a topic for another day, let's just say I think "IT Demand Management Process" is not relevant today, and I propose that this is just "old-school-IT" & that its days are numbered).  

What followed then from business was the typical:

• "Why don't add another 20 development teams to make the six months timeline?"
• "Why can't we have CFTs (cross-functional-teams) like the other divisions have?" 
• "Are you sure you have the right engineers? Why is the system so brittle? Why must it take so long? Should we take our business elsewhere?"
• "Why can't we have bespoke technical teams attached to each business owner?"

What also makes this quite a difficult conversation is the CULTURE - where technology teams have historically been at the receiving end as the source of all business problems (the cause of business frustration), the underdogs, just taking a beating from business, without any confidence to push back & have constructive conversations. It's a shame really that a large number of technology teams just have to "submit" and "do as you're told" by business...surely it can't go on like this??

So this is what I've reflected so far - and it calls on both Technology & Business leaders to come together to the table, for me it's simply about ALIGNMENT. In order to align, one needs to really LISTEN. I mean REALLY LISTEN and PARTICIPATE in dialogue, after-all, this is what PARTNERSHIP supposed to be about, right??

Dear Technology Leader...

I think a large part of the issue is technology leaders not doing enough to communicate and simplify the reality of their world. Assuming you're not building a greenfields stack, instead you're building on existing systems (not legacy, but still probably reaching end-of-life), more needs to be done to manage expectations. I would avoid too much abstraction & side-stepping: be open and transparent about everything, no hiding...here's some searching questions:

• Have you stopped yourself to reflect on why there is such a disconnect or lack of understanding? What are you doing wrong? Is there a fundamental issue of principle?
• Have you taken your customers through the system architecture and current challenges?
• Have you communicated your assumptions clearly? For example - does the business understand maybe you're trying to build a common platform/product stack that can support multiple business requirements (e.g. in a multi-country scenario) in one-go?
• Is this assumption valid? What if the businesses are quite divergent in terms of customer requirements?
• Why is it important to provide a common technology platform anyway? For who's benefit?
• Would it not be easier to branch the code & platform to serve individual businesses? Risk duplication but gain in business agility? 
• Who cares if the code is forked anyway, has this option ever been tabled?
• How can you explain the nuances of common stack software development to non-technical people?
• Does business understand how you've structured your delivery teams?
• For example, you might be running with component teams, specialisation instead of generalisation and not ready for cross-functional full-stack teams yet?
• Are you clear where your bottlenecks are?
• Is your software engineering principles solid - i.e. can it support multiple release streams simultaneously?
• Have you shown to business the complexities of what it means to spawn up multiple teams (i.e. this rather stupid notion of adding more people to get the job done sooner)?
• Have you established a way-of-working between business & technology that aims to at least address some of the challenges, and set forth an ultimate vision / aspiration of what ideal could look like?
• Are you doing enough to manage expectations upwards and thus protect your teams?
• Why do you have a need to insert "Business Relationship Managers (BRMs)? Are your technology teams not equipped in dealing with business stakeholders directly? Why do technology units decide that an intermediary BRM would help solve relationship problems?
• Are you empowered enough to challenge priorities & requirements if they don't make sense to you?
• Have you communicated that systems generally have to refresh every 3-5 years? Have you considered presenting a technology roadmap showing all the technical initiatives planned whilst simultaneously expected to keep the business lights on (business-as-usual) and still stay ahead of the curve to keep your stack technically relevant?

Dear Business Leader...

Just as there may be some gaps with your technology leader counterparts, you should consider what gaps are on your side. For starters, have you really invested time to LISTEN to your technology partners?

• Have you done enough to convey your business vision, strategy & goals in a clear manner that informs your technology partners of the role they play & your dependence on them for your overall success?
• Have you resisted the urge to bark orders without first trying to listen?
• Do you have an understanding of how your technology teams have built the system & components? Perhaps you had a role to play in this - i.e. ending up in the current state because of business owners?
• Did you perhaps grow up with the industrial mindset (factory methods) & expect that software systems follow similar patterns "it's just a production line outputting widgets"?
• Do you understand what it means to write common software code and deploy common  software & infrastructure to serve the needs of multiple businesses (with its own set of rules)?
• Do you have a clear picture in your mind of the distinctions and challenges of software teams: Component teams versus cross-functional teams?
• How do you as business owners come together to align on overall business requirements (assuming you're in multi-country set-up) that shares a common platform?
• Did you even agree with technology that a common platform is the way to go?
• Are you doing enough to review your own project's backlog and business requirements - by priority?
• Do you accept that your technology team would appreciate from focus - i.e. do one thing of value at a time?
• Are you doing enough to make sense of your own business priorities?
• Have you read "The Mythical Man-Month"? Do you actually believe that "adding more people to a software project will get it done sooner?" - Are you not being naive, have you stopped yourself to reflect that maybe you're just not getting it??
• What are you doing to get closer to understanding your technology provider's challenges?
• Do you use the IT/technology systems on a daily basis?
• Have you considered sitting closer to your technology team?
• What is your role in the partnership? Is it too one-sided (i.e. your side)? Do you get immense joy at squeezing your technology providers, adding pressures and artificial deadlines to make your case?
• Why do you feel you need an intermediary between yourself & technology - and insist on this concept of a BRM? Are you not interested to learn more about your technology stack or develop closer relationships with the people building your technology? 

Attention Business-and-Technology Leaders - Don't look for that Silver Bullet...you'll have to discover this for yourselves, there's nothing off-the-shelf to plug-and-play for you...

You might have attempted every fad at improving process improvement, this "agile" topic has become so misused that both parties can't really tell if agile is working for you or not - and you're quick to blame the process, or attempt to try a new way, and even get professional consultants in to help...but...as I've experienced agile in a variety of forms in the past ten+ years, I still come back to the essence - and I really feel that people have misunderstood the core. This core that I refer to, is the founding principles espoused in the Agile Manifesto. Simple statements but deeply profound, and it all starts there. I feel this essence is often just skimmed over in favour of an execution process which has led to the downfall of many IT/Business relationships, which is why we keep coming back to the rather archaic IT/Business management frameworks that calls for IT Demand/Capacity Management and Business Relationship Managers...

Go back, breakdown the silos, no pedestals or ivory towers allowed in today's age, re-align & commit to participating in respectful, sincere dialogue, meeting each other half-way. Identify the root of the pain-points, agree on sensible ways to manage classic "capacity challenges" which simply implies some focused management of priorities by business people...if after you've tried to re-align, and still there's an expectation of infinite capacity from IT delivery...then...maybe there's just no hope left...

Yours sincerely,

A weathered-and-battered-technology-leader-looking-for-hope!

Whiners never win

Whiners never win!

What simple yet powerful, three words? 

These are words my electrical engineering professor used on me back in my second year, when I challenged my paper being wrongly graded.  I remember waiting a long time at his office just to see him, the prof just gave me one minute "Mr. Khan, here's some advice for the rest of your life - whiners never win". That was it, and then he left, my paper unchanged.

This stuck with me as I completed my degree, started my first job, took a chance one year after graduating by emigrating to two countries, marriage, parenthood, etc. I keep coming back to this simple message "whiners never win" and so just get on with life or work...

Yet, coming back to South Africa, now 8 years and counting, I was surprised to learn how much whining happens in the workplace! It is emotionally draining. Almost everyone has a complaint, not happy with this or that, unable to separate personal emotions from being professional at the workplace. Just a whole lot of whining going on. I wonder if it's just the context of the nation, just whining, starting with the commute into work listening to radio stations (like 702) which I've stopped completely. I actually don't listen to such news anymore, it starts the day of on the wrong note, stepping into the office with negativity and stress. Don't forget the stress of driving on SA roads...

I was also surprised how people tend to bring their home issues into workplace as well, affecting how they show up, perform, etc. My mindfulness, listening and empathy skills have seen an accelerated growth since 2011...

Still, eight years on, I still see it a lot of whining in the workplace...come on folks, stop whining. 

Whining never wins. 

Bring your best self to work, show up and get through the struggle...

Stop whining. "Whining never wins"...

Don't get me wrong, I've got loads of empathy...but at some point, people need to step up and just stop whining as whiners never win!

On using metaphors for Tech Ops

I like finding metaphors from other worlds outside of technology or engineering that can help simplify concepts for business people. Using such comparisons "is like a..." can actually be quite a powerful way of connecting or contextualising, especially when these people (business stakeholders mostly) are not interested in the technical details at all. All these folks care about, when there's an operational issue impacting business/customers, are: "What's the issue? Why did it happen? When will it be fixed? I need 15 minute updates please. Not interested in detail. Just want to know when it will be resolved. And don't forget Root Cause Analysis RCA document & Consequence Management!" When I get these remarks,  I'm like...left dumbfounded...and decided to find another way of communicating to manage expectations better...

Enter...
#draintheswamp
#ICU

#draintheswamp

Despite the term being popularised by Trump last year, which was incidentally around the same time I introduced the term - in the software engineering community, "draining the swamp" is used to describe some refactoring and possibly restructuring of code and architecture, to address key bottlenecks, instabilities and optimisations. This could also mean revisiting original design & implementation decisions. Some people in software might put this under the bucket of "paying down technical debt" whilst others might just put this down as a natural part of the software evolution.

I used this term to prepare business people on the impact of the changes we planned to make. This was in response to a business ask of anticipating the biggest load of the tech platform stack - will the system cope with a "Black Friday-like event"? The simple answer was NO, not in its current form, that we would need to make some aggressive changes to the platform - like scaling out to multiple datacentres. The stack was built as a monolith (multiple services & components, some micro services, running on traditional VM infrastructure, not containerised, etc.) and so the best chances of scaling for load was to move the stack to run off multiple data centres, where we could scale up on each datacentre as well, but still had bottlenecks with share database cluster off one primary datacentre.

Suffice to say, this stuff goes above the heads of the business people. So I said, it's like draining the swamp. When that happens, we are bound to find some nasty things lying beneath the surface, things we don't see today, and can only see when we start draining the swamp. When this happens, and we hit some boulders or some unforeseen monsters of the swamp, we are bound to experience downtime and an outage or two. Don't panic, this is expected, given our working constraints, that is, make the changes on live production environment.

#draintheswamp went down pretty well with business. They got it. Left us alone. They trusted us! They managed the customer & business impact when we needed help, and our tech teams worked heroic hours to get the stack running off multiple datacentres, just in time for the biggest event of the business for the year (2018), our user numbers reached record highs, and the platform did not fall over! So round one of #draintheswamp was done, but the swamp was not completely drained out...

Until earlier this year (2019), when we kicked-off another round of #draintheswamp to start the migration to cloud stacks, starting with containerisation...which led the platform to suffer a series of outages at the most inappropriate moments...customers were pissed, business impact teams were on the back-foot, social media was killing us, the changes for #draintheswamp was starting to kill the platform...when I resorted to introducing another metaphor: #ICU. Folks, the tech stack is in some severe TLC, we are instigating #ICU mode. Life support is initiated, vitals are not great, but patient is surviving, and needs high level of critical care...

Technical Operations / Site Reliability Engineering is not so different to running a hospital's ER Emergency Department...at any time, despite monitoring vital signs of existing patients (system components of tech stack), or doing day-to-day operational management of the ward (infra maintenance), an event could occur that can just spike, like a natural disaster, accident or terrorist attack (hardware failure, critical component dies, database crash unanticipated, etc.)...ER staff have to triage quickly (Tech Ops also have to triage), make life/death calls (Tech Ops when to call a P1 incident & inform business), decide on severity of the injury (requires immediate attention, operate now, or can wait??)...the same holds true with bringing back a technical platform from death to living healthy operations...so why wouldn't we want to reuse medical terms?? I think it makes perfect sense!


#ICU

The gist of this metaphor is simple: ICU/CCU means Intensive/Critical Care Unit. When a patient is in ICU, it is serious stuff, urgent priority, focused attention to monitoring, diagnostics & multiple treatment options, using whatever means necessary to enable a positive outcome for the patient.

The typical flow to recovery to health looks for these transitions:

• Start in ICU/CCU, remain there until a period of time where interventions applied & life-support is no longer necessary, then...
• Transfer to High-Care facility, remaining close to ICU, but stable enough to warrant reduced focus and attention as compared to being in ICU, but be prepared for surprises, so best be close to ICU ward and not far away...wait until doctors give the green light to move on to...
• Normal / General Ward...the last stay before checking out to go home. Vitals and all other required checks all pass before discharged for home...

Although being in #ICU is rather stressful for everyone, there is a high sense of urgency, and the pressure to respond to business is quite intense (especially when Risk/Governance demands "Consequence Management"), we can draw additional parallels from medical ICU, like:

• Remaining calm, address the topics / unknowns in logical manner, using tools of diagnosis you've trained for (Technical tools like Five Whys, etc.)
• Running tests, take blood samples, etc. (Tech/Data forensics, logging, test hypotheses, etc.)
• Call on other medical experts to bounce diagnosis / brainstorm (Involve as many tech experts to offer new perspectives)
• Implement treatment plan according to hypothesis, wait for new results (Fix something, wait, analyse result, before making additional changes).
• Communicate clearly to patient's stakeholders (Communicate to business stakeholders transparently without hiding or being defensive...come clean).
• Pray :-)

Unpacking the medical terms...

According to Wikipedia, an ICU:

An intensive care unit (ICU), also known as an intensive therapy unit or intensive treatment unit (ITU) or critical care unit (CCU), is a special department of a hospital or health care facility that provides intensive treatment medicine.

Intensive care units cater to patients with severe or life-threatening illnesses and injuries, which require constant care, close supervision from life support equipment and medication in order to ensure normal bodily functions. They are staffed by highly trained physicians, nurses and respiratory therapists who specialize in caring for critically ill patients. ICUs are also distinguished from general hospital wards by a higher staff-to-patient ratio and to access to advanced medical resources and equipment that is not routinely available elsewhere.

Patients may be referred directly from an emergency department if required, or from a ward if they rapidly deteriorate, or immediately after surgery if the surgery is very invasive and the patient is at high risk of complications

When a technical platform goes into #ICU, we do the following:

• We're on red alert - life threatening to business (customer experience is tanking)
• Stop all other development work, or reduce planned work as much as possible by pulling in all the people we need to help (speaks to higher-staff-to-patient ratio)
• Pull out all the stops, bring in experts, tool-up with advanced resources and equipment
• Communicate daily to all business stakeholders
• Perform multiple surgeries if needed (hotfixes, patches, etc.)
• Run multiple diagnostics in parallel (think Dr. House)

According to Wikipedia, a High Care/Dependency Unit:

A high-dependency unit is an area in a hospital, usually located close to the intensive care unit, where patients can be cared for more extensively than on a normal ward, but not to the point of intensive care. It is appropriate for patients who have had major surgery and for those with single-organ failure. Many of these units were set up in the 1990s when hospitals found that a proportion of patients was requiring a level of care that could not be delivered in a normal ward setting.^[1] This is thought to be associated with a reduction in mortality.^[1] Patients may be admitted to an HDU bed because they are at risk of requiring intensive care admission, or as a step-down between intensive care and ward-based care.

According to HealthTalk, the last point to recovery is General/Normal Ward:

People are transferred from the intensive care unit to a general ward when medical staff decide that they no longer need such close observation and one-to-one care. For many people, this move is an important step in their progress from being critically ill to recovering..

'Nuff said, IMHO the similitude mentioned above should be more than self explanatory ;-)

On company values

"Disagree and commit"
"Embrace the elephant"
"Don't be a d!ck"
"Listen. Challenge. Commit. A good leader has the humility to listen, the confidence to challenge, the wisdom to know when to stop arguing and commit."
"We use data to drive decision making. Data-driven-decisions."

These are are just some of the catchy phrases that some "modern" workplaces aspire to implement as company culture....great words, but easier said than done in practise IMHO. There's this somewhat unreasonable expectation that human beings can just implement this stuff - but we all know too well, humans are emotional and predictably irrational...

If you're a leader...
You're bound to issue instructions, ideas or directives that are not going to sit well with your teams. You have a duty to listen to feedback, alternative points of view, and use these as inputs into your thought process and ultimately make the call. It's not about paying lip service and say "I've listened" without truly listening...what if you're wrong, and you might even be the HIPPO in the room? Even if your decision doesn't change, and you set the directive, you're ultimately responsible and accountable for the outcome.
What if the directive appears to be wrong? Do you stick it out till the end because of your ego and risk losing credibility? Some might say a real leader will step up, and own up saying "Hey, I was wrong guys, but we gained a lot of learning from this."
Still, as a leader, you then trust (expect) your team to follow-through no matter what...but is this even as simple as it seems? Probably not...how do you behave when you find out that there's still resistance? Do you flip your lid? What kind of a leader does that make you? Some might argue that being decisive, forthright, firm and tough on naysayers are great attributes for a leader...."the buck stops here!".

If you're on the receiving end of the instruction / strategic directive and you disagree...
You've said your piece, provided feedback, you may have even fundamentally disagreed with the decision...and now you're faced with understanding yourself: Can you let go, can you commit, even though you violently disagreed? Are you serious about the best outcome for the team regardless of your personal opinion? How will you stop yourself from unconsciously falling back into dissent-mode?
As a team player, your leader expects you do so. But have you prepared yourself to work towards that outcome?
How do you stop yourself from sounding like a stuck record?
My humble advice: stop being this guy. If you can't let go, then you're limiting your own growth.
BUT...ask yourself this: What am I willing to walk away from?
If the directive conflicts fundamentally with your core professional or personal value system, then what do you do?
My view: if it gets to that level of personal dilemma, and you're so sure of your value system, then leave, quit the company altogether, or change teams...it depends on how serious you are about this value system of yours.
If the conflict is not even near enough to compromising your value system, then ask yourself how can you adapt your own behaviour & approach, how can you help to solving the problem, and how can you help influence the outcome? How can you show your leader you're committed, no matter what?

This isn't always easy...the emotional forces at play can be intense, pulling you back...but once committed, your leader is expecting following through and be a team player.

Both Leader and Follower need to have a keen handle on self-awareness, "Know thyself"...some call this "mindfulness"...

On choices, decisions, initiative, career planning: lessons, looking back

Image Source

At a recent town hall with my technology division (100+ people: frontend & backend engineers, platforms & infrastructure systems engineers, system architects, agile specialists, AI/ML engineers, Agile PMO & technical ops monitors), I opened up by giving my perspective on expected behaviours & responsibilities of the individual, especially when it comes to career development & growth aspirations.

The message was about taking a personal ownership for one's own growth, rather than leaving it up to the company or one's manager (a message that surfaced a few times in recent OfficeVibe feedback) - that the responsibility largely lies with the individual. Yes, managers/leaders are there to support you & guide you along the way, but only you know what defines you as a person, so don't leave it up to others to determine a path for you...

Reflecting on my own journey, it all comes down to understanding your current reality, weighing the choices on the table, defining your aspirations, taking initiative,  processing & reflecting through assessing the outcome of the initiatives, finding great people to learn from, tracking your trajectory on the path to growth. The path is not always clear, sometimes adjustments need to be made, sometimes a little backtracking is needed to enable the next leap forward - still, it all comes down to one's own personal ownership & level of commitment to controlling one's own future.

I thought I'd share my own timeline as guidance for people who might be stuck. Interestingly enough, although I only recently started formally implementing my own management framework around life/work planning by way of my RAGE model, that I was actually instinctively using this decision-making model all the time.

My timeline table shows the major periods in my career, commenting on reality of the situation at the time, choices I faced, decision made & eventual outcome. I think anyone who's considering what to do next with their career plan should do a similar exercise for their own sense-making.

My Perspectives

1. Gain a keen appreciation for your current situational reality and take responsibility for it. Yes, reality sucks sometimes, but you got to play the hand you're dealt, don't let that get you down.  It is possible to change your reality. I made hard choices based on my reality of being caught in a low-income family, living through Apartheid.
2. Discover your key motivations and use them as your guiding compass, some call it your "value system". I believed in myself and my ability to make things happen. If I felt my knowledge was lacking, I would learn & close the gaps myself. Don't assume you know everything, there are tons of smart people out there. I got a huge awakening when I went overseas, so much so that I had to learn software engineering & computer science all over again.
3. Don't go seeking hand-outs or help, but if people or companies do extend their generosity, don't naively turn them down. There will be good people helping you along the way. After trying many avenues of financial aid/scholarships without success, I thought help would never come my way, but it eventually did.
4. Don't bog yourself down with "If Only", or "What If" - this creates negativity & unnecessary anxieties. Move on, look forward. Sure, reflect on the past, learn from it, but never let it hold you back. You're in control of shaping your own reality. I chose a path that was the most practical, I switched jobs just as I was going to be promoted, I left big projects just as they were about to land, I left a start-up thinking I had a job lined up (but it didn't happen), I left what others would say is madness (left the stability of UK to return to volatile SA). Leaving UK was very difficult for me from a professional experience sacrifice, but I never allowed doubt and negativity to bog me down.
5. If you want a good measure of your skills or experience an alternate reality, leave your country & work overseas. Even though the world has gone smaller through globalisation, that even in South Africa we do work with international teams, I still think getting overseas exposure is one of the best things one can do. Living and working in different countries exposes you to a different world of experiences. If you're under the age of 35, then you should try it. It doesn't have to mean relocating or emigrating, it could be a temporary secondment for a year or two. I was fortunate to experience working with many cultures across the globe on some really big projects. I learnt so much in a short space of time, it took my engineering skills to another level. As for Planning, Management & Execution principles, in my opinion, the UK ethic is world-class.
6. Become comfortable with uncertainty & embrace the unknown, even it means leaving your home town/country for another one. You get this only through experience, and having been through at least one transition into the unknown. I've seen a few - it's not so bad, you must embrace your fear of uncertainty.
7. Develop a learning & growth mindset - in any new role, work hard to learn as much as you can, by reading, studying, latching on to people as mentors, read other people's code through open source projects, etc. I became expert in a few areas: MPEG/DVB protocol spec & implementation, C & C++ coding, Voice synthesis & Text-to-Speech, Project, Agile Program Management & Execution, Professional services consulting and more recently Leadership skills. This doesn't come easy: I read a ton, implement the tools, learn from experienced, the wise, still remaining open to new experiences, no matter how edgy they might make me feel.
8. Be ready to start-over again more than once - switch roles, domains or industries, sometimes what might seem to be a step or two backwards, actually turns out to be better than hoped. I started over at least 5 times in my career of 20 years. If you're in software, sure you can specialise (and there's nothing wrong with that) but you must then become expert at what you do. To grow in software, my view is to learn as many tools as possible, switch every two years. One of the best ways to do this is side projects, open source communities. Don't wait for your company to reserve hackathon sprints, or 20% time - take ownership. I taught myself text-to-speech synthesis on my own, developed POCs in my spare time and proved to company the potential innovation. If I had not taken initiative, I probably would not have landed the ultimate technical role I dreamt of.
9. Don't pass the responsibility for your career on to someone else (your manager or company), rather you should have a view of your own map, your end goal. Your company or leaders can help with options available, guide on the gaps you need to fill - and it's even better when the company has a decent career ladder in place. Never pass the buck on and make excuses that your company / manager does not care, that you don't have enough syncs 1:1s or feedback sessions with your manager. You need to take ownership, period. In the companies I worked for, we at least had a decent career-ladder in place, showing all the upwards, sideways opportunities available. I made it clear to my leaders at the time, that my ultimate goal was to become a "Jack of all trades, but Master of SOME" T/PI-Shaped skills. They knew that when I'd enter a new role, I would learn, produce outcomes and then move on, thankfully, I had very good leaders that did not stand in my way. If you feel obstructed by your leader / team / company, first dig deep within yourself to reflect on whether your own behaviours need improving, and if you're still convinced it's not you, then leave, change your environment, change your circumstances.
10. Don't get complacent or too confident your role is secure, retrenchments & redundancies are a reality, business-is-business. I got my first taste of layoffs when I was still a junior software engineer, naively thinking I was in a good place by virtue of being part of a cool new product team, and owning some key components. Since that first-and-only layoff (in 20 years), I developed my "spider senses" - and decided that it would always be me that decides whether I stay or leave, not a market event or the company.
11. Do not grow an entitled mindset, or have unrealistic expectations from your employer. Say you studied hard and earned additional paper qualifications: MSc, MBA, PMP, etc. Don't expect the company to automatically increase your salary or grant you a promotion. Say your own personal life changes, you get married or have children, so you have more responsibility at home. Why should you expect your company to give you an increase, if the work you're doing hasn't materially changed, or your output is still the same, and you're still working at the level the role expects?? At the end of the day, it's up to you to manage your personal circumstances, it's not the company responsibility now to just automatically reward you or make your life easier financially - NO - you have to work at it. If you gained new qualifications, you need to show an interest in contributing your newly acquired knowledge, showing value. If you're seeking a promotion, you must show you've actively contributed covering much of the roles/output of the next role you're seeking. Companies don't owe you anything - so don't come up with unreasonable expectations or feel entitled. It's all about your output, meritocracy  is the only thing that matters.
12. Becoming your own boss, running your own consultancy is hard work, be prepared to fail in this area. Although branching out on your own can be enormously liberating & exciting, unless you have a large network to tap into, moving from one consulting engagement to another, building up clientele & a pipeline of work, growing your team - whilst a lot of people have made this a successful venture - it is actually quite hard to do. I was fortunate to secure the company I left as my major client, although, the client only wanted to work with me, so in effect, I never really left! Without a strong network, the going was tough trying to break into other clients, even after doing some pro bono consulting work. You must invest a lot of time & energy, unpaid hours to build your own consultancy, something I didn't do, which showed I wasn't really fully invested in this venture. So I shut my company down, and was pulled back in by the strong gravitational forces of the big company. I learnt a great deal, became a better salesman, and became confident in interacting with C-Level executives. Consulting however, is a sure way to make extra money than being a permanent employee, but it comes with its own set of risks.
13. Show gratitude. Whilst you might think that you're in control and the result of your successes are due to your own hard work, sweat and tears, never become arrogant and ignore that other forces helped you get this far. Take time to seriously reflect on this, and you will soon identify people or events that helped, and when these surface - be thankful & show your gratitude, develop humility. Although I was brought up in low-income household, I never once felt not having a complete home, or solid upbringing on life skills - if anything - this actually shaped my personal motivational value system. I never regretted or blamed my parents, I had a good childhood, was taught responsibility & key life skills. I've acknowledged people, friends and family that shaped my reality, the leaders in the various companies I worked with, were supportive, friendly and encouraging - I learnt so much from them, and still continue to learn from them today. Sometimes, when you're in the thick of the day-to-day job, you might not like what your manager says or does (over controlling, micromanaging, lecturing), and it's only when you leave, you realise the wisdom and lessons being taught. It's not always about you, and if you're leading teams, show appreciation for your teams as well. Your success is a result of your team's output. As you develop into senior roles, your visible output might become less-and-less, but you're still working hard through people, in the background. Never think you're the sole reason behind success - there's so much more that goes on, that we're often blindsided - don't get blindsided, actively seek out your blind spots.
14. Be patient. Patience is linked to gratitude. Be patient with your role, allow enough time to learn the essence of the domain. Once you're comfortable & confident in your appreciation of the essence or core principles and you've remained long enough in the role to complete 1-2 major pieces of work / projects, then allow yourself the opportunity to move on. But don't rush things through, learning needs time to soak. Personally I'm always doubtful when I come across people's CVs hopping from one permanent role to the next in less than 18 months (this is because the major initiatives usually run for at least 18 months). From my experience, this (9-15 months) is just not enough time to do justice or have made a serious contribution in terms of outcomes (unless the gig was to rescue, recover or revive a project as a major intervention, or consulting gig). For me, it's been roughly a minimum of 18 months provided I felt confident in my results. I just about completed my engineer-in-training role after university when I took a big chance, although successful, in retrospect, I had big gaps to close anyway. The more higher one climbs the ladder, the more patient one needs to be, which means 18 months could grow to 24-36 months minimum. Currently I'm resisting the urge to switch, knowing that I still have another year to go before I can claim to have truly owned the role, so patience becomes a necessity.
15. It's not always about the money or job title - neither does "years in role" contribute to "seniority". Although I might risk passing a value judgement on other people here, what I found is that money should not be a driving motivation, if you've set your sights on a learning and growth mindset. Sure, you can hop from one job/company to the next every 12 months or so, on each move your salary jumps - but if you're effectively not learning new skills or growing, is it worth it? Job titles are also relative, what's in a name after all? What matters is what you can do, and the value you bring to the table. What I found also trips people up is this complex of "seniority" based on "I've worked so many years in this role, hence I deserve a promotion as a senior xyz". In my journey, I sacrificed salary growth for knowledge, experience and a wide/deep toolbox of skills/capabilities. Later when things became challenging financially, other opportunities opened up that boosted my income, which wouldn't have been possible if I'd not honed by capabilities & demonstrated value as a result of experience. I once had a manager who, in his previous company was a VP of Engineering only to become a development manager in his next role - two steps back. I'd asked him why he made the move, this was when I learnt that work/life isn't just about the title. He restarted two levels down and in a short space of time, was back to being a Director of Engineering. I had a team leader once who was performing in the role of manager, but was so humble and patient that having the title did not bother him much. I have seen engineers who effectively remain doing similar activities for years, expecting a promotion by virtue of being doing the same job, even though the competencies haven't grown (e.g. influencing group/country/global teams, taking ownership, showing initiative) - so if you want a promotion, you need to earn it! I've also seen people who are content being a software engineer (with no aspirations of seniority/leadership), but who are expert at what they do, adding so much value, that the company provides enough incentives to keep the person happy (at times a brilliant software engineer could earn a higher salary, have indirect influence greater than his/her manager). At the end it does come down to personal motivations, and when the time does come around to being a financial/happiness constraint, then don't expect the company to help you - it's time for you to change!

My Timeline (Click here to see full table)

2018 Year In Review & 2019 Thoughts

It's that time of the year again where I look into the data for timekeeping over the past year to make sense where I spent my time, and cross-check with reality, in terms of my aspirations and goals that are split between different areas of my life by Personal & Professional areas. The idea is to gain insights from this data and re-calibrate areas of focus for the upcoming year.

Unfortunately, I haven't made much in the way of drastic progress in 2018 compared to 2017, the profile is largely the same - that my professional (work) life has consumed more time (a constant theme), my personal life in terms of family commitments are reasonably on track, and the biggest area of neglect is on my own personal, individual streams (hobbies, ideas, health and fitness). However, since I started the RAGE model experiment, I have made some good wins on other areas, so the model works, but there's still more to be done in this area - specifically around outcome-based goals.

Recap my RAGE Model (Reality, Aspirations, Goals, Expectations) Tracking

I wanted to map out the different areas of my life that defines me, split by two major areas: Personal & Professional. 

For each area, I defined some personas or streams. For example - in the Personal area, I'm a Husband, Father, Family-Man, Individual, etc. For the Professional area, I'm a professional (my main job), a colleague, etc.

For each Persona, I map out my current Reality, ideal Aspirations, and then from those two, set some Goals & align some Expectations to achieve.

Another aspect to this experiment was tracking my overall happiness and well-being. If I spent time in the areas of priority, the theory is it should count towards me being happier.

I prioritise the Personas using a simple matrix that defines how I value and rank them. I then use detailed time tracking to track the time spent doing activities for each area. The theory is that I should be focusing time and energy on the areas that reflect the ranking and priorities I set. This is the core data for analysis. For more details on how I actually do this, I am still using the manual ways as described in the original posting. Since the original version of the Personas, I'm now on version 6 which currently looks like this:

Current Persona Matrix Rankings

What this ranking does is basically show my decision-making rules: Life (Husband, Father, Family) is more important than Work (Job), which is more important than my Individual Personal streams (Hobbies, Pet Projects, Me Time, etc.). A simple-enough value system that I use to navigate through challenges.

It all looks good on the paper model, but in reality, the data highlights some food for thought that I really need to take action in 2019.

What are the big challenges for me?

What triggered my creating the RAGE model and start the detailed time tracking activities was essentially about accounting for my time, addressing the hard realities:

• Being a professional, i.e. having a stable income stream is currently a hard reality constraint. I need an income to support my family - so this is something I should not compromise on - but I still need to control it - I should not be overworking, at the expense of my personal life interests. So the data on work hours is important. When I read about pursuing dreams, passions, etc. about breaking out, taking risks, quitting your job etc. the reality around finances, lifestyle and family stability kicks-in - it's not an easy thing to do...this is a hard reality. I have to think of ways to ensure I do my best work, provide the best service to my employer, but not let work over consume me at the expense of my other streams (which as the data shows is not actually happening). Unless I can find a way to create a passive income stream outside of my main job, this constraint is going to be around for a while. The outcome for this year must be to bring my work-hours to a decent level freeing up time to focus on my personal interests.
• It's quite difficult to switch off from the office these days - even whilst at home, social media chats through Slack & WhatsApp are disruptions, that can't be tracked in hours - so whilst in "family mode", I must respond to chats - this time is unaccounted for. If I were to include all these moments as "work", then work-time increases...which is why this topic of "work-life integration" is becoming more mainstream.
• The functions of resting and sleeping for a human being is something that must be respected, and can't be compromised. I maintain an average of 8 hours of sleep per night, so this is a hard constraint. Eight hours has been validated as the ideal, I can go on for a few days with less sleep, but it's not sustainable. This means that the rest of my time must be allocated around this, with Work/Professional consuming a large chunk of this.
• Giving focus to my own personal interests as an individual, outside of being a Husband / Father / Family-Man, etc. seems like I'm sacrificing my own interests, i.e. putting myself last - and this is where the spiritual dimension of being a Muslim comes in to help - it's sometimes a struggle to deal with these frustrations.

2018 Data Insights - Tracking the Time / Hours spent

There was roughly 8,766 hours in 2018 (365.25 24-hour days), which I tracked and logged a total of 8,820 hours, which is not bad, the errors could be attributed to human error in logging (54 extra hours).

Where did the time go?

2018 Persona Hours, focus on Working Hours & Overall Happiness at work

There were 250 work days in 2018, excluding 12 public holidays. I took 34 days off as leave days, which left 216 work-days. Assuming an 8 hour workday, the capacity for work hours would be 1728 hours.

My actual work hours logged were 2155.4 hours, an excess of 427.4 hours as "overtime" which I don't really get paid for. This equates to an extra 53.4 days worked, an additional 2.5 months of extra work - which took away time from other areas, like for instance, time I could have used for my personal projects.

The table below summarises the major streams - highlighting in red, the months where work-hours exceeded the normal bounds. Overall split between life and work remains consistent with the previous two years, with my own personal individual streams taking much of the knock:

So it's clear that work-hours is an area to be controlled to free up time for allocating to personal, individual streams.

General Happiness & Well-Being at Work

I continue to track my overall happiness levels at work on a daily basis. I have also started tracking the bad days in terms of sleepless nights. In 2018 I had my fair share of stress at work that resulted in some nights where I hardly slept a wink, tossing and turning, worrying about issues at the office as well as my future in the company. This is a new stress that introduced itself last year, that I need to better manage. I tracked 12 sleepless nights, this is about 4% of the time. However, in general, the picture still looks good - I'm enjoying the work on the whole:

The last quarter of 2018 did see an increase in bad days - we had some challenges with the platform in terms of stability, new boss, changes in teams, and overall tensions and conflicts to manage & people resigning. This what triggered my blog post on workplace 3.0 last year. Additionally, my health took a bad turn as well in November. The number of "indifferent / neutral" days seems to be increasing - an area to keep track of. But in the grand scheme of things, there is more upside than downside at work, so this is still looking good! However, I do need to plan in personal time (leave days) more in the year to maintain a healthy balance.

Life in general is good, but my Personal streams are taking a knock

In general, overall - the time keeping does not contradict the priorities and rankings for my personas. I am also in a good space - health, wealth and mental space is in equilibrium (maybe this is because I've just done three weeks of of chilling out, relaxed summer vacation - back to office come Monday...). As I introduced the post earlier, on paper, I should be happy that my hours are reflecting the priorities I set & committed to...however, I can't help but feel a little down that overall my individual self interests are taking a knock...

In terms of the streams I see as my personal individual persona (i.e. the things I'm interested as person, outside of me being a Husband / Father / etc.), here's a view of the percentage focus per month (still highlighting the months where work exceeded the norm):

The positives highlighted in green:

• Reading books - did well enough to track against my goal of 24 books, ended up completing 21 books last year. Not bad.
• Spirituality - tracking very nicely, spent 10 days in seclusion in June, time well spent.
• Me Time - this is time I spend by myself, thinking, exploring or even binge-watching shows ;-)

The areas of concern are:

• Health & Fitness - dismal year. I had some health issues creep up on me last year. Not good.
• Cycling - very little done. According to my Strava logs: 29 days, 47 hours, 585km.
• Blogging - regressed big time - need to do at least one post a month
• Entrepreneur - my ideas still collecting dust. Pathetic, makes me really wonder about this!

Major Outcomes for 2019: Intent

Broadly speaking, here's some outcomes I'd like to focus on this year, the detail in terms of specific targets is still tracked manually in the spreadsheet template:

• Professional
• Get work-hours down to a manageable level, target no more than 170 hours per month
• Continue to build a strong leadership team thus removing me as a bottleneck, focus on strategic outcomes, less operational
• Spread in more leave time, reduce sleepless nights to zero
• Personal - Individual - Entrepreneur 
• Spec out my Rage/Personametry product idea and get an MVP done, even if it's manual
• Attend at least 6 Start-Up meet-ups this year
• Personal - Individual - Health
• Improve current fitness state - there should be no 0% activities in a month
• Personal - Life & Family
• Introduce consistency & routine into happiness building activities - at least once a month we should develop a routine around the things we enjoy doing as a family

Am I hunting antelope or field mice?

I came across this question from Tim Ferriss' Tools of Titans during my summer reading, it's quite apt  for my first post of 2019, as I have a feeling it will probably be the question I will pivot around again and again during 2019. It features as question #13 of Tim's 17 questions that changed his life. I've lifted the description here on this blog so I can keep coming back to it.

Am I hunting antelope or field mice?

Quoted from Tools of Titans, 17 Questions that changed my life:

Newt Gingrich is one of the most successful political leaders of our time...Now that he's in the private sector, Newt uses a brilliant illustration to explain the need to focus on the big things and let the little stuff slide: the analogy of the field mice and the antelope.

A lion is fully capable of capturing, killing, and eating a field mouse. But it turns out that the energy required to do so exceeds the caloric content of the mouse itself. So a lion that spent its day hunting and eating field mice would slowly starve to death. A lion can't live on field mice. A lion needs antelope. Antelope are big animals. They take more speed and strength to capture and kill, and once killed, they provide a feast for the lion and her pride. A lion can live a long and happy life on a diet of antelope. 

The distinction is important. Are you spending all your time and exhausting all your energy catching field mice? In the short term it might give out a nice, rewarding feeling. But in the long run you're going to die. So ask yourself at the end of the day, "Did I spend today chasing mice or hunting antelope?"

Initial Thoughts?

Courtesy

Five years ago, I used this illustration to pitch my situation at work to my then bosses - I felt like a lion trapped and being tamed by a corporate structure that was stifling the value I could bring to the organisation especially in terms of working across group-silos. This pitch resulted in me being freed up from the HR constraints and kick-started my journey into consulting which lasted about 4.5 years...

Then having spent enough time consulting, it felt more and more that I ended up hunting field mice again (consultants were excluded from big meaty execution decisions and did not have a seat at the table), so I went back into permanent with the same company...I am getting my share of antelopes now with a seat at the table, but more often I find myself  being dragged into field mice problems. My solution to this problem is through effective delegation and empowering my people, the challenge is that my peers and bosses have trouble appreciating this tactic...what I view as field mice problems is not always seen in the same light by the people that further up in the food chain - so maybe I'm not a lion after-all ;-) or maybe I need to find another pride of lions with the same mindset as I...??

It might well be that my current job is not the antelope I'm searching for...that my real antelope is still out there evading me, and could it be  that my fear is the only thing that is holding me back??

So that's thoughts on professional life...in terms of family life, this story could also be applied as well. Being a husband and a father to three children, whilst very rewarding on levels that can't really be measured or quantified well, is probably the most challenging experience one can have...in my case it's about letting go of being a control-freak, not sweating the small stuff (field mice) and focus on the bigger important stuff...whilst I'm an expert project manager in the professional world, the project of a family and raising children is THE most intense project to manage in life...

2019 must be the year I either bag an antelope or at least make strides in identifying bigger game...My RAGE model and my focus on the 80/20 principle in all aspects of my work and life are tools that are helping me on this journey - and if I were to assess my progress since starting with my RAGE model tracking, then things don't look bad at all.

My gut tells me that 2019 is going to be an interesting year for me professionally...

[Update: August 2023 - I did indeed bringing down my antelope in 2019, reaching heights in my professional career to C-level, as well as step-changing my family-life away from field-mice problems...but the interesting thing now in mid 2023, feel I need bigger antelopes again, since I'm in Africa, I need to hunt a buffalo]