On: The Office Life

another one of my #thismightnotwork posts (inspired by Seth Godin)

Deep down...

We know that nobody owes us anything.

We know that we are just cogs in a machine, replaceable.

We know that company loyalty does not really exist, no really, it IS about the bottom-line!

We know that we are only as good as our last project, even though ten projects earlier, we shot the lights out.

We know that no matter what we like to believe, most relationships in the office are simply transactional, although we would like it to be deeper.

We know that it does not matter how much we try, we can't (neither should we care to), change people's perceptions or deeply entrenched biases.

We know that mediocrity can be contagious if we stick around for too long, and unable to really influence change in performance and behaviour.

We know that tribes, cliques and clubs exist, it's natural (especially in Africa when it comes to racial divides), but we either feign ignorance or hope it gets better.

When a company value is "give benefit of the doubt" and we don't see it in action, what taste this leave us with?

We know that we are so much bigger than just our jobs...or do we really now (hint: count the number of times per day you find yourself immersed in thoughts about the office, even during your personal time)?

It's all about the bottom line....

It's business they say, you must develop a thick skin.

New age leadership is all bullshit they say...it's capitalism and darwinism all the way man...power, politics and Machiavelli are role models of the day.

Empathy is so overrated, they say.

Empowerment? Let the team decide? What's all that fuss about? 

Humility, Modesty, Authentic, Integrity, Fairness...that's not leadership they say...but great for selling lots of books & makes for a booming consulting/coaching industry...but, kid, they won't get you through the real world...skin in the game is what you need, and be damn sure to fight to the death to protect it...

Attitude...attitude is good as long its conforms to groupthink aligned to culture (not the culture deck written down, but the real culture, yep they're different!).

...Yet knowing all of this, there are some people who still stick it out...

Wow, there walks about a man with grit & resilience they say...

...an immediate assumption is: this guy must keep a cool head, he needs the money/income, responsibilities he has to his family, etc. comes first...just see it through, it will all be okay...

Dig a little deeper, and we often realise it's usually much more than just that....it's quite personal actually, positively personal...sometimes deeply inspiring, confusing & at times bewildering...

Such people have a cause, they're fully in tune with their why, their self-worth and are completely aware of  not only themselves but of those around them. They know they stick out, are nonconformist, often mistaken as a threat to the status quo & risk being played out of the system...yet they still remain behind, firmly footed, digging their heels in - why, mostly because they take commitment seriously & sincerely. Such are those people, who believe in their craft, make their art, do things differently because they truly care deeply, and will not leave until they say "my work here is done, I've come as far as I'm willing"...they don't leave through external forces or pressure, instead they leave on their own terms, when they're ready to leave it all behind and never look back. And often to their surprise, they've built up a tribe, left a following behind...even though they didn't intentionally start out that way.

If you'd like to find out more about these people, check out Seth Godin & Simon Sinek's work...

Highly recommend getting your hands on:

What to do when it's your turn

The Infinite Game

The Courage to be Disliked

LinkedIn Profile - probably a bad example

I spent a lot of time contemplating what should go as my "About" summary for LinkedIn, it's quite tough actually.

So I had this written down below for about 18 months as my profile, and I'm about to change it to just keep it simple and brief. Looking back, it's too much waffle..

Helping Africa's largest PayTV company build an internet TV platform, building local African engineering team & skills. This platform serves 50+ countries, multi-tenanted, multi-product & content packs. Scaled platform 6X through nurturing teamwork, MAU on a steady increasing trajectory, overall site traffic growing nicely YoY.

CTO for DStv Now, Box Office & Content Discovery Recommendations Tech stack, managing 100+ engineers: Enterprise Architecture, Software Delivery, AI/ML Scientists, Agile PMO, Infra/Networks & Ops/Site Reliability engineers.

I'm NOT strictly a builder anymore though. I haven't written code in ten years, but was once an expert coder. I don't do architecture anymore, but used to love models, abstractions, API design & authoring technical documents. I don't write or review test cases & test plans anymore but instead drive innovation & optimisation in automation & DevOps continuous delivery. I don't review code or do systems integration anymore but can write (have written) treatise on these software engineering topics.

Yes, I'm building an E2E tech stack commanding a v.large budget, in a fast-paced VUCA world.
Yes, I handle tech conversations & make tough decisions, engaging and challenging C-level executives.
Yes, I do write technical papers.
Yes, I can indeed get into engineering detail too...BUT...

I CHOOSE to LEAD through PEOPLE: empower, protect, listen, steer, guide, groom, mentor, coach, train, show & tell, nurture-to-catapulting NextGen leaders we so desperately need today...

Some people call me a turnaround specialist - rescuing distressed projects & teams...
Others have said I have a knack for making the complex look simple...
I've built full stack software from device firmware, middleware, apps to server-side systems.
I was once an expert technical project and program manager, but that too is no longer who I am...
I was also a scrum master, agile coach, devops & agile delivery release manager too...I was also a senior management consultant...

So who/where am I now??

I am now a builder of LEADERS, who happens to be a senior engineering guy with PI-shaped skills, who loves to solve perceived intractable problems and thinks big: org, people, radical ideas, disrupting, transformation.

I'm known to challenge status quo, ruffle feathers & doing things differently - radical candor. Sometimes seen as a threat to old-school thinkers.

Autonomy & Trust is where I thrive - democratising the workplace!

But I tend to get bored easily if I'm not stetched or challenged enough. Typical cadence is 3 years before changing! 

On Leadership: Be Like Water

So I took a chance this morning and posted my first article on LinkedIn:

We talk a lot about transformation these days. Executives want to gear up for the future, speak the latest buzzwords"agile, flexible, adapt-to-change, level up for the digital world, challenge-status-quo, call-things-out" etc. All great intents I suppose, but what is sometimes ironic is that these same people are actually unwilling to let go, unwilling to adapt, maintaining the old-school mentality, staying within their comfort zone, symptomatic of being too afraid of conceding their power.

I've been in a state of continuous adaptation for as long as I can remember, my own leadership style has flexed and morphed along the way. It wasn't easy. It's actually very, very hard. Changing habits, value systems and mental models can be straining intellectually & psychologically. Practising self-awareness (i.e. embracing change in your leadership style) reminds me of the epic battle with Gandalf the Grey & the Balrog in Lord of the Rings...

Rigidity is a kin to brittleness. Brittleness leads to breakages. When things break, it's not always possible to put the pieces back together again. Fluidity is thus an important trait for today's leader. If you insist your ways don't need changing, then I'm afraid you'll soon realise "being out-of-phase, completely misaligned & out-of-sync" with the tunes of the current workplace vibe.

Sure, hierarchy will not disappear anytime soon and thus deserves respect. Sure, we need visionary leaders, but more so than ever, I believe we need execs to display courage, courage to be called out now and again, a healthy willingness to stop hiding behind positional hierarchy and instead, listen more, maybe at times be brave enough to step-aside and let things "just flow", being more like water. 

Hence, I am reminded of Bruce Lee's "be like water" teaching:

"Be like water making its way through cracks. Do not be assertive, but adjust to the object, and you shall find a way around or through it. If nothing within you stays rigid, outward things will disclose themselves.
Empty your mind, be formless. Shapeless, like water. If you put water into a cup, it becomes the cup. You put water into a bottle and it becomes the bottle. You put it in a teapot, it becomes the teapot. Now, water can flow or it can crash.
Be water, my friend."
- Bruce Lee

What makes OTT Streaming platforms different to DTH/DTT STB Broadcast?

In the early days of this blog, I shared in deep detail the technical and project landscape of PayTV systems for the traditional broadcast headend and set-top-box (STB) software platforms and projects as subject matter expert in this area.

In this post, I will provide a high level overview of the core differences and challenges between broadcast DTH/SetTopBox systems versus OTT platforms. OTT has become a hot topic, with the nascent rise of OTT (Over-the-Top) internet streaming platforms that many a traditional PayTV operator is now transitioning to, in light of the growing competition for eyeballs & cord-cutters driven by the likes of primarily SVOD (Subscription Video On Demand) players like Netflix, Amazon & the like. The challenge though, is that these PayTV operators tend to trip up by bringing PayTV/DTH thinking into the OTT-world, which is why I felt the need to write about this, because I have, and still am currently experiencing these challenges. Suffice to say, it becomes very tricky to disagree can call out a high level broadcast TV executive's assertions (expecting like-for-like DTH & OTT experience) as not completely accurate when it comes to OTT engineering...

The main aim of this post is thus targeted at senior executives who come from a broadcast world, and may not fully grasp the nuances of an online streaming world, specifically pointing out the spectrum of control of the end-to-end platform that PayTV operators generally enjoyed much comfort in that world, in contrast to the new world of online, where it becomes impractical to expect the level of control previously enjoyed - basically you can't control all the components of the internet, unless you invest in owning a large part of the network, which some operators have attempted & haven't really succeeded!! Simply put, the internet was not designed for Live TV streaming unlike traditional broadcast TV - the internet is somewhat chaotic & messy to say the least. In addition to this level of infrastructure control & lack thereof, an OTT platform is largely software systems designed for internet-scale (i.e. massively distributed server-side backend components) which requires a different skillset and experiences traditionally enjoyed in the closed ecosystem of broadcast & set top box device software development teams.

Another point of this post responds to the standpoint from traditional PayTV executives who may have spent a large part of their work in building broadcast systems with an engineering mindset anchored to a past era that is no longer relevant in the OTT world. Expressing opinions such as "engineering systems either work or they don't, it's either a 1 or 0, no in between states..with an expectation of 100% always-on availability..." in my view is somewhat misinformed & borders on belittling the engineering challenges faced in the online world. Whist every responsible engineer understands such demands and expectations of any system, a key point to remember is that broadcast TV has benefited from many years of refinement, standardisation, systems & processes specialised for the domain of TV - and when it comes to the internet - well, it was never really designed for video consumption, the speed of innovation has outpaced the development of standards & so is a much more messy world compared to traditional broadcast TV (satellite/terrestrial).

Disclaimer: This post is not about downplaying the broadcast engineering world, in fact, a lot of OTT operations highly depend on the upstream capabilities from PayTV operator's existing broadcast engineering infrastructure. Broadcast engineering systems are a marvel of engineering in its own respects, systems are highly complicated & superior engineering. OTT systems, and the point of my post, is to show the extension of the value chain, highlighting the new software & systems engineering challenges that present itself in the OTT world, that will be new concepts for existing broadcast engineering teams to learn & appreciate.

Key points for TL;DR folks (especially the PayTV Exec CXOs)

• It's not a simple thing to compare OTT platforms wearing hat of a traditional DTH broadcast guy, it  is not simply a like-for-like comparison.
• Don't be naive and pass value-judgements just because you were an engineer in the broadcast TV world (Headend or Set Top Box) does not necessarily make you qualified to understand OTT engineering domain. You can't bring broadcast/STB-thinking-mindset to the world of OTT.
• PayTV operators have historically enjoyed a high level of end-to-end control, not so in OTT world. The internet is outside the PayTV operator's control.
• There are far more components in the value chain of OTT than broadcast (backend, frontend and device software stack is more complicated in OTT than STB software).
• High level of device & platform fragmentation in OTT that a traditional PayTV operator must consider.
• The internet was never designed for TV consumption & has innovated at a much faster pace than traditional broadcast TV, the internet is a messy place, broadcast TV is somewhat cleaner and more determinstic.
• OTT engineering teams have different challenges compared to traditional broadcast & set top box engineering.
• PayTV operators need to plan carefully the convergence (DTH & OTT) of engineering teams and be prepared for a "clash of worlds".
• Set Top Box software systems enjoy far more security control & protection than its OTT sibling.
• Peak events & load challenges are unique to OTT platforms compared to broadcast DTH STB (scaling challenges don't really exist in DTH/STB).
• Redundancy, Fast Rollbacks, Failover & Disaster Recovery are easier to achieve in OTT than in broadcast DTH/STB systems.

Open letter to Business & Tech Leaders

Dear Business & Technology Owners...

Have you ever stopped to reflect on why is there always tension and conflict between business stakeholders and technology providers? Instead of there being a "healthy tension" between the two partners (assuming you take the terms "partnership" seriously and not just paying lip service to the term), the relationship is often clouded with doubt, suspicion & mistrust on all sides?

I've been in this technology software business for all of my professional life (~20 years), spending the first half working on technology product & platform development (company run by engineers & not accountants), and the latter half on the other (dark?) side - the corporate business - with in-house technology / IT "service" divisions, where business (accountants mostly non-technical) people run the show - an environment where there is often just a complete lack of alignment, understanding & appreciation for what goes on in both worlds: technology-and-business-worlds alike.

[Bias disclaimer: Yes, I do I have a bias - I believe if a company thinks itself to be a technology-driven business, then I expect the CEO to at least have an engineering/science background, or the CTO/CIO must have written code & built solid (industrial-strength) products in a previous lifetime. If a CEO is not technical then at least develop a keen appreciation for the engineering challenges and then trust your technology owners to do the responsible thing & you move out the way....]

At a recent ExCo of which I'm just an observer - this lack of alignment reared it's ugly head again when "the business" could not understand how a six-month's projects wish-list from their side, translated into a two-and-a-half-year implementation estimate from the technology team, making a poor attempt at explaining, the classic "IT demand & capacity planning" challenges (what I still find quite surprising in today's time, a topic for another day, let's just say I think "IT Demand Management Process" is not relevant today, and I propose that this is just "old-school-IT" & that its days are numbered).  

What followed then from business was the typical:

• "Why don't add another 20 development teams to make the six months timeline?"
• "Why can't we have CFTs (cross-functional-teams) like the other divisions have?" 
• "Are you sure you have the right engineers? Why is the system so brittle? Why must it take so long? Should we take our business elsewhere?"
• "Why can't we have bespoke technical teams attached to each business owner?"

What also makes this quite a difficult conversation is the CULTURE - where technology teams have historically been at the receiving end as the source of all business problems (the cause of business frustration), the underdogs, just taking a beating from business, without any confidence to push back & have constructive conversations. It's a shame really that a large number of technology teams just have to "submit" and "do as you're told" by business...surely it can't go on like this??

So this is what I've reflected so far - and it calls on both Technology & Business leaders to come together to the table, for me it's simply about ALIGNMENT. In order to align, one needs to really LISTEN. I mean REALLY LISTEN and PARTICIPATE in dialogue, after-all, this is what PARTNERSHIP supposed to be about, right??

Dear Technology Leader...

I think a large part of the issue is technology leaders not doing enough to communicate and simplify the reality of their world. Assuming you're not building a greenfields stack, instead you're building on existing systems (not legacy, but still probably reaching end-of-life), more needs to be done to manage expectations. I would avoid too much abstraction & side-stepping: be open and transparent about everything, no hiding...here's some searching questions:

• Have you stopped yourself to reflect on why there is such a disconnect or lack of understanding? What are you doing wrong? Is there a fundamental issue of principle?
• Have you taken your customers through the system architecture and current challenges?
• Have you communicated your assumptions clearly? For example - does the business understand maybe you're trying to build a common platform/product stack that can support multiple business requirements (e.g. in a multi-country scenario) in one-go?
• Is this assumption valid? What if the businesses are quite divergent in terms of customer requirements?
• Why is it important to provide a common technology platform anyway? For who's benefit?
• Would it not be easier to branch the code & platform to serve individual businesses? Risk duplication but gain in business agility? 
• Who cares if the code is forked anyway, has this option ever been tabled?
• How can you explain the nuances of common stack software development to non-technical people?
• Does business understand how you've structured your delivery teams?
• For example, you might be running with component teams, specialisation instead of generalisation and not ready for cross-functional full-stack teams yet?
• Are you clear where your bottlenecks are?
• Is your software engineering principles solid - i.e. can it support multiple release streams simultaneously?
• Have you shown to business the complexities of what it means to spawn up multiple teams (i.e. this rather stupid notion of adding more people to get the job done sooner)?
• Have you established a way-of-working between business & technology that aims to at least address some of the challenges, and set forth an ultimate vision / aspiration of what ideal could look like?
• Are you doing enough to manage expectations upwards and thus protect your teams?
• Why do you have a need to insert "Business Relationship Managers (BRMs)? Are your technology teams not equipped in dealing with business stakeholders directly? Why do technology units decide that an intermediary BRM would help solve relationship problems?
• Are you empowered enough to challenge priorities & requirements if they don't make sense to you?
• Have you communicated that systems generally have to refresh every 3-5 years? Have you considered presenting a technology roadmap showing all the technical initiatives planned whilst simultaneously expected to keep the business lights on (business-as-usual) and still stay ahead of the curve to keep your stack technically relevant?

Dear Business Leader...

Just as there may be some gaps with your technology leader counterparts, you should consider what gaps are on your side. For starters, have you really invested time to LISTEN to your technology partners?

• Have you done enough to convey your business vision, strategy & goals in a clear manner that informs your technology partners of the role they play & your dependence on them for your overall success?
• Have you resisted the urge to bark orders without first trying to listen?
• Do you have an understanding of how your technology teams have built the system & components? Perhaps you had a role to play in this - i.e. ending up in the current state because of business owners?
• Did you perhaps grow up with the industrial mindset (factory methods) & expect that software systems follow similar patterns "it's just a production line outputting widgets"?
• Do you understand what it means to write common software code and deploy common  software & infrastructure to serve the needs of multiple businesses (with its own set of rules)?
• Do you have a clear picture in your mind of the distinctions and challenges of software teams: Component teams versus cross-functional teams?
• How do you as business owners come together to align on overall business requirements (assuming you're in multi-country set-up) that shares a common platform?
• Did you even agree with technology that a common platform is the way to go?
• Are you doing enough to review your own project's backlog and business requirements - by priority?
• Do you accept that your technology team would appreciate from focus - i.e. do one thing of value at a time?
• Are you doing enough to make sense of your own business priorities?
• Have you read "The Mythical Man-Month"? Do you actually believe that "adding more people to a software project will get it done sooner?" - Are you not being naive, have you stopped yourself to reflect that maybe you're just not getting it??
• What are you doing to get closer to understanding your technology provider's challenges?
• Do you use the IT/technology systems on a daily basis?
• Have you considered sitting closer to your technology team?
• What is your role in the partnership? Is it too one-sided (i.e. your side)? Do you get immense joy at squeezing your technology providers, adding pressures and artificial deadlines to make your case?
• Why do you feel you need an intermediary between yourself & technology - and insist on this concept of a BRM? Are you not interested to learn more about your technology stack or develop closer relationships with the people building your technology? 

Attention Business-and-Technology Leaders - Don't look for that Silver Bullet...you'll have to discover this for yourselves, there's nothing off-the-shelf to plug-and-play for you...

You might have attempted every fad at improving process improvement, this "agile" topic has become so misused that both parties can't really tell if agile is working for you or not - and you're quick to blame the process, or attempt to try a new way, and even get professional consultants in to help...but...as I've experienced agile in a variety of forms in the past ten+ years, I still come back to the essence - and I really feel that people have misunderstood the core. This core that I refer to, is the founding principles espoused in the Agile Manifesto. Simple statements but deeply profound, and it all starts there. I feel this essence is often just skimmed over in favour of an execution process which has led to the downfall of many IT/Business relationships, which is why we keep coming back to the rather archaic IT/Business management frameworks that calls for IT Demand/Capacity Management and Business Relationship Managers...

Go back, breakdown the silos, no pedestals or ivory towers allowed in today's age, re-align & commit to participating in respectful, sincere dialogue, meeting each other half-way. Identify the root of the pain-points, agree on sensible ways to manage classic "capacity challenges" which simply implies some focused management of priorities by business people...if after you've tried to re-align, and still there's an expectation of infinite capacity from IT delivery...then...maybe there's just no hope left...

Yours sincerely,

A weathered-and-battered-technology-leader-looking-for-hope!

Whiners never win

Whiners never win!

What simple yet powerful, three words? 

These are words my electrical engineering professor used on me back in my second year, when I challenged my paper being wrongly graded.  I remember waiting a long time at his office just to see him, the prof just gave me one minute "Mr. Khan, here's some advice for the rest of your life - whiners never win". That was it, and then he left, my paper unchanged.

This stuck with me as I completed my degree, started my first job, took a chance one year after graduating by emigrating to two countries, marriage, parenthood, etc. I keep coming back to this simple message "whiners never win" and so just get on with life or work...

Yet, coming back to South Africa, now 8 years and counting, I was surprised to learn how much whining happens in the workplace! It is emotionally draining. Almost everyone has a complaint, not happy with this or that, unable to separate personal emotions from being professional at the workplace. Just a whole lot of whining going on. I wonder if it's just the context of the nation, just whining, starting with the commute into work listening to radio stations (like 702) which I've stopped completely. I actually don't listen to such news anymore, it starts the day of on the wrong note, stepping into the office with negativity and stress. Don't forget the stress of driving on SA roads...

I was also surprised how people tend to bring their home issues into workplace as well, affecting how they show up, perform, etc. My mindfulness, listening and empathy skills have seen an accelerated growth since 2011...

Still, eight years on, I still see it a lot of whining in the workplace...come on folks, stop whining. 

Whining never wins. 

Bring your best self to work, show up and get through the struggle...

Stop whining. "Whining never wins"...

Don't get me wrong, I've got loads of empathy...but at some point, people need to step up and just stop whining as whiners never win!

On choices, decisions, initiative, career planning: lessons, looking back

Image Source

At a recent town hall with my technology division (100+ people: frontend & backend engineers, platforms & infrastructure systems engineers, system architects, agile specialists, AI/ML engineers, Agile PMO & technical ops monitors), I opened up by giving my perspective on expected behaviours & responsibilities of the individual, especially when it comes to career development & growth aspirations.

The message was about taking a personal ownership for one's own growth, rather than leaving it up to the company or one's manager (a message that surfaced a few times in recent OfficeVibe feedback) - that the responsibility largely lies with the individual. Yes, managers/leaders are there to support you & guide you along the way, but only you know what defines you as a person, so don't leave it up to others to determine a path for you...

Reflecting on my own journey, it all comes down to understanding your current reality, weighing the choices on the table, defining your aspirations, taking initiative,  processing & reflecting through assessing the outcome of the initiatives, finding great people to learn from, tracking your trajectory on the path to growth. The path is not always clear, sometimes adjustments need to be made, sometimes a little backtracking is needed to enable the next leap forward - still, it all comes down to one's own personal ownership & level of commitment to controlling one's own future.

I thought I'd share my own timeline as guidance for people who might be stuck. Interestingly enough, although I only recently started formally implementing my own management framework around life/work planning by way of my RAGE model, that I was actually instinctively using this decision-making model all the time.

My timeline table shows the major periods in my career, commenting on reality of the situation at the time, choices I faced, decision made & eventual outcome. I think anyone who's considering what to do next with their career plan should do a similar exercise for their own sense-making.

My Perspectives

1. Gain a keen appreciation for your current situational reality and take responsibility for it. Yes, reality sucks sometimes, but you got to play the hand you're dealt, don't let that get you down.  It is possible to change your reality. I made hard choices based on my reality of being caught in a low-income family, living through Apartheid.
2. Discover your key motivations and use them as your guiding compass, some call it your "value system". I believed in myself and my ability to make things happen. If I felt my knowledge was lacking, I would learn & close the gaps myself. Don't assume you know everything, there are tons of smart people out there. I got a huge awakening when I went overseas, so much so that I had to learn software engineering & computer science all over again.
3. Don't go seeking hand-outs or help, but if people or companies do extend their generosity, don't naively turn them down. There will be good people helping you along the way. After trying many avenues of financial aid/scholarships without success, I thought help would never come my way, but it eventually did.
4. Don't bog yourself down with "If Only", or "What If" - this creates negativity & unnecessary anxieties. Move on, look forward. Sure, reflect on the past, learn from it, but never let it hold you back. You're in control of shaping your own reality. I chose a path that was the most practical, I switched jobs just as I was going to be promoted, I left big projects just as they were about to land, I left a start-up thinking I had a job lined up (but it didn't happen), I left what others would say is madness (left the stability of UK to return to volatile SA). Leaving UK was very difficult for me from a professional experience sacrifice, but I never allowed doubt and negativity to bog me down.
5. If you want a good measure of your skills or experience an alternate reality, leave your country & work overseas. Even though the world has gone smaller through globalisation, that even in South Africa we do work with international teams, I still think getting overseas exposure is one of the best things one can do. Living and working in different countries exposes you to a different world of experiences. If you're under the age of 35, then you should try it. It doesn't have to mean relocating or emigrating, it could be a temporary secondment for a year or two. I was fortunate to experience working with many cultures across the globe on some really big projects. I learnt so much in a short space of time, it took my engineering skills to another level. As for Planning, Management & Execution principles, in my opinion, the UK ethic is world-class.
6. Become comfortable with uncertainty & embrace the unknown, even it means leaving your home town/country for another one. You get this only through experience, and having been through at least one transition into the unknown. I've seen a few - it's not so bad, you must embrace your fear of uncertainty.
7. Develop a learning & growth mindset - in any new role, work hard to learn as much as you can, by reading, studying, latching on to people as mentors, read other people's code through open source projects, etc. I became expert in a few areas: MPEG/DVB protocol spec & implementation, C & C++ coding, Voice synthesis & Text-to-Speech, Project, Agile Program Management & Execution, Professional services consulting and more recently Leadership skills. This doesn't come easy: I read a ton, implement the tools, learn from experienced, the wise, still remaining open to new experiences, no matter how edgy they might make me feel.
8. Be ready to start-over again more than once - switch roles, domains or industries, sometimes what might seem to be a step or two backwards, actually turns out to be better than hoped. I started over at least 5 times in my career of 20 years. If you're in software, sure you can specialise (and there's nothing wrong with that) but you must then become expert at what you do. To grow in software, my view is to learn as many tools as possible, switch every two years. One of the best ways to do this is side projects, open source communities. Don't wait for your company to reserve hackathon sprints, or 20% time - take ownership. I taught myself text-to-speech synthesis on my own, developed POCs in my spare time and proved to company the potential innovation. If I had not taken initiative, I probably would not have landed the ultimate technical role I dreamt of.
9. Don't pass the responsibility for your career on to someone else (your manager or company), rather you should have a view of your own map, your end goal. Your company or leaders can help with options available, guide on the gaps you need to fill - and it's even better when the company has a decent career ladder in place. Never pass the buck on and make excuses that your company / manager does not care, that you don't have enough syncs 1:1s or feedback sessions with your manager. You need to take ownership, period. In the companies I worked for, we at least had a decent career-ladder in place, showing all the upwards, sideways opportunities available. I made it clear to my leaders at the time, that my ultimate goal was to become a "Jack of all trades, but Master of SOME" T/PI-Shaped skills. They knew that when I'd enter a new role, I would learn, produce outcomes and then move on, thankfully, I had very good leaders that did not stand in my way. If you feel obstructed by your leader / team / company, first dig deep within yourself to reflect on whether your own behaviours need improving, and if you're still convinced it's not you, then leave, change your environment, change your circumstances.
10. Don't get complacent or too confident your role is secure, retrenchments & redundancies are a reality, business-is-business. I got my first taste of layoffs when I was still a junior software engineer, naively thinking I was in a good place by virtue of being part of a cool new product team, and owning some key components. Since that first-and-only layoff (in 20 years), I developed my "spider senses" - and decided that it would always be me that decides whether I stay or leave, not a market event or the company.
11. Do not grow an entitled mindset, or have unrealistic expectations from your employer. Say you studied hard and earned additional paper qualifications: MSc, MBA, PMP, etc. Don't expect the company to automatically increase your salary or grant you a promotion. Say your own personal life changes, you get married or have children, so you have more responsibility at home. Why should you expect your company to give you an increase, if the work you're doing hasn't materially changed, or your output is still the same, and you're still working at the level the role expects?? At the end of the day, it's up to you to manage your personal circumstances, it's not the company responsibility now to just automatically reward you or make your life easier financially - NO - you have to work at it. If you gained new qualifications, you need to show an interest in contributing your newly acquired knowledge, showing value. If you're seeking a promotion, you must show you've actively contributed covering much of the roles/output of the next role you're seeking. Companies don't owe you anything - so don't come up with unreasonable expectations or feel entitled. It's all about your output, meritocracy  is the only thing that matters.
12. Becoming your own boss, running your own consultancy is hard work, be prepared to fail in this area. Although branching out on your own can be enormously liberating & exciting, unless you have a large network to tap into, moving from one consulting engagement to another, building up clientele & a pipeline of work, growing your team - whilst a lot of people have made this a successful venture - it is actually quite hard to do. I was fortunate to secure the company I left as my major client, although, the client only wanted to work with me, so in effect, I never really left! Without a strong network, the going was tough trying to break into other clients, even after doing some pro bono consulting work. You must invest a lot of time & energy, unpaid hours to build your own consultancy, something I didn't do, which showed I wasn't really fully invested in this venture. So I shut my company down, and was pulled back in by the strong gravitational forces of the big company. I learnt a great deal, became a better salesman, and became confident in interacting with C-Level executives. Consulting however, is a sure way to make extra money than being a permanent employee, but it comes with its own set of risks.
13. Show gratitude. Whilst you might think that you're in control and the result of your successes are due to your own hard work, sweat and tears, never become arrogant and ignore that other forces helped you get this far. Take time to seriously reflect on this, and you will soon identify people or events that helped, and when these surface - be thankful & show your gratitude, develop humility. Although I was brought up in low-income household, I never once felt not having a complete home, or solid upbringing on life skills - if anything - this actually shaped my personal motivational value system. I never regretted or blamed my parents, I had a good childhood, was taught responsibility & key life skills. I've acknowledged people, friends and family that shaped my reality, the leaders in the various companies I worked with, were supportive, friendly and encouraging - I learnt so much from them, and still continue to learn from them today. Sometimes, when you're in the thick of the day-to-day job, you might not like what your manager says or does (over controlling, micromanaging, lecturing), and it's only when you leave, you realise the wisdom and lessons being taught. It's not always about you, and if you're leading teams, show appreciation for your teams as well. Your success is a result of your team's output. As you develop into senior roles, your visible output might become less-and-less, but you're still working hard through people, in the background. Never think you're the sole reason behind success - there's so much more that goes on, that we're often blindsided - don't get blindsided, actively seek out your blind spots.
14. Be patient. Patience is linked to gratitude. Be patient with your role, allow enough time to learn the essence of the domain. Once you're comfortable & confident in your appreciation of the essence or core principles and you've remained long enough in the role to complete 1-2 major pieces of work / projects, then allow yourself the opportunity to move on. But don't rush things through, learning needs time to soak. Personally I'm always doubtful when I come across people's CVs hopping from one permanent role to the next in less than 18 months (this is because the major initiatives usually run for at least 18 months). From my experience, this (9-15 months) is just not enough time to do justice or have made a serious contribution in terms of outcomes (unless the gig was to rescue, recover or revive a project as a major intervention, or consulting gig). For me, it's been roughly a minimum of 18 months provided I felt confident in my results. I just about completed my engineer-in-training role after university when I took a big chance, although successful, in retrospect, I had big gaps to close anyway. The more higher one climbs the ladder, the more patient one needs to be, which means 18 months could grow to 24-36 months minimum. Currently I'm resisting the urge to switch, knowing that I still have another year to go before I can claim to have truly owned the role, so patience becomes a necessity.
15. It's not always about the money or job title - neither does "years in role" contribute to "seniority". Although I might risk passing a value judgement on other people here, what I found is that money should not be a driving motivation, if you've set your sights on a learning and growth mindset. Sure, you can hop from one job/company to the next every 12 months or so, on each move your salary jumps - but if you're effectively not learning new skills or growing, is it worth it? Job titles are also relative, what's in a name after all? What matters is what you can do, and the value you bring to the table. What I found also trips people up is this complex of "seniority" based on "I've worked so many years in this role, hence I deserve a promotion as a senior xyz". In my journey, I sacrificed salary growth for knowledge, experience and a wide/deep toolbox of skills/capabilities. Later when things became challenging financially, other opportunities opened up that boosted my income, which wouldn't have been possible if I'd not honed by capabilities & demonstrated value as a result of experience. I once had a manager who, in his previous company was a VP of Engineering only to become a development manager in his next role - two steps back. I'd asked him why he made the move, this was when I learnt that work/life isn't just about the title. He restarted two levels down and in a short space of time, was back to being a Director of Engineering. I had a team leader once who was performing in the role of manager, but was so humble and patient that having the title did not bother him much. I have seen engineers who effectively remain doing similar activities for years, expecting a promotion by virtue of being doing the same job, even though the competencies haven't grown (e.g. influencing group/country/global teams, taking ownership, showing initiative) - so if you want a promotion, you need to earn it! I've also seen people who are content being a software engineer (with no aspirations of seniority/leadership), but who are expert at what they do, adding so much value, that the company provides enough incentives to keep the person happy (at times a brilliant software engineer could earn a higher salary, have indirect influence greater than his/her manager). At the end it does come down to personal motivations, and when the time does come around to being a financial/happiness constraint, then don't expect the company to help you - it's time for you to change!

My Timeline (Click here to see full table)

Experiments with XL: Extreme Leadership (Paired Management)

I recently (in early June) took some time off from life and work, decided to be completely unplugged for ten full days, with no access to any electronics whatsoever. This break from work coincided with the last 10 days of Ramadan, a practice called Itikaf, which is about secluding (surrendering) one's self to God by living day/night within the precincts of the Masjid boundaries, where one is engaged in various acts of worship, including self reflection, self-awareness & meditation. Itikaf is about leaving it all behind, including your own family.  I had actually made my decision sometime in April, a feeling that suddenly inspired me as something I just needed to do, without too much thought or planning. I immediately fired off an email at 5AM informing by boss (the CEO) of my upcoming intent, not really seeking his approval - I merely informed him that I'll be away from the office, completely unreachable for 10 full days, a personal life quest I needed to fulfil...thankfully, the boss didn't get in my way ;-)

When people, both friends and colleagues included, found out about my leave plans, they immediately enquired how I was going to manage the work topic. I lead a technology team responsible for an online video platform for DStv (providing live/linear streaming channels as well as video-on-demand Catch Up services, we also support a major part of the business with a full digital/web platform infrastructure). Around that time, my teams were working flat out to stabilise & scale the platform for increased load, in preparation for the FIFA world cup. We will only be ready to "freeze" a week before world cup - how could I possibly leave the team at such a critical period??  On top of that, my division was part of an ongoing group-wide organisational restructure, in those two weeks, there was much HR work to do, including rationalising job roles across various technology divisions, job grading, role & people profiling, as well as one-on-one consultations. How could I leave the team leaderless through this period?? And to add a cherry on the top, whilst I would be on leave, the group's yearly Ops Review would've taken place, I would need to input into this stream as well.  As if this wasn't enough, there were concerns that my leadership team was fairly new, people had some doubts...

I wasn't really concerned by these, instead I saw it as the perfect opportunity to test my leadership team. Whilst a fairly new team that's been only recently formally announced, they are still in between the storming/norming phase (even though it's been over a year since they're under my leadership transformation). I wanted to test out a few things:

Update: Reflections on my career in Software Engineering & Management

Two years ago around this time, I explored my progress with respect to my career choices by performing some self-reflection, by asking myself some searching questions. Later I shared my findings on this blog, in this post: Reflections on my career in the hope my story could resonate with people who may be experiencing similar challenges. I'm glad I did so since people did actually reach out, thanking me for the post & providing feedback.

Anyway, two years have since passed since I last shared the cross-road I found myself at, since I'd started my journey with this path in mind...

a) Software Team Lead -> Software Manager -> Senior Manager -> VP -> Director -> CEO

b) Principal Engineer -> Senior Principal -> Technical Director -> CTO -> CEO

b') Principal Engineer -> Architect -> Senior Architect -> Director -> CTO -> CEO

c) Technical Project Manager -> Senior Project Manager -> Program Manager -> Director -> CEO

...but instead found myself being in the technical project management space for too long, that people started naturally profiling me as the "rockstar program manager" (check out my LinkedIn recommendations and you'll immediately see why). Whilst this is a great place to be (don't get me wrong, I think a career in Project Management is one of the most versatile, lucrative and flexible professions out there, I highly encourage the move), for me, I felt I'd learnt and experienced enough, that I didn't see myself doing that for much longer. I did enjoy the project leadership, but I wanted more. Another factor that was causing me anxiety was that my role as a management consultant was getting a bit boring, what I imagined it to be versus the reality were not fully aligned. My time was fully consumed, leaving me little time to explore my own ideas to look at new ideas/products (my own start-up), I might as well have been a permanent employee - I was living an illusion, in reality I was basically a "perm-tractor". I had built up enough personal equity, credibility to enjoy a decent level of referent power to indirectly influence outcomes in my favour, I still wanted more - I wanted to feel more alive than being a neutral facilitator (which in itself I found quite rewarding but also quite energy-draining).

So what did I do? I went back to my RAGE model, here's what I had for my persona as a professional:

• As a software professional, I would like to learn & grow, seek out individuals, companies and interactions, to reach heights of excellence, so that I can not only enjoy the profession, but take me to new opportunities & experiences. I want to surround myself with people that motivate me, journey together to grow to the next level.
• Want to work with inspiring, motivated leaders that I can learn from. Want to surround myself with deeply technical, bright people. Want to work with people who know what they're doing or unafraid to take chances. Want to work with disruptors, people unafraid to push boundaries, challenging status quo. Want to work with people who are equally, if not, more motivated than me. Want to learn from people so that I can grow and do my own thing one day. Want to be with fellow professionals that will help take me to the next level. Want to work on projects and products that are interesting and cutting edge, not "me-too, copy-cat products. Want to stay at the cutting edge of software, be involved in the next wave like cloud services, mobile app development, car infotainment / self-driving cars, drone software, cloud, etc. Want a chance to start-up my own business in ideas in product development, services-space like crowd-based testing, etc.
• As a professional, I want to run a company, lead my own division. I believe the experiences and skills acquired over the years puts me a good position to do this, regardless of technology stack. I haven't been successful in launching my own start-up, so the best place would be to go back to corporate, be part of story much bigger than myself, and get the experience I need.

I also came to the conclusion that being a specialist is not a bad thing, so I'm now settled with the fact that I'm a Digital TV Technology Specialist, so I should just focus my energy in this area. I can still keep abreast of new technologies, but the road to my continued success is to build upon this experience - the rest is noise - if an opportunity comes my way for investing or if there is something truly exciting with a lot of upside, then I still might consider it ;-)

So what's happened in the last two years?
I made a decision to leave project leadership behind. I explored opportunities that aligned with my aspiration of running my own division. I took a chance by breaking the perception that I'm the guy to call in to rescue failing projects - landing an engagement as interim GM/CTO. A year later, I decided to leave consulting (248 weeks consulting) altogether and enter the corporate world as a permanent employee, taking on a CTO/Head of Technology role :-)

So my path has indeed played out a little different but now seems to be back on track:
Software Engineer > Senior Engineer > Technical Project Manager > Senior Project/Program Manager > Principal Engineer > Program Manager > Management Consultant > CTO (now) > CEO (next)

Lessons learnt / myths busted?

Who says you can't change tracks in between (especially switch to project management) and switch back to technology leadership? It can definitely be done!

Be prepared to Leave it All Behind as long as you believe you're heading in the general direction you seek (maintain your guiding compass always).

Take time to process your situation with Life/Work by investing the time in self-reflection & planning. I found my RAGE model to be a constant source of guidance. It does take some self-control, but it will be worth it in the end, just keep at it...

It is indeed possible to start from humble beginnings and change your life for a better outcome...

On Managing Change: Damped Sine Wave

I recently read a piece from the June edition of ACM, Q&A with Erik Meijer, which I found quite apt as it speaks to my current situation at work. Whilst Meijer talks specifically to software projects, we can apply this to any kind of topic, be it personal or professional, work & life - everything we encounter when it comes to dealing with change & uncertainty (career, new teams, family, new projects, etc.).

This is especially relevant to the period I'm in now, a leader driving change - i.e. changes to the way we working, kicking-off a focused management war room, and the recent announcements around delivery priorities for the next three months. This is a large corporate, multi-million dollar industry, the amazing part is that things are so fast-paced and always changing that one can argue that the only constant in this company is one that of Change!

So we need to deal with this reality, it's not going to go away anytime soon. My experience is pretty much aligned to Meijer: try to manage the level of uncertainty (balanced by one's appetite for risk) as quickly as possible through communication & stakeholder engagement, converge on well-bounded known-knowns based on the information we receive, agree to execute & deliver within the binding time constraints. Add to this is a sense of measured calm and patience, start practising ways to control your default responses too...

Courtesy: ACM

Below is the excerpt of the Q&A:

<quote>

What is your team process? How does work get done? How do you communicate status? 

A lot of what you read about process and agile has very little evidence behind it. I don't believe a lot of process is scientific. Instead, I define general guidelines about what I want to see happen, and within those I don't care how things happen.

My thinking has two main sources of inspiration: the military and the hacker way.

Over thousands of years, armies have figured out how to get things done and achieve their goals in an environment that is really chaotic and unpredictable. That is the environment we live in as developers as well. If you read the U.S. Marine Corps Warfighting manual, and replace the word war with software, everything in there holds true.

So how do you deal with uncertainty? When people attempt to solve with process, they are trying to fight or control uncertainty. For example, someone can say just adopt zero inbox and your life will be awesome. In reality though, that isn't really the case.

One of the things I like about Facebook is "the hacker way". It is an approach to creating software that involves continuous improvement and feedback. It is about computational thinking: how do you program the system, and how do you make the system do things that no one thought possible?

Being agile is about communication.

The process needs to change with the situation. You have a big picture of where you want to go, but any plan or process will shatter immediately when you hit your first bug or something happens out of your control.

In most projects there are two phases: an exploratory phase and an execution phase. Your project should progress like a damped sine wave, where the amplitude gets smaller over time (see picture above). You have to figure out what to build, and figure out what question you are trying to answer. In the beginning you want to increase the vertical velocity to get the uncertainty under control, and then you want horizontal velocity to increase when you get into execution.

With prescriptive processes. people are looking for a silver bullet to solve problems, but it doesn't exist...the world is super-confusing, and you have to embrace it and work with it.

</quote>

On driving change: Kotter's Model

I've not only been studying organisational change management for a while now but also been part of some of the process in my recent engagements with clients. The subject has started to fascinate me, which has created a deeper sense of appreciation for system dynamics in the workplace.

I've decided to capture some frameworks on my blog for note keeping, since it will become a likely go-to place for me to reference. I'll start with Kotter's model and build from there. 

The source of my information in this post, is cited & referenced from Leach's Critical Chain Project Management book, Chapter 11, Pages 283-286.

1. Create urgency

The first step is to get a sense of urgency building within the organisation that something must be done. Most people in most organisations feel overwhelmed by keeping up with the daily workload so suggesting doing something more to change the way the organisation works looks at best to be just more work and at worst to be something that is going to make things worse than they presently are. People need something to motivate them.  Kotter suggested some things that work:

• Show others the need for change with a compelling object that they can actually see, touch, and feel.
• Show people valid and dramatic evidence from outside the organisation that demonstrates that change is required.
• Look constantly for cheap and easy ways to reduce complacency.
• Do not underestimate how much complacency, fear, and anger exists in your organisation.

2. Build team

One person can only succeed to cause change in a very small organisation. Many people are not even able to cause changed behaviour in one person: themselves. Think of how many people succeed at losing wait or stopping smoking? Planning real change at an organisation level needs help, you can't do it alone. You need to enlist the leaders of the organisation who have bought into the sense of urgency. Kotter suggested the following that could work:

• Show enthusiasm and commitment to help draw the right people into the group.
• Model the trust and teamwork needed in the group.
• Structure meetings for the guiding team to minimise the frustration and increase trust.
• Put your energy into step 1 (raising energy / urgency) if you feel you cannot move on to step 2.

3. See vision

People need to be able to see the proposed change because that is what can begin to create an emotional feeling that will motivate them to change. A vision should be a picture of the end result. If you describe it in words, the words need to evoke an image. Kotter suggested:

• Try to see - literally - possible futures.
• Make the vision so clear that you can articulate in one minute or write, or better yet draw, it on one page.
• Supply a moving (emotional) vision such as serving people.
• Put forth bold strategies to make the vision real.
• Focus on how to quickly make the change.

4. Communicate

So people can feel the change, you need to communicate:

• The vision in terms of the benefits people will see when they change their behaviour.
• What has to be done to make the vision a reality.
• Reinforcements when people exhibit the right new behaviours.
• "Wins" by people and groups who do the new behaviours.
• Anything and everything else about the change that will keep at the top of people's agenda.

Kotter suggests some ideas on communicating:

• Keep communication simple and heartfelt.
• Do your homework before communicating, especially to understand what people are feeling.
• Speak to anxieties, confusion, anger, and distrust.
• Rid the communication channels of junk so that important messages get through above the noise.
• Use current technologies to help people see the vision.

5. Empower action

You need to empower action: make sure people know that they are expected to take action now and that they are free to do it as the see fit. Empowering action is as much about removing obstacles to action (pulling) as it is about causing people to act. Kotter's suggestions:

• Find individuals with change experience to bolster people's self-confidence with "we-won-you-can-too" stories.
• Recognise and reward in ways that inspire, promote optimism, and build self-confidence.
• Deal with disempowering managers through coaching or move them out of the way.

6. Create wins

Your team needs to coach people to create successes: wins. Then you need to reinforce the behaviour of those who created the wins and communicate their wins and reinforcements to the rest of the organisation. Pilots are a powerful tool to create short term wins but you need to ensure that people who live those wins with the pilots do not immediately go back to prior behaviours. Kotter's suggestions:

• Early wins that come fast.
• Wins that are as visible as possible to as many people as possible.
• Wins that go through emotional defences.
• Wins that are meaningful.
• Early wins that speak to powerful players whom you need to engage.
• Wins that are cheap and easy even if small.

7. Do not let up

The leadership team has to keep the desired change at the top of agenda through and well beyond the planned-for successes. There will be obstacles and there will be some failures along the way but the winning teams take failure as a learning and motivating experience to add vigour to the change process. Kotter's ideas:

• Rid yourself of work that wears you down - tasks that mattered in past but may not matter now or tasks that you can delegate.
• Constantly look for ways to keep up the urgency.
• Use new situations opportunistically to launch the next waves of change.
• Show 'em, Show 'em, Show 'em...

8. Make it stick

Once you have completed the first round of getting the organisation to exhibit the desired new behaviours, you need to continue right on to improve what you have accomplished. If you do not continue to improve, the organisation will revert to the previous behaviours in a surprising short period of time. Kotter's ideas that work:

• Never, never, never give up on step 7.
• Use new employee orientation to demonstrate what matters most in the organisation.
• Use the promotion process to place people who exhibit the new behaviours into influential positions.
• Tell vivid stories over and over about how things now work.
• Ensure continuity of behaviour and results that help sustain and grow the new culture.

The Golden Circle

In this post I share a snippet of information referenced from the book Start with Why by Simon Sinek. The ideas and concepts provided in the book talk mostly about how some companies are more successful than others (using Apple as a major theme). It advocates that most companies often don't take time to appreciate the WHY they in business in the first place, and more often focus on WHAT they do. By focusing on the WHAT they tend to lose a captive customer audience, failing to address to the core beliefs of customers, which is one of the main reasons customers choose to go a brand, or become an ardent follower of the company.

I found this pretty interesting, with a wider relevance outside of just corporate strategic management. The golden circle can be applied to both personal and professional topics, and links nicely to my RAGE model. It can also be applied to team structures as well. In fact, in the organisational sense, the golden circle flows from top-to-bottom: starting with the company vision and flowing down to each person in the company.

What follows is the core explanation from the book, chapter 3, The Golden Circle. Simon Sinek's TED talk is also one of the highest ranking talks on TED, video is embedded at the end of this post. Although Sinek makes reference to many stories and cases that appear as old news in today's time, they are still nevertheless relevant, at the very least a useful reminder...

The Golden Circle

According to Sinek, the golden circle can be used as a guide to vastly improving leadership, corporate culture, hiring, product development, sales and marketing. It even explains loyalty and how to create enough momentum to turn an idea into a social movement. And it all starts from the inside out. It all starts with Why.

Starting from the outside in, Sinek describes the terns:

WHAT: Every single company and organisation on the planet knows WHAT they do. This is true no matter or big or small, not matter what the industry. Everyone is able to describe the products or services a company sells or the job function they have within that system. WHATs are easy to identify.

HOW: Some companies and people know HOW they do WHAT they do. Whether you call them a "differentiating value proposition", "proprietary process" or "unique selling proposition," HOWs are often given to explain how something is different or better. Not as obvious as WHATs, many think these are the differentiating or motivating factors in a decision. It would be false to assume that's all that is required. There is one missing detail:

WHY: Very few people or companies can clearly articulate WHY they do WHAT they do. When I say WHY, I don't mean to make money - that's a result. By WHY I mean what is your purpose, cause or belief? WHY does your company exist? WHY do you get out of bed every morning? And WHY should anyone care?

When most organisations or people think, act or communicate they do so from the outside in, from WHAT to WHY. And for good reason - they go from the clearest thing to the fuzziest thing. We say WHAT we do, we sometimes say HOW we do it, but we rarely say WHY we do WHAT we do.

But not the inspired companies. No the inspired leaders. Every single one of them, regardless of their size or their industry, thinks, acts and communicates from the inside out. They start with WHY...

Get the book, watch the video!

On E2E architect role in projects

I have shared my thoughts in the past on the subject of architects in the domain of digital TV software systems which can be found here. In this post I will share further thoughts about the specific role of an E2E (End-to-End) architect from a program/project management perspective, including my expectations of the role as well as the associated challenges I've faced with this concept, especially where the role either does not exist, or is not clearly defined in cases where companies don't have a clear enough understanding or have not yet set up a formal architecture competency.

My Background & My possible Anchor Bias

In my first decade of working experience with software and systems engineering projects, I was fortunate to work with world class technology service providers (S3 in Ireland & NDS/Cisco in UK). This experience helped shaped the way I would view future technology projects, including product & project management, software engineering, integration and delivery functions. It's the kind of experience that never leaves you, especially when you've witnessed first hand, one successful delivery after another both as an engineer and manager. So I'd come to see the world in a particular light, taking with me the style of project execution where ever I went, and assumed that all companies naturally followed similar concepts. And the cool thing about this experience was a well grounded appreciation for software & systems engineering design principles - principles which should not be confused with implementation methods, as what is a hot topic of debate these days, that of Agile V Waterfall, that has been the bulk of my recent challenges in projects in the last six years.

On returning back home to South Africa* still working in Digital Media TV domain, I soon realised that things are a little different to say the least (absence of architect roles, working with business analysts where I'd been used to analysis being a function of architecture, no defined role of systems integration, no method of continuous delivery & lack of solid post-launch monitoring & operations support systems). My first major program, involved a large-scale overhaul of the end-to-end broadcast system, including a video-on-demand component and a new set top box software stack. In order for the program to have a decent chance of success, I not only had to restructure & reshape the program, but also had to introduce functional roles that for me, were always obvious: an architecture team with a specific role of E2E architect, E2E systems integration, test and delivery teams. With a lot of hard work of convincing senior management these roles were necessary for project delivery, we managed to implement the structures I proposed that ultimately helped in the successful delivery of the project.

Strangely enough, as I took on other engagements in large programs in the same group of companies, I found myself repeating the same. It could be because I have a bias that is so strong and deep that does not allow me to see any other way, even though I always try to adapt my style to the culture and teams currently at my disposal, yet I still find myself coming back to these principles because its been proven and tested, that I can't really find any real fault in - possibly it is because these are indeed engineering principles that have stood the test of time, especially this concept of an End-to-End Solution Architect role. As the saying goes, never leave home without it, if I'm running a large program of work, that involves many systems & component suppliers, I never run a program without having an E2E architect attached to my programs, as step one. Once that is established (often as a result of canvassing support which takes up some time and energy), I move on to structuring the role of E2E systems integration, which in turn drives the processes and behaviours expected from component engineering and test teams. The program timeline then shows a simple picture of the high-level milestones involved, where the bulk of the program management and execution is a function of coordinating delivery plans with respective project delivery teams (who maintain the detailed project plans) as well as managing stakeholder engagement (which is a vital component of running large programs). This hasn't been smooth sailing all the time, as I'll try to share some of the challenges that frequently come up, later on in this post.

*Disclaimer: Although my writing is about an experience with primarily one industry domain in South Africa (Digital PayTV), I'm mindful about not passing a broad value judgement against all large corporates with a software competency. At first I thought these challenges of projects and roles like BAs for example was limited to just this environment, but the more I researched by skimming through job specs, attending networking events, technical conferences, meetups, training courses and meeting people from other corporates, I later realised the patterns run across all the major corporates like financial services, banking and the big telcos, that more often than not, the structures in place mirror more traditional-IT governance than hardcore technical software development structures that I was used to, in working in a pure technology services environment. So I'm reasonably confident there is no broad value judgement being applied here, that these experiences are likely to resonate with large companies operating outside PayTV.

Typical Landscape

The picture below aims to illustrate a typical landscape of this ecosystem. This could be separate entities run independently, or even group under a technology division, with different functional expertise, that in reality are still perceived as separate silos of responsibility:

This picture speaks to a typical reality of technology service providers in a digital TV value chain. Each silo represents a set of core technology expertise that exist in separate functional lines either under a technology division, or could exist as separate lines of business altogether. 

Each silo may have its own set of speciality with architects and development teams. Each unit may also likely to have a very unique culture: people, skill-set and ways of working. Each silo may also have its own operating model: how work gets prioritised and injected, build, test and release procedures. These units might also have their own terminology, a lexicon / vocabulary for their specific domains. They may also not be entirely self-sufficient, relying on external third-party component suppliers that form part of their system. These third-party suppliers are just like another silo, each with their own operating model, from architects to development teams, to ways of working as stipulated by their underlying contractual agreements.

These systems & teams usually come together in delivering a product or service that directly impacts a customer (end user). Typically launching a major new product (or platform, say for example a new internet product like a streaming app for video on demand) will impact at least all these systems. To do so, a project is set up as a program delivery initiative that will call for the co-ordination of all systems coming together in delivering the new product or feature to market. So a program team would usually be setup to manage this execution and delivery. Typically a program manager would be appointed to manage the full end-to-end delivery, which is not only the technology arm, but also the wider business streams for delivering product to market (marketing, customer care, sales, training, retention, communications, legal & regulatory). Sometimes, the overall program manager is supported by an end-to-end technical program manager, but this is not always the case. More often than not, the overall program manager assumes co-ordination of the technology ecosystem as well. For now though, I use program manager to either mean end-to-end technical program manager or overall program manager interchangeably.

The need for an End-to-End Solutions Architect

The picture below illustrates the positioning for a dedicated role of E2E Solution Architect to a project or program of work:

Just as you would appoint an end-to-end project / program manager to the role of maintaining the coherency of the overall delivery project, so too should you assign an end-to-end solution architect role, to maintain the integrity and coherency of the overall technical solution design. This is the crux of my proposition. Related to architecture is a close relative called end-to-end systems integrator role, I've touched on systems integration topic on a previous post called Worlds of System Integration, so will not discuss integration in this post. Related to end-to-end integration is testing, in my past experience, integration implied testing or QA, but as I later learnt on coming to back to South Africa, most companies still see QA as a separate activity, so I have another post that talks to the Worlds of QA testing.

Why?

Because it makes running a project so much simpler! 

When I work with an E2E architect, I use it as a vital supporting pillar of the project. The E2E architect will take care to understand the business or product requirements from the business owner, works with all the respective technology domain experts (usually the assigned system architects) on mapping out a solution architecture, defining the new technology dictionary and model (if needed), highlights the system building blocks and supporting components that would be impacted, agrees on high level technical requirements from such systems, as well as important integration points, technology risk assessment, along with unearthing required non-functional requirements (performance, stability, redundancy, etc.) that would be needed for a complete solution. All of this is kept high-level, with just enough information to allow the project team to shape and scope the project, as well as prepare the technical teams of design constraints to think about. All of this is done in the early stages of shaping and scoping a project, has proven in my experience to be absolutely vital in a successful project outcome.

To be assigned an E2E Solution Architect to a project is no small feat. The expectations are quite high indeed. The role not only requires a sound grasp of technical concepts, but also mandates a personality that is comfortable with extreme forms of collaboration, managing and dealing with diverse stakeholders, personalities and egos (don't forget egos - tech guys can be a difficult bunch of people with huge egos), a strong suite of leadership skills, self-discipline, self motivation and the ability to work with ambiguity, a responsible attitude to self-management of tasks (without depending on a project manager to co-ordinate every meeting for example). Added to that, the E2E Architect must have excellent communication skills, demonstrating both in written and verbal form, the ability not only to handle technical communications, but also be the person to translate to business stakeholders in a simple, non-technical manner.

I rely on this architect to also grasp technical and architectural issues that might come up in delivery, and be able to provide solutions and advise on ideas and proposals to manage any impediments on the overall solution provisioning.

Why can't you just feed the business requirements to each division?

The short answer is the risk of lack of coherency of an overarching, end-to-end solution design. The absence of an E2E architect acting as the gatekeeper for maintaining overall integrity and coherency of the business requirements and related solution design, runs the risk of each division designing and implementing fairly independent solutions, leading to fragmentation, increased integration and last-minute rework to get a coherent solution delivered. This also leads to a system implemented without foresight, lacking the vision or with the end in mind. An E2E architect would fill this void, save a lot of time and energy in putting all the pieces of the puzzle together. 

Though not impossible to complete a project without the E2E architect, it is often through a lot of last minute co-ordination efforts of a project management team, or some efforts of a few technical heroes coming together, taking collective ownership of the problem. Generally these units have other work and projects going on concurrently, where focus on a single project is difficult to achieve. If the environment of very open collaboration and team work, where there aren't really any silo-mentality, then yeah, it's possible. But the reality is that in most organisations, silos exist, collaboration is not the norm, and the overhead of a project management team coordinating these activities is too high, let alone the implied technical knowledge of these project managers to make this a worthwhile activity.

This reminds me of the story about Christopher Wren, the architect responsible for rebuilding St. Paul's Cathedral after the Great Fire of London. Wren was walking the length of the partially rebuilt cathedral when he asked three bricklayers what they were doing. The first bricklayer responded, "I'm working." The second said, "I'm building a wall.". The third paused, looked up, and then said, "I'm building a cathedral to the Almighty."**

How many teams fully appreciate the work they do in the context of the end-to-end ecosystem? It is not expected every team have this kind of vision, it would be great if they could all see the big picture and be visionary, but in reality they're not - and there's nothing wrong with that. The instructive part of Wren's story is that he didn't come up with a sense of purpose himself and pound the vision into everyone's head. Each bricklayer cared about something different, even though all three were working on the same thing. Wren's role was to listen, to recognise the significance of what he heard, and to create working conditions that allowed everyone to find meaning in their own way.

In the same way that Wren appreciated the vision, the E2E architect plays the same, working with, collaborating, listening and negotiating with individual teams. Feeding a business requirements spec to each technical division independently, without a golden thread joining everything together, is not the most effective way of maintaining coherency.

[Next section: Why a solution architect and not a technical project manager instead?...]