Wednesday, 4 April 2018

Update: Reflections on my career in Software Engineering & Management

Two years ago around this time, I explored my progress with respect to my career choices by performing some self-reflection, by asking myself some searching questions. Later I shared my findings on this blog, in this post: Reflections on my career in the hope my story could resonate with people who may be experiencing similar challenges. I'm glad I did so since people did actually reach out, thanking me for the post & providing feedback.

Anyway, two years have since passed since I last shared the cross-road I found myself at, since I'd started my journey with this path in mind...

a) Software Team Lead -> Software Manager -> Senior Manager -> VP -> Director -> CEO
b) Principal Engineer -> Senior Principal -> Technical Director -> CTO -> CEO
b') Principle Engineer -> Architect -> Senior Architect -> Director -> CTO -> CEO
c) Technical Project Manager -> Senior Project Manager -> Program Manager -> Director -> CEO

...but instead found myself being in the technical project management space for too long, that people started naturally profiling me as the "rockstar program manager" (check out my LinkedIn recommendations and you'll immediately see why). Whilst this is a great place to be (don't get me wrong, I think a career in Project Management is one of the most versatile, lucrative and flexible professions out there, I highly encourage the move), for me, I felt I'd learnt and experienced enough, that I didn't see myself doing that for much longer. I did enjoy the project leadership, but I wanted more. Another factor that was causing me anxiety was that my role as a management consultant was getting a bit boring, what I imagined it to be versus the reality were not fully aligned. My time was fully consumed, leaving me little time to explore my own ideas to look at new ideas/products (my own start-up), I might as well have been a permanent employee - I was living an illusion, in reality I was basically a "perm-tractor". I had built up enough personal equity, credibility to enjoy a decent level of referent power to indirectly influence outcomes in my favour, I still wanted more - I wanted to feel more alive than being a neutral facilitator (which in itself I found quite rewarding but also quite energy-draining).

So what did I do? I went back to my RAGE model, here's what I had for my persona as a professional:

  • As a software professional, I would like to learn & grow, seek out individuals, companies and interactions, to reach heights of excellence, so that I can not only enjoy the profession, but take me to new opportunities & experiences. I want to surround myself with people that motivate me, journey together to grow to the next level.
  • Want to work with inspiring, motivated leaders that I can learn from. Want to surround myself with deeply technical, bright people. Want to work with people who know what they're doing or unafraid to take chances. Want to work with disruptors, people unafraid to push boundaries, challenging status quo. Want to work with people who are equally, if not, more motivated than me. Want to learn from people so that I can grow and do my own thing one day. Want to be with fellow professionals that will help take me to the next level. Want to work on projects and products that are interesting and cutting edge, not "me-too, copy-cat products. Want to stay at the cutting edge of software, be involved in the next wave like cloud services, mobile app development, car infotainment / self-driving cars, drone software, cloud, etc. Want a chance to start-up my own business in ideas in product development, services-space like crowd-based testing, etc.
  • As a professional, I want to run a company, lead my own division. I believe the experiences and skills acquired over the years puts me a good position to do this, regardless of technology stack. I haven't been successful in launching my own start-up, so the best place would be to go back to corporate, be part of story much bigger than myself, and get the experience I need.
I also came to the conclusion that being a specialist is not a bad thing, so I'm now settled with the fact that I'm a Digital TV Technology Specialist, so I should just focus my energy in this area. I can still keep abreast of new technologies, but the road to my continued success is to build upon this experience - the rest is noise - if an opportunity comes my way for investing or if there is something truly exciting with a lot of upside, then I still might consider it ;-)

So what's happened in the last two years?
I made a decision to leave project leadership behind. I explored opportunities that aligned with my aspiration of running my own division. I took a chance by breaking the perception that I'm the guy to call in to rescue failing projects - landing an engagement as interim GM/CTO. A year later, I decided to leave consulting (248 weeks consulting) altogether and enter the corporate world as a permanent employee, taking on a CTO/Head of Technology role :-)

So my path has indeed played out a little different but now seems to be back on track:
Software Engineer > Senior Engineer > Technical Project Manager > Senior Project/Program Manager > Principal Engineer > Program Manager > Management Consultant > CTO (now) > CEO (next)

Lessons learnt / myths busted?
Who says you can't change tracks in between (especially switch to project management) and switch back to technology leadership? It can definitely be done!
Be prepared to Leave it All Behind as long as you believe you're heading in the general direction you seek (maintain your guiding compass always).
Take time to process your situation with Life/Work by investing the time in self-reflection & planning. I found my RAGE model to be a constant source of guidance. It does take some self-control, but it will be worth it in the end, just keep at it...
It is indeed possible to start from humble beginnings and change your life for a better outcome...

Tuesday, 9 January 2018

2017 Life/Work Balance Review

2017 was a year that went by lightning fast. It seems like not so long ago I was reflecting on 2016 when I did the write up of 80/20 rule on my life/work balance timing review around this time last year. Now a year later, I'm reflecting on how my time was spent in 2017. Interestingly enough the overall profile of my life/work balance hasn't changed drastically if you look at the summaries, but once I dive a bit more into the detail it becomes evident that my time did take a knock in some areas.

I've been tracking my time on how I spend my life for three years and counting. I've still not developed an automated system yet for tracking, need to write an app that simplifies all this manual work, however I need to find the time to do this :-) Part of my personal projects which had taken a knock last year. Once I've automated this through an app, it should make Life/Work goals tracking much simpler, lets see what 2018 brings!

My Personal Value System - Personas

At the turn of 2017, I'd optimised my value system to the following elements, split between Personal & Professional lives:

This was supposed to reflect the priorities in my overall life, with the view that when it comes to time management, how I consume time should be relative to these priorities. In hindsight, this is more difficult to achieve in reality.

What happened in 2017?

Overall, the data for 2017 is shown below:

Putting this in pictures, it looks like this:

Comparing this to 2016, it looks pretty much similar at the high level:

Quick Analysis

2017 compared to 2016, the data shows roughly a balanced life/work split, although in reality I don't feel like it was balanced at all. In May 2017 I started a new job, which had me focused on work more than anything else. If I ignore the hard reality constraints, which is something I define as unavoidable, must-happen activities like Rest/Sleep, Driving-Car and Household maintenance, the picture looks a little different:

In 2017, I worked a total of 2383 hours.
The income-generating portion was spent working as a consultant, totals 2339 hours.
Accounting for 168 hours as the legal working hours per month, this works out to 13.33 months.
Assuming a full calendar month for leave & public holidays (11 working months), I've worked an extra 2.33 months in 2017!

Whilst time spent with Family exceeded Work time, my personal time for my own individual interests took a knock. I've hardly spent time on hobbies, pet projects as well as general health/fitness/well-being.

2017 Lessons Learnt

I worked more in 2017 than 2016, in 2018 I need to reduce work-time, to focus more on personal time. If Personal time invested as an Individual is prioritised higher than Work then I need to figure out a way of allocating more time in this area. I can't compromise on the Family aspect so the time has to come from somewhere else.  Even if I leave consulting and enter full-time employment, I need to create space to value my personal endeavours. Pet projects and new ideas have had a dismal focus in 2017, making me wonder if it's worth any focus at all - the reality is quite different from aspirations or expectations. General Health & Well-being took big hits as well. Becoming an entrepreneur is hard, running a consulting company is also hard - are these just dreams, or am I serious about these? It maybe enough just to have three major focus areas: Family, Life & Work, unless Work converts to Entrepreneur / Running my own business - need to ask myself some serious questions here. Career-wise, my trajectory is looking promising but I don't know if I should resign to the notion of just spending the rest of my years working in a corporate and just settle...

Tracking 1:1 time with each child is also very challenging, with three kids where the age gaps are not that huge. The numbers do reflect quite poorly, what I've found though is that the interactions are captured as Family time - so it might not make sense anymore to focus on sole 1:1 time with each child. I still think this quite important, and need to figure out a way of creating these 1:1 times more frequently. I may have to kick-off the weekly family retrospectives to get this going again.

How did I enjoy Work in 2017?

So if I feel I'm working more than anything else, and the data shows a major part of my life is spent in the office, am I having a good time? Am I enjoying the work? Am I spending my valuable life-hours just to get by as a matter-of-fact, or am I actually enjoying the gig, and getting some fulfilment out of it? I've been tracking my level of enjoyment at work for the last few years - here's what the data shows.

I track feelings as: 
Enjoyed - Good! When I had a good productive day, relationships good, achieved something, good flow-state.
Bad When I feel I should really be doing something else, had a bad encounter, things pear-shaped.
Indifferent / Neutral Neither good nor bad, just another day-at-the-office, uneventful.
Personal Time Time off to focus on personal topics: Leave, Family-Time or Personal Time Out

The theory is that if the number of Bad days increases to a level of causing alarm, then this signals an event for me to start reconsidering my options - like just leave the work and find something better. Looking at the data, there is still far greater upside than downside - 2017 has been a year of increased enjoyment at work, so there is really no reason for me to look elsewhere...unless the level of uncertainty in employment is increasing - which as I write this, might just be the case. I've learnt through the years never to be complacent, no matter how "secure" you might as I start 2018, I must still keep my options open.

2018 Key Goals/Objectives

In 2018 I need to get back to re-evaluating my state every three months. The themes to focus on for now:
  1. Find a way to reduce work hours to a point that it is actually sustainable, and not be the only thing that consumes me (even if I'm having a good time at work). Goal is to maintain a consistent level of 168 hours maximum per month on work. Need to do this by building an awesome management team, foster leadership & responsibility downstream. Implement behaviours from "Turn this Ship Around".
  2. Create space for Personal endeavours, focusing on leisure activities as well as health & fitness. 
    1. My 3D printing experiments seized in 2017. Start again.
    2. Cycling virtually non-existent in 2017. Start again.
    3. I have a boxed Lego Robotics Mindstorm set waiting for me (gifted this toy for my birthday)
    4. Five programming books to read and new languages to code - waiting for a year, unopened. 
    5. Read 24 books this year. In 2017 I completed just 9 books of the 38 I'd planned to read!
    6. Create the software for my RAGE tracking tool.
  3. Continue to nurture the family time to be as memorable & enjoyable as possible.

Thursday, 3 August 2017

On Managing Change: Damped Sine Wave

I recently read a piece from the June edition of ACM, Q&A with Erik Meijer, which I found quite apt as it speaks to my current situation at work. Whilst Meijer talks specifically to software projects, we can apply this to any kind of topic, be it personal or professional, work & life - everything we encounter when it comes to dealing with change & uncertainty (career, new teams, family, new projects, etc.).

This is especially relevant to the period I'm in now, a leader driving change - i.e. changes to the way we working, kicking-off a focused management war room, and the recent announcements around delivery priorities for the next three months. This is a large corporate, multi-million dollar industry, the amazing part is that things are so fast-paced and always changing that one can argue that the only constant in this company is one that of Change!

So we need to deal with this reality, it's not going to go away anytime soon. My experience is pretty much aligned to Meijer: try to manage the level of uncertainty (balanced by one's appetite for risk) as quickly as possible through communication & stakeholder engagement, converge on well-bounded known-knowns based on the information we receive, agree to execute & deliver within the binding time constraints. Add to this is a sense of measured calm and patience, start practising ways to control your default responses too...

Courtesy: ACM

Below is the excerpt of the Q&A:
What is your team process? How does work get done? How do you communicate status? 
A lot of what you read about process and agile has very little evidence behind it. I don't believe a lot of process is scientific. Instead, I define general guidelines about what I want to see happen, and within those I don't care how things happen.
My thinking has two main sources of inspiration: the military and the hacker way.
Over thousands of years, armies have figured out how to get things done and achieve their goals in an environment that is really chaotic and unpredictable. That is the environment we live in as developers as well. If you read the U.S. Marine Corps Warfighting manual, and replace the word war with software, everything in there holds true.
So how do you deal with uncertainty? When people attempt to solve with process, they are trying to fight or control uncertainty. For example, someone can say just adopt zero inbox and your life will be awesome. In reality though, that isn't really the case.
One of the things I like about Facebook is "the hacker way". It is an approach to creating software that involves continuous improvement and feedback. It is about computational thinking: how do you program the system, and how do you make the system do things that no one thought possible?
Being agile is about communication.
The process needs to change with the situation. You have a big picture of where you want to go, but any plan or process will shatter immediately when you hit your first bug or something happens out of your control.
In most projects there are two phases: an exploratory phase and an execution phase. Your project should progress like a damped sine wave, where the amplitude gets smaller over time (see picture above). You have to figure out what to build, and figure out what question you are trying to answer. In the beginning you want to increase the vertical velocity to get the uncertainty under control, and then you want horizontal velocity to increase when you get into execution.
With prescriptive processes. people are looking for a silver bullet to solve problems, but it doesn't exist...the world is super-confusing, and you have to embrace it and work with it.

Sunday, 18 June 2017

On driving change: Kotter's Model

I've not only been studying organisational change management for a while now but also been part of some of the process in my recent engagements with clients. The subject has started to fascinate me, which has created a deeper sense of appreciation for system dynamics in the workplace.

I've decided to capture some frameworks on my blog for note keeping, since it will become a likely go-to place for me to reference. I'll start with Kotter's model and build from there. 

The source of my information in this post, is cited & referenced from Leach's Critical Chain Project Management book, Chapter 11, Pages 283-286.

1. Create urgency

The first step is to get a sense of urgency building within the organisation that something must be done. Most people in most organisations feel overwhelmed by keeping up with the daily workload so suggesting doing something more to change the way the organisation works looks at best to be just more work and at worst to be something that is going to make things worse than they presently are. People need something to motivate them.  Kotter suggested some things that work:

  • Show others the need for change with a compelling object that they can actually see, touch, and feel.
  • Show people valid and dramatic evidence from outside the organisation that demonstrates that change is required.
  • Look constantly for cheap and easy ways to reduce complacency.
  • Do not underestimate how much complacency, fear, and anger exists in your organisation.

2. Build team

One person can only succeed to cause change in a very small organisation. Many people are not even able to cause changed behaviour in one person: themselves. Think of how many people succeed at losing wait or stopping smoking? Planning real change at an organisation level needs help, you can't do it alone. You need to enlist the leaders of the organisation who have bought into the sense of urgency. Kotter suggested the following that could work:
  • Show enthusiasm and commitment to help draw the right people into the group.
  • Model the trust and teamwork needed in the group.
  • Structure meetings for the guiding team to minimise the frustration and increase trust.
  • Put your energy into step 1 (raising energy / urgency) if you feel you cannot move on to step 2.

3. See vision

People need to be able to see the proposed change because that is what can begin to create an emotional feeling that will motivate them to change. A vision should be a picture of the end result. If you describe it in words, the words need to evoke an image. Kotter suggested:
  • Try to see - literally - possible futures.
  • Make the vision so clear that you can articulate in one minute or write, or better yet draw, it on one page.
  • Supply a moving (emotional) vision such as serving people.
  • Put forth bold strategies to make the vision real.
  • Focus on how to quickly make the change.

4. Communicate

So people can feel the change, you need to communicate:
  • The vision in terms of the benefits people will see when they change their behaviour.
  • What has to be done to make the vision a reality.
  • Reinforcements when people exhibit the right new behaviours.
  • "Wins" by people and groups who do the new behaviours.
  • Anything and everything else about the change that will keep at the top of people's agenda.
Kotter suggests some ideas on communicating:
  • Keep communication simple and heartfelt.
  • Do your homework before communicating, especially to understand what people are feeling.
  • Speak to anxieties, confusion, anger, and distrust.
  • Rid the communication channels of junk so that important messages get through above the noise.
  • Use current technologies to help people see the vision.

5. Empower action

You need to empower action: make sure people know that they are expected to take action now and that they are free to do it as the see fit. Empowering action is as much about removing obstacles to action (pulling) as it is about causing people to act. Kotter's suggestions:
  • Find individuals with change experience to bolster people's self-confidence with "we-won-you-can-too" stories.
  • Recognise and reward in ways that inspire, promote optimism, and build self-confidence.
  • Deal with disempowering managers through coaching or move them out of the way.

6. Create wins

Your team needs to coach people to create successes: wins. Then you need to reinforce the behaviour of those who created the wins and communicate their wins and reinforcements to the rest of the organisation. Pilots are a powerful tool to create short term wins but you need to ensure that people who live those wins with the pilots do not immediately go back to prior behaviours. Kotter's suggestions:
  • Early wins that come fast.
  • Wins that are as visible as possible to as many people as possible.
  • Wins that go through emotional defences.
  • Wins that are meaningful.
  • Early wins that speak to powerful players whom you need to engage.
  • Wins that are cheap and easy even if small.

7. Do not let up

The leadership team has to keep the desired change at the top of agenda through and well beyond the planned-for successes. There will be obstacles and there will be some failures along the way but the winning teams take failure as a learning and motivating experience to add vigour to the change process. Kotter's ideas:
  • Rid yourself of work that wears you down - tasks that mattered in past but may not matter now or tasks that you can delegate.
  • Constantly look for ways to keep up the urgency.
  • Use new situations opportunistically to launch the next waves of change.
  • Show 'em, Show 'em, Show 'em...

8. Make it stick

Once you have completed the first round of getting the organisation to exhibit the desired new behaviours, you need to continue right on to improve what you have accomplished. If you do not continue to improve, the organisation will revert to the previous behaviours in a surprising short period of time. Kotter's ideas that work:
  • Never, never, never give up on step 7.
  • Use new employee orientation to demonstrate what matters most in the organisation.
  • Use the promotion process to place people who exhibit the new behaviours into influential positions.
  • Tell vivid stories over and over about how things now work.
  • Ensure continuity of behaviour and results that help sustain and grow the new culture.

Saturday, 20 May 2017

The Golden Circle

In this post I share a snippet of information referenced from the book Start with Why by Simon Sinek. The ideas and concepts provided in the book talk mostly about how some companies are more successful than others (using Apple as a major theme). It advocates that most companies often don't take time to appreciate the WHY they in business in the first place, and more often focus on WHAT they do. By focusing on the WHAT they tend to lose a captive customer audience, failing to address to the core beliefs of customers, which is one of the main reasons customers choose to go a brand, or become an ardent follower of the company.

I found this pretty interesting, with a wider relevance outside of just corporate strategic management. The golden circle can be applied to both personal and professional topics, and links nicely to my RAGE model. It can also be applied to team structures as well. In fact, in the organisational sense, the golden circle flows from top-to-bottom: starting with the company vision and flowing down to each person in the company.

What follows is the core explanation from the book, chapter 3, The Golden Circle. Simon Sinek's TED talk is also one of the highest ranking talks on TED, video is embedded at the end of this post. Although Sinek makes reference to many stories and cases that appear as old news in today's time, they are still nevertheless relevant, at the very least a useful reminder...

The Golden Circle

According to Sinek, the golden circle can be used as a guide to vastly improving leadership, corporate culture, hiring, product development, sales and marketing. It even explains loyalty and how to create enough momentum to turn an idea into a social movement. And it all starts from the inside out. It all starts with Why.

Starting from the outside in, Sinek describes the terns:

WHAT: Every single company and organisation on the planet knows WHAT they do. This is true no matter or big or small, not matter what the industry. Everyone is able to describe the products or services a company sells or the job function they have within that system. WHATs are easy to identify.

HOW: Some companies and people know HOW they do WHAT they do. Whether you call them a "differentiating value proposition", "proprietary process" or "unique selling proposition," HOWs are often given to explain how something is different or better. Not as obvious as WHATs, many think these are the differentiating or motivating factors in a decision. It would be false to assume that's all that is required. There is one missing detail:

WHY: Very few people or companies can clearly articulate WHY they do WHAT they do. When I say WHY, I don't mean to make money - that's a result. By WHY I mean what is your purpose, cause or belief? WHY does your company exist? WHY do you get out of bed every morning? And WHY should anyone care?

When most organisations or people think, act or communicate they do so from the outside in, from WHAT to WHY. And for good reason - they go from the clearest thing to the fuzziest thing. We say WHAT we do, we sometimes say HOW we do it, but we rarely say WHY we do WHAT we do.

But not the inspired companies. No the inspired leaders. Every single one of them, regardless of their size or their industry, thinks, acts and communicates from the inside out. They start with WHY...

Get the book, watch the video!

Monday, 17 April 2017

On E2E architect role in projects

I have shared my thoughts in the past on the subject of architects in the domain of digital TV software systems which can be found here. In this post I will share further thoughts about the specific role of an E2E (End-to-End) architect from a program/project management perspective, including my expectations of the role as well as the associated challenges I've faced with this concept, especially where the role either does not exist, or is not clearly defined in cases where companies don't have a clear enough understanding or have not yet set up a formal architecture competency.

My Background & My possible Anchor Bias

In my first decade of working experience with software and systems engineering projects, I was fortunate to work with world class technology service providers (S3 in Ireland & NDS/Cisco in UK). This experience helped shaped the way I would view future technology projects, including product & project management, software engineering, integration and delivery functions. It's the kind of experience that never leaves you, especially when you've witnessed first hand, one successful delivery after another both as an engineer and manager. So I'd come to see the world in a particular light, taking with me the style of project execution where ever I went, and assumed that all companies naturally followed similar concepts. And the cool thing about this experience was a well grounded appreciation for software & systems engineering design principles - principles which should not be confused with implementation methods, as what is a hot topic of debate these days, that of Agile V Waterfall, that has been the bulk of my recent challenges in projects in the last six years.

On returning back home to South Africa* still working in Digital Media TV domain, I soon realised that things are a little different to say the least (absence of architect roles, working with business analysts where I'd been used to analysis being a function of architecture, no defined role of systems integration, no method of continuous delivery & lack of solid post-launch monitoring & operations support systems). My first major program, involved a large-scale overhaul of the end-to-end broadcast system, including a video-on-demand component and a new set top box software stack. In order for the program to have a decent chance of success, I not only had to restructure & reshape the program, but also had to introduce functional roles that for me, were always obvious: an architecture team with a specific role of E2E architect, E2E systems integration, test and delivery teams. With a lot of hard work of convincing senior management these roles were necessary for project delivery, we managed to implement the structures I proposed that ultimately helped in the successful delivery of the project.

Strangely enough, as I took on other engagements in large programs in the same group of companies, I found myself repeating the same. It could be because I have a bias that is so strong and deep that does not allow me to see any other way, even though I always try to adapt my style to the culture and teams currently at my disposal, yet I still find myself coming back to these principles because its been proven and tested, that I can't really find any real fault in - possibly it is because these are indeed engineering principles that have stood the test of time, especially this concept of an End-to-End Solution Architect role. As the saying goes, never leave home without it, if I'm running a large program of work, that involves many systems & component suppliers, I never run a program without having an E2E architect attached to my programs, as step one. Once that is established (often as a result of canvassing support which takes up some time and energy), I move on to structuring the role of E2E systems integration, which in turn drives the processes and behaviours expected from component engineering and test teams. The program timeline then shows a simple picture of the high-level milestones involved, where the bulk of the program management and execution is a function of coordinating delivery plans with respective project delivery teams (who maintain the detailed project plans) as well as managing stakeholder engagement (which is a vital component of running large programs). This hasn't been smooth sailing all the time, as I'll try to share some of the challenges that frequently come up, later on in this post.

*Disclaimer: Although my writing is about an experience with primarily one industry domain in South Africa (Digital PayTV), I'm mindful about not passing a broad value judgement against all large corporates with a software competency. At first I thought these challenges of projects and roles like BAs for example was limited to just this environment, but the more I researched by skimming through job specs, attending networking events, technical conferences, meetups, training courses and meeting people from other corporates, I later realised the patterns run across all the major corporates like financial services, banking and the big telcos, that more often than not, the structures in place mirror more traditional-IT governance than hardcore technical software development structures that I was used to, in working in a pure technology services environment. So I'm reasonably confident there is no broad value judgement being applied here, that these experiences are likely to resonate with large companies operating outside PayTV.

Typical Landscape

The picture below aims to illustrate a typical landscape of this ecosystem. This could be separate entities run independently, or even group under a technology division, with different functional expertise, that in reality are still perceived as separate silos of responsibility:

This picture speaks to a typical reality of technology service providers in a digital TV value chain. Each silo represents a set of core technology expertise that exist in separate functional lines either under a technology division, or could exist as separate lines of business altogether. 

Each silo may have its own set of speciality with architects and development teams. Each unit may also likely to have a very unique culture: people, skill-set and ways of working. Each silo may also have its own operating model: how work gets prioritised and injected, build, test and release procedures. These units might also have their own terminology, a lexicon / vocabulary for their specific domains. They may also not be entirely self-sufficient, relying on external third-party component suppliers that form part of their system. These third-party suppliers are just like another silo, each with their own operating model, from architects to development teams, to ways of working as stipulated by their underlying contractual agreements.

These systems & teams usually come together in delivering a product or service that directly impacts a customer (end user). Typically launching a major new product (or platform, say for example a new internet product like a streaming app for video on demand) will impact at least all these systems. To do so, a project is set up as a program delivery initiative that will call for the co-ordination of all systems coming together in delivering the new product or feature to market. So a program team would usually be setup to manage this execution and delivery. Typically a program manager would be appointed to manage the full end-to-end delivery, which is not only the technology arm, but also the wider business streams for delivering product to market (marketing, customer care, sales, training, retention, communications, legal & regulatory). Sometimes, the overall program manager is supported by an end-to-end technical program manager, but this is not always the case. More often than not, the overall program manager assumes co-ordination of the technology ecosystem as well. For now though, I use program manager to either mean end-to-end technical program manager or overall program manager interchangeably.

The need for an End-to-End Solutions Architect

The picture below illustrates the positioning for a dedicated role of E2E Solution Architect to a project or program of work:

Just as you would appoint an end-to-end project / program manager to the role of maintaining the coherency of the overall delivery project, so too should you assign an end-to-end solution architect role, to maintain the integrity and coherency of the overall technical solution design. This is the crux of my proposition. Related to architecture is a close relative called end-to-end systems integrator role, I've touched on systems integration topic on a previous post called Worlds of System Integration, so will not discuss integration in this post. Related to end-to-end integration is testing, in my past experience, integration implied testing or QA, but as I later learnt on coming to back to South Africa, most companies still see QA as a separate activity, so I have another post that talks to the Worlds of QA testing.


Because it makes running a project so much simpler! 

When I work with an E2E architect, I use it as a vital supporting pillar of the project. The E2E architect will take care to understand the business or product requirements from the business owner, works with all the respective technology domain experts (usually the assigned system architects) on mapping out a solution architecture, defining the new technology dictionary and model (if needed), highlights the system building blocks and supporting components that would be impacted, agrees on high level technical requirements from such systems, as well as important integration points, technology risk assessment, along with unearthing required non-functional requirements (performance, stability, redundancy, etc.) that would be needed for a complete solution. All of this is kept high-level, with just enough information to allow the project team to shape and scope the project, as well as prepare the technical teams of design constraints to think about. All of this is done in the early stages of shaping and scoping a project, has proven in my experience to be absolutely vital in a successful project outcome.

To be assigned an E2E Solution Architect to a project is no small feat. The expectations are quite high indeed. The role not only requires a sound grasp of technical concepts, but also mandates a personality that is comfortable with extreme forms of collaboration, managing and dealing with diverse stakeholders, personalities and egos (don't forget egos - tech guys can be a difficult bunch of people with huge egos), a strong suite of leadership skills, self-discipline, self motivation and the ability to work with ambiguity, a responsible attitude to self-management of tasks (without depending on a project manager to co-ordinate every meeting for example). Added to that, the E2E Architect must have excellent communication skills, demonstrating both in written and verbal form, the ability not only to handle technical communications, but also be the person to translate to business stakeholders in a simple, non-technical manner.

I rely on this architect to also grasp technical and architectural issues that might come up in delivery, and be able to provide solutions and advise on ideas and proposals to manage any impediments on the overall solution provisioning.

Why can't you just feed the business requirements to each division?

The short answer is the risk of lack of coherency of an overarching, end-to-end solution design. The absence of an E2E architect acting as the gatekeeper for maintaining overall integrity and coherency of the business requirements and related solution design, runs the risk of each division designing and implementing fairly independent solutions, leading to fragmentation, increased integration and last-minute rework to get a coherent solution delivered. This also leads to a system implemented without foresight, lacking the vision or with the end in mind. An E2E architect would fill this void, save a lot of time and energy in putting all the pieces of the puzzle together. 

Though not impossible to complete a project without the E2E architect, it is often through a lot of last minute co-ordination efforts of a project management team, or some efforts of a few technical heroes coming together, taking collective ownership of the problem. Generally these units have other work and projects going on concurrently, where focus on a single project is difficult to achieve. If the environment of very open collaboration and team work, where there aren't really any silo-mentality, then yeah, it's possible. But the reality is that in most organisations, silos exist, collaboration is not the norm, and the overhead of a project management team coordinating these activities is too high, let alone the implied technical knowledge of these project managers to make this a worthwhile activity.

This reminds me of the story about Christopher Wren, the architect responsible for rebuilding St. Paul's Cathedral after the Great Fire of London. Wren was walking the length of the partially rebuilt cathedral when he asked three bricklayers what they were doing. The first bricklayer responded, "I'm working." The second said, "I'm building a wall.". The third paused, looked up, and then said, "I'm building a cathedral to the Almighty."**

How many teams fully appreciate the work they do in the context of the end-to-end ecosystem? It is not expected every team have this kind of vision, it would be great if they could all see the big picture and be visionary, but in reality they're not - and there's nothing wrong with that. The instructive part of Wren's story is that he didn't come up with a sense of purpose himself and pound the vision into everyone's head. Each bricklayer cared about something different, even though all three were working on the same thing. Wren's role was to listen, to recognise the significance of what he heard, and to create working conditions that allowed everyone to find meaning in their own way.

In the same way that Wren appreciated the vision, the E2E architect plays the same, working with, collaborating, listening and negotiating with individual teams. Feeding a business requirements spec to each technical division independently, without a golden thread joining everything together, is not the most effective way of maintaining coherency.

[Next section: Why a solution architect and not a technical project manager instead?...]