I have shared my thoughts in the past on the subject of architects in the domain of digital TV software systems which can be found here. In this post I will share further thoughts about the specific role of an E2E (End-to-End) architect from a program/project management perspective, including my expectations of the role as well as the associated challenges I've faced with this concept, especially where the role either does not exist, or is not clearly defined in cases where companies don't have a clear enough understanding or have not yet set up a formal architecture competency.
My Background & My possible Anchor Bias
In my first decade of working experience with software and systems engineering projects, I was fortunate to work with world class technology service providers (S3 in Ireland & NDS/Cisco in UK). This experience helped shaped the way I would view future technology projects, including product & project management, software engineering, integration and delivery functions. It's the kind of experience that never leaves you, especially when you've witnessed first hand, one successful delivery after another both as an engineer and manager. So I'd come to see the world in a particular light, taking with me the style of project execution where ever I went, and assumed that all companies naturally followed similar concepts. And the cool thing about this experience was a well grounded appreciation for software & systems engineering design principles - principles which should not be confused with implementation methods, as what is a hot topic of debate these days, that of Agile V Waterfall, that has been the bulk of my recent challenges in projects in the last six years.
On returning back home to South Africa* still working in Digital Media TV domain, I soon realised that things are a little different to say the least (absence of architect roles, working with business analysts where I'd been used to analysis being a function of architecture, no defined role of systems integration, no method of continuous delivery & lack of solid post-launch monitoring & operations support systems). My first major program, involved a large-scale overhaul of the end-to-end broadcast system, including a video-on-demand component and a new set top box software stack. In order for the program to have a decent chance of success, I not only had to restructure & reshape the program, but also had to introduce functional roles that for me, were always obvious: an architecture team with a specific role of E2E architect, E2E systems integration, test and delivery teams. With a lot of hard work of convincing senior management these roles were necessary for project delivery, we managed to implement the structures I proposed that ultimately helped in the successful delivery of the project.
Strangely enough, as I took on other engagements in large programs in the same group of companies, I found myself repeating the same. It could be because I have a bias that is so strong and deep that does not allow me to see any other way, even though I always try to adapt my style to the culture and teams currently at my disposal, yet I still find myself coming back to these principles because its been proven and tested, that I can't really find any real fault in - possibly it is because these are indeed engineering principles that have stood the test of time, especially this concept of an End-to-End Solution Architect role. As the saying goes, never leave home without it, if I'm running a large program of work, that involves many systems & component suppliers, I never run a program without having an E2E architect attached to my programs, as step one. Once that is established (often as a result of canvassing support which takes up some time and energy), I move on to structuring the role of E2E systems integration, which in turn drives the processes and behaviours expected from component engineering and test teams. The program timeline then shows a simple picture of the high-level milestones involved, where the bulk of the program management and execution is a function of coordinating delivery plans with respective project delivery teams (who maintain the detailed project plans) as well as managing stakeholder engagement (which is a vital component of running large programs). This hasn't been smooth sailing all the time, as I'll try to share some of the challenges that frequently come up, later on in this post.
*Disclaimer: Although my writing is about an experience with primarily one industry domain in South Africa (Digital PayTV), I'm mindful about not passing a broad value judgement against all large corporates with a software competency. At first I thought these challenges of projects and roles like BAs for example was limited to just this environment, but the more I researched by skimming through job specs, attending networking events, technical conferences, meetups, training courses and meeting people from other corporates, I later realised the patterns run across all the major corporates like financial services, banking and the big telcos, that more often than not, the structures in place mirror more traditional-IT governance than hardcore technical software development structures that I was used to, in working in a pure technology services environment. So I'm reasonably confident there is no broad value judgement being applied here, that these experiences are likely to resonate with large companies operating outside PayTV.
Typical Landscape
The picture below aims to illustrate a typical landscape of this ecosystem. This could be separate entities run independently, or even group under a technology division, with different functional expertise, that in reality are still perceived as separate silos of responsibility:
This picture speaks to a typical reality of technology service providers in a digital TV value chain. Each silo represents a set of core technology expertise that exist in separate functional lines either under a technology division, or could exist as separate lines of business altogether.
Each silo may have its own set of speciality with architects and development teams. Each unit may also likely to have a very unique culture: people, skill-set and ways of working. Each silo may also have its own operating model: how work gets prioritised and injected, build, test and release procedures. These units might also have their own terminology, a lexicon / vocabulary for their specific domains. They may also not be entirely self-sufficient, relying on external third-party component suppliers that form part of their system. These third-party suppliers are just like another silo, each with their own operating model, from architects to development teams, to ways of working as stipulated by their underlying contractual agreements.
These systems & teams usually come together in delivering a product or service that directly impacts a customer (end user). Typically launching a major new product (or platform, say for example a new internet product like a streaming app for video on demand) will impact at least all these systems. To do so, a project is set up as a program delivery initiative that will call for the co-ordination of all systems coming together in delivering the new product or feature to market. So a program team would usually be setup to manage this execution and delivery. Typically a program manager would be appointed to manage the full end-to-end delivery, which is not only the technology arm, but also the wider business streams for delivering product to market (marketing, customer care, sales, training, retention, communications, legal & regulatory). Sometimes, the overall program manager is supported by an end-to-end technical program manager, but this is not always the case. More often than not, the overall program manager assumes co-ordination of the technology ecosystem as well. For now though, I use program manager to either mean end-to-end technical program manager or overall program manager interchangeably.
The need for an End-to-End Solutions Architect
The picture below illustrates the positioning for a dedicated role of E2E Solution Architect to a project or program of work:
Just as you would appoint an end-to-end project / program manager to the role of maintaining the coherency of the overall delivery project, so too should you assign an end-to-end solution architect role, to maintain the integrity and coherency of the overall technical solution design. This is the crux of my proposition. Related to architecture is a close relative called end-to-end systems integrator role, I've touched on systems integration topic on a previous post called Worlds of System Integration, so will not discuss integration in this post. Related to end-to-end integration is testing, in my past experience, integration implied testing or QA, but as I later learnt on coming to back to South Africa, most companies still see QA as a separate activity, so I have another post that talks to the Worlds of QA testing.
Why?
Because it makes running a project so much simpler!
When I work with an E2E architect, I use it as a vital supporting pillar of the project. The E2E architect will take care to understand the business or product requirements from the business owner, works with all the respective technology domain experts (usually the assigned system architects) on mapping out a solution architecture, defining the new technology dictionary and model (if needed), highlights the system building blocks and supporting components that would be impacted, agrees on high level technical requirements from such systems, as well as important integration points, technology risk assessment, along with unearthing required non-functional requirements (performance, stability, redundancy, etc.) that would be needed for a complete solution. All of this is kept high-level, with just enough information to allow the project team to shape and scope the project, as well as prepare the technical teams of design constraints to think about. All of this is done in the early stages of shaping and scoping a project, has proven in my experience to be absolutely vital in a successful project outcome.
To be assigned an E2E Solution Architect to a project is no small feat. The expectations are quite high indeed. The role not only requires a sound grasp of technical concepts, but also mandates a personality that is comfortable with extreme forms of collaboration, managing and dealing with diverse stakeholders, personalities and egos (don't forget egos - tech guys can be a difficult bunch of people with huge egos), a strong suite of leadership skills, self-discipline, self motivation and the ability to work with ambiguity, a responsible attitude to self-management of tasks (without depending on a project manager to co-ordinate every meeting for example). Added to that, the E2E Architect must have excellent communication skills, demonstrating both in written and verbal form, the ability not only to handle technical communications, but also be the person to translate to business stakeholders in a simple, non-technical manner.
I rely on this architect to also grasp technical and architectural issues that might come up in delivery, and be able to provide solutions and advise on ideas and proposals to manage any impediments on the overall solution provisioning.
Why can't you just feed the business requirements to each division?
The short answer is the risk of lack of coherency of an overarching, end-to-end solution design. The absence of an E2E architect acting as the gatekeeper for maintaining overall integrity and coherency of the business requirements and related solution design, runs the risk of each division designing and implementing fairly independent solutions, leading to fragmentation, increased integration and last-minute rework to get a coherent solution delivered. This also leads to a system implemented without foresight, lacking the vision or with the end in mind. An E2E architect would fill this void, save a lot of time and energy in putting all the pieces of the puzzle together.
Though not impossible to complete a project without the E2E architect, it is often through a lot of last minute co-ordination efforts of a project management team, or some efforts of a few technical heroes coming together, taking collective ownership of the problem. Generally these units have other work and projects going on concurrently, where focus on a single project is difficult to achieve. If the environment of very open collaboration and team work, where there aren't really any silo-mentality, then yeah, it's possible. But the reality is that in most organisations, silos exist, collaboration is not the norm, and the overhead of a project management team coordinating these activities is too high, let alone the implied technical knowledge of these project managers to make this a worthwhile activity.
This reminds me of the story about Christopher Wren, the architect responsible for rebuilding St. Paul's Cathedral after the Great Fire of London. Wren was walking the length of the partially rebuilt cathedral when he asked three bricklayers what they were doing. The first bricklayer responded, "I'm working." The second said, "I'm building a wall.". The third paused, looked up, and then said, "I'm building a cathedral to the Almighty."**
How many teams fully appreciate the work they do in the context of the end-to-end ecosystem? It is not expected every team have this kind of vision, it would be great if they could all see the big picture and be visionary, but in reality they're not - and there's nothing wrong with that. The instructive part of Wren's story is that he didn't come up with a sense of purpose himself and pound the vision into everyone's head. Each bricklayer cared about something different, even though all three were working on the same thing. Wren's role was to listen, to recognise the significance of what he heard, and to create working conditions that allowed everyone to find meaning in their own way.
In the same way that Wren appreciated the vision, the E2E architect plays the same, working with, collaborating, listening and negotiating with individual teams. Feeding a business requirements spec to each technical division independently, without a golden thread joining everything together, is not the most effective way of maintaining coherency.
[Next section: Why a solution architect and not a technical project manager instead?...]
