Agile team goals - straw man...

Over the past few years, going back since 2004, when I first started working in an agile-like fashion to present day, I've seen companies and teams transform their development practices, maintaining a strong desire to continuously improve from past experiences. I've seen such transformations driven top-down (where senior management saw inefficiencies with existing processes, accepted a step change was needed to survive growth challenges and thus set about a strategy to implement new processes). I've also experienced the change coming from within the team themselves, experimenting at first (in the background), proving the value added to not only the local team, but to the wider organisation as a whole (although it wasn't easy, it is generally harder to influence bottom-up).

In order to derive some success from these transformational initiatives, there is usually a person who takes on the role of catalyst or driver, maintaining a clear vision of the goal and desired outcome - keeping people together, constantly reinforcing the message, always constant reminding. Generally this person is energetic, evangelic and carries enough of an aura for people to just believe & follow, the mark of a true leader. This is also made possible by having a team that has been through the mill, have battle-scars to show for it, and are really sincere in adopting the new strategy.

But what do you do, when you have a relatively young development organisation, cobbled together from various parts of the world, all differing skill levels & competencies, and when the leadership is lacking (incoherent) such that no one in the department has ever delivered a project successfully using Agile/Scrum before, have never really seen the development, people & cultural transformation required to realise the change from start-finish, and not have had the patience & privilege to live the agile journey of a few years??

What do you do when a team of aspiring young engineers that are just starting out on delivering formidable, real world-class products for the first time, who have been used to the heavy-handed old school project management styles of yesteryear -- suddenly fall in love with the premise of Agile -- but unfortunately lack the depth and breadth of understanding the real challenges of software engineering (product development, not proof-of-concepts)??

In the business of Set-Top-Box software product development, this is the software that provides your TV services like Personal Video Recording, Video On Demand, Linear TV, Electronic Program Guide, etc. A mixture of hardware design, low-level drivers, middleware operating system & high level user interface. Technologies are Java, Flash, HTML5, C/C++, embedded environment. This space is becoming highly competitive, how do such software providers remain competitive??

The answer generally touted is "We're adopting Lean/Agile/Scrum - we deliver fast, work with a rapidly changing market, flexible & practical" principles for product delivery -- but in reality ain't that easy - Agile adoption is a journey of at least three years, including time for experimentation & failure - it doesn't happen overnight. Some foundations need to be in place, principles that become the substrate for the future - a team without these foundations is operating in chaos, or as they say "Paying lip service to Agile"...

In this post I provide a straw man around goals that a development team could strive for in their quest for Agile. Just as they teach you in business school, that you need to have a plan guided by a mission, vision, strategy & plan of action -- so too can a development team ground themselves around goals that are SMART (Specific, Measurable, Attainable, Realistic, Timely). Moreover, if a team can latch onto some Real, Tangible, Concrete, Credible messages, these can be quite powerful in galvanising the teams around a singular purpose:

• Straw-man goals to drive Agile Development
• Sample Software Development Team Charter

Review: The Deadline

I came across a review I wrote for Amazon sometime back 2010:

The Deadline : A Novel About Project Management is a novel attempt at describing life as a practising project manager in the software development industry. Choosing fiction as a means for distilling this experience was the right call, since many books on software project management talk about mostly the process, but less so on the fundamental human experiences.

This book attempts just that, a perspective on the people elements, and it doesn't do too bad a job either: It tells the story of a Mr. Tompkins who takes on the job of managing an army of software engineers, to create and deliver 6 software products from scratch, limitless resources. So he sets off creating an experiment: create 3 project teams for each product, each with the same goal of delivering the product by an artificial deadline (that was later made real by a tyrant manager bringing the deadline many months forward) and set about observing the progress of each team, thereby understanding the implications of their actions - in an effort to understand the secrets behind real world processes...

Whilst this book does get my recommendation, it must be noted that this was written and published over 10 years ago - which makes much of the experiments seem a bit out-dated and irrelevant. The shrink wrapped software industry age has passed on, which is why I feel the book has missed out on exploring modern practises seen with open source development, off-shoring, outsourcing; large scale distributed development, test driven development, agile, etc - with real world demanding customers, where companies are driven by competition, so hard that there is really no time for proper planning, estimating, playing around with models and predictions is a luxury rather than necessity - the deadline is really about winning new business and keeping your customers happy; and your competitors at bay; delivering software that is of acceptable quality to your customer, stressing less on established processes like CMM level 3/4/5 (depends on the industry of course), etc.

This book is considered  a work of fiction but it does contain real world references for follow-up reading. One such interesting reference is the topic of modelling your hunch-base, using for example, iThink a systems thinking tool that allows you to model business processes, using parameters for tuning and testing output of various scenarios. This, along with archaeological project data mining for metrics, will provide invaluable resource to a team when considering new projects.

If you're an experienced software manager however, then most of the encounters will not be new to you. It feels like there should be a sequel to this book, updated to support and test out current theories, offering more detailed explanations of each experiment, which is lacking here. One gets the overall idea that the project management laboratory is an interesting topic, but there is so much more to play with rather than just over staffing and design philosophy...

All in all, I am nevertheless pleased I read it and would definitely recommend to anyone involved in software projects.

2013 Update: Although times have changed since DeMarco & Lister's experiences, much of the essence with the challenges of Software Projects nevertheless remain relevant today. For those of you not familiar with DeMarco, I strongly recommend you follow-up on their works - I've read & own all of these - highly recommended, I learnt quite a bit from these, further amplifying my passion & respect for the Software Engineering Profession:

Summer Reading List over December

Since returning home* to South Africa in June 2011, life has been really pretty damn hectic, they say relocating your family & household contents is one of the most stressful things you can do in life - added to that, working with a new company presented itself with a massive culture shock. So I had my hands full with handling not only work but also family transitioning to the new life. Not to mention I quit my permanent job and ventured into consulting territory. This transformation has set me back someways: in particular, my reading list fell on the back burner, and it's only now that I'm starting to rev it up again. Another reason for not maintaining my reading as I used to in UK, is that South Africa is way behind in the online shopping space, the deals I used to get on Amazon were so great in terms of price (not to mention service!), that when I pick up books from the shelves in stores here in SA, or even online (Kalahari), I often cringe at the price tags (definitely cheaper in UK, wait). I still haven't made the leap to the Kindle yet which is the sensible thing to do (although the Pound/Rate exchange rate is crazy!), I was an early adopter a few years back, but I really wasn't impressed at the time - so still prefer my hardbound copies! I do have my eye on the Kindle Fire HDX though...

Here's a snippet of the handful of books piled on my bedside table for reading over the South African summer break:

Nissan Motors Production Line and Software Systems Development

This week I was invited by @Farid from Crossbolt to tour the Nissan Production Line in Pretoria, to see first-hand the Lean / Kanban / Kaizen production model in action: Cars come off the line every four minutes (cycle time), Zero defects straight-through rate.

I jumped at the opportunity without hesitation. Although I've ran software projects as a factory-line model before, I had never actually been in a real car production factory before. Fareed had decided to take his IT Operations team on this tour, after doing the theoretical knowledge-share on Lean.

He felt it would make more sense for his team to witness first hand the theory in action, where the metaphors can be easily visualised and applied to an IT Ops / Systems Integration space. Not a bad idea...

This post is just a brief dump of my personal take-aways from the visit. I will try to expand on them in later posts once I've let the ideas play around my head for a while to morph into shape. Despite what some people might say: Software is a creative art, it can't be rushed and boxed into a production line, when it comes to pushing out consumer production software, I beg to differ: there are indeed many parallels from the manufacturing sector (which is seriously way more mature than software development) that can be drawn upon and applied to software teams - incidentally this is the roots of Scrum / Lean anyway - just taking the same concepts and applying it to software teams...

There's plenty of info on Kaizen / Kanban / Lean / Poka Yoke / Scrum - I won't go into detail here. For context though, the line we visited was a multi-model line. This means that the line produces more than one type of vehicle in a continuous flow. Any team on the line could be working on a different model at the same time. Some car production lines specialise in just one model, but this line builds at least four different models. Because the flow is continuous, a car comes off the production line every four minutes, like clockwork. The team working on the line therefore, are cross-functional and cross-skilled. 

Take-aways:

Career Ladders: Avoiding chaos and anarchy in Software / Systems Projects

This post is part rant & part structured discussion around the topic of career development in the domain of Software & Systems Engineering. I have been in the business of core software engineering for fifteen years (15), and was fortunate to experience working with highly rated professional companies (from Silicon Valley, internationally renowned) whose sole purpose was producing software systems from scratch, including as providing software design services to other software companies - these companies maintained true to engineering as a discipline.

A graduate engineer would enter the company with a fairly good idea of the where he/she is starting, and the various options & opportunities that lay ahead in terms of career growth, and, depending on the company - this graduate could spend

the next 10-15 years with just one company alone and never get bored, traversing many job disciplines as he/she so desired. 

This is the baggage (rightly or wrongly) that I come with, and hence my surprise to learn that some companies in South Africa are still making a serious mistake of not having a simple template that maps roughly the career ladders for the team, covering Development/Test/Integration Engineers, Architects and Managers. I firmly believe this is a recipe for chaos & anarchy that must be avoided as far as possible, and the solution to this problem is to map out a simple Career Jobs Ladder for your technical department.

Don't get me wrong: I am all for meritocracy, flexibility and not bureaucracy - but it really irks me to see things happen in the workplace that just don't make sense, especially when promotions happen when there is no real truth, track record or backed-up peer recognition that warrants a role / title change. 

I have seen the following happen as an example of chaos:

• An integration engineer with no prior design or architecture experience is promoted to Architect
• An software engineer with no prior architecture accolades is promoted to Architect
• A software engineer with no prior team leading, people management or facilitation experience is promoted to Scrum Master
• A team lead with no prior project management or scrum mastering track record becomes an Agile Manager
• An integration engineer with no product management experience becomes a Product Manager
• A new recruit with no prior experience in the technology domain joins as a Senior Solutions Specialist
• A new architect who has never architected any product before enters as a Solutions Architect
• An enterprise architect who has never delivered to market any real enterprise-class systems product that has a user base of more than fifty is made Enterprise Delivery Architect
• A component architect who's only worked on a single software module / component becomes Enterprise-wide Solutions Architect
• An integration engineer who's only experience in embedded devices joins an enterprise systems integration team as a Senior Integration Specialist
• An automation specialist or tech lead who is misunderstood as the Head of Automation

To an observer, the above scenarios are candidates for chaos (What do all these roles mean? What's the job spec? Is there a clear map that shows the progression of one role to another?, etc. etc.), although these cited migrations would not be that far fetched if there was a career ladder to hand, that facilitates the growth path - that can be used to take the individual on the journey to reaching his/her desired goal....

And this is where anarchy comes in (again, a little rant on my part):

Agile Africa Conference - Content made available

Earlier this year I wrote about my experiences from the first Agile Conference hosted in Africa:

• Day One
• Day Two

The kind folks at JCSE have now shared the documents from the conference here, and allowed me to link independently to my own copies, should their site ever disappear. But for now, you can find the slides for each here:

• Martin Cronje – How Teams increased their value
• Sam & Karen – Agile Testing Mindset
• Riaan du Doit – Agile Adoption with Parental Guidance
• Aslam Khan – Apartheid and democracy
• Derrydean Dadzie – Case of 2 worlds
• Ana Forssman – To be Agile or not to be Agile

Clips have also been made available on youtube, click here for the full playlist.

Enjoy!

Common Challenges with Shared Component Development in Agile

In a previous post, I shared a method of Software Development & Integration that component teams adopted for a very large scale project, where the development & integration team spanned in excess of 250 people, geographically dispersed across the world, where the system software stack was a Set-Top-Box Middleware product, consisting of eighty (80+) software components, each component owned by individual component teams, a component team size being anywhere between five and twenty people (all software engineers). These component teams implemented Agile/Scrum at their own individual team level, and had to also deliver their components into project integration streams, with multiple project delivery timelines overlapping simultaneously using a common iteration heartbeat of six week cycles.

This was grand-scale, iterative development & continuous integration that did come with an enormous infrastructure overhead as well as quite a structured implementation of Agile Product Management. Please refer to my earlier post that introduced that particular case study.

In that case study, I maintained that the principles of development & engineering strategies equally apply to small, singular component teams as well as large-scale, distributed teams. In the end, it is just managing a component development & delivery stream.

The purpose of this post is to drill into some of the challenges that impact Set-Top-Box Software Development projects, regardless of the component impacted - especially when component teams have to maintain a common product code-base, yet at the same time manage multiple project deliveries to customers, often with overlapping timelines. I will touch on some of the scenarios development teams face, and highlight the implications of co-ordination & control aspects, in relation to how going Agile/Scrum possibly makes it simpler or more complicated...

This post is structured as follows:

• Typical Project/Product Management Flow
• Implications on Maintaining Common Architecture / Code Base for Component Teams
• Pros/Cons of various Options
• Concluding Remarks

Lightweight Open (Open Source) Streaming Software Stack

Continuing with putting myself out there and sharing some of the ideas I had in the past for possible start-ups, here is one that I journal'ed in October 2006, back when really no one was talking about user-generated content or become your own private-broadcaster.  Youtube only released live channels somewhere around 2010/2011, so the emphasis for me here is that my ideas were not that wildly off... So yeah, we have most of this already, but in 2006 when I had the idea, no one really was pushing publicly for this, and it flowed naturally as a feature extension to you-tube, and now other social networking sites as well!

-----------------------------

Posted 17 October 2006 

I imagine in the not-too-distant-future that any device with a display (reasonable size), integrated camera (this is the future), network connection (ethernet, wifi, cell radio, usb) and built-in speaker/microphone - will want to participate in the live media streaming phenomenon. Be it mobile phones, handheld games, integrated digital cameras(camera with networking, combo PDAs, etc), PSPs, GameCubes, XBox'es, STBs, etc, etc.

The numbers speak for themselves: Mobile phones have surpassed the 1 billion mark in 2004/2005. Handheld games have topped the 100 million mark; and the number of digital cameras is growing all the time. Not forgetting integrated PDAs that are GPS receivers, mobile phones and media players all-in-one.

All these devices provide for some form of content consumption. Coupled with a network connection, the next stage would be to share this content or even stream it...(Forget power requirements for now - assume the power/battery life is already sorted)

Basic Idea: Provide an open lightweight Streaming Software Stack (not a thick middleware) that can be easily ported to a variety of platforms; and integrated seamlessly with other technologies. The software will enable any device with a network connection with or without an integrated display to stream to anyone that's interested (variety of models will probably exist here). It would be better if there was an integrated camera that you could use to stream your live video (could act as video phone).

Could charge a royalty for this stack, or sell it once off - easily re-use STB middleware IPR to provide this stack - If I had my own start-up; I would definitely want to include this software stack in my portforlio!!

If this were possible (bandwidth, network & streaming issues aside) the scenario would be:

- A service provider on the web provides the facility to members of streaming content live (near realtime) or recorded

- Members will have user accounts, etc

- Members themselves will be able to setup individual user accounts that grant web surfers access to their content

For example: My portable (A/V capture) device will connect to the intermediary streaming server. The streaming server will receive and simulataneous stream the content to interested users (i.e. connected).

This will really open up the world of content sharing - literally, anyone can be a broadcaster. People can act as reporters, reporting all sorts of interesting/weird things - people can share live experiences with family/friends located at different ends of the globe, etc, etc....

PayTV Operators should STOP re-inventing the wheel and should NOT see Google as Evil!

I have been working within the PayTV Software & Systems space for almost fifteen years now, and from the very beginning with Set-Top-Box software, I really wasn't impressed by the software technology. Apart from the hardware being fairly interesting, basically a device for decoding video/audio/data streams based on MPEG / DVB / ATSC / etc protocols and standards, when it came to software, there wasn't really a "wow" factor in it. Of course, we can't forget the really crucial element, the heart of the system, the crown jewels, the revenue-generator, the very interesting & complicated black magic technology called Conditional Access (CA) which is really really cool, the rest of the building blocks was really around the Set-Top-Box user experience / application, which really wasn't all that new: 

• Essentially the device needed an operating system, a way to draw stuff on-screen, a user navigation interface, and some data source driving the application, traditionally called the Electronic Program Guide (EPG). The STB software thus re-applied knowledge well-known in the computer industry for decades (Model-View-Controller MVC design pattern emerged in the late 1970s), simple operating system and driver / hardware abstraction layers, C-code...

As all hardware devices tend to follow Moore's law at some stage, Set-Top-Boxes (STBs) have evolved to quite powerful machines allowing for migration to newer, modern software implementations (although nowhere near the computing power & rich experience offered in the latest smartphones & tablets - post for another day), the software too, has become more accessible than ever, with more STBs using Free Open Source Software (FOSS), particularly the dominance of the Linux Kernel as the popular operating system of note, displacing historical dominance of VxWorks, STOs, NucleusOS, uCos, etc, etc.

However, there are some components in the STB software stack that remain fundamentally closed: The STB Middleware, EPG, Conditional Access (CA) clients. Okay, ignoring the CA client, which has always been fundamental and will never go away for years to come -- the classic Middleware/EPG components really don't need to be that closed anymore. 

There is also the backend / headend information service data generators that are traditionally closed, although vendors purport "open protocols", the PayTV Operator generally obfuscates the openness by forcing business-specific modifications in the protocols, apparently unique to each PayTV Operator/Broadcaster...

Traditionally & historically though, these software components were provided by third-party vendors that PayTV Operators just accepted as the norm. Highly closed, difficult to integrate with open systems, these vendors capitalised on providing a closed system, to the extent of locking in the PayTV Operator to the entire stack, some vendors reaped the benefits of providing the end-to-end system, one-stop-shop for everything. Later, PayTV operators decided to take more control, diversify the ecosystem by enforcing the use of multiple components, not being locked-in by just one vendor, promoting open standards for integration, and more recently taking more ownership of some of the development and integration...

Yet, the models within which most PayTV operators continue to work - is still pretty much a closed one. There is an aversion to sharing, opening up technology to other parties with a view to extending partnerships as well as creating new strategic relationships. There is an huge element of mistrust, not-invented-here attitude, we-can-implement-this-in-house, etc, etc!! I am really dumbfounded with that approach...

Agile Africa 2013 Conference - Day Two

Yesterday was Day Two, concluding the very first Agile Conference in Africa, an event that was organised in a short space of time (2 months) by a team of just five people - hats off to the event organisers and sponsors. Indeed it was quite a significant milestone, a stake firmly grounded that Agile in Africa is growing. 

The second day was filled to the brim by an impressive lineup of speakers, a bit over ambitious in my humble opinion, since sometimes the audience needs more time with the keynote speakers (1 hour isn't enough), and on starting late and not keeping to time, something that could be improved for the next conference. Having said that, the day was awesome, personally, I took away a different kind of appreciation for the companies in Africa, bold enough to push Agile adoption, the openness and willingness to experiment and being OK with failure...

The talks that really resonated well with me included Martin Fowler's keynote, Ana Forrsman's review of JSE's agile experiences, Derrydean Dadzie's first time mistakes clash of Corporate/Start-up culture & Aslam Khan's sharing of personal challenges of teams operating Beyond Apartheid & Democracy (so relevant today, but people walk on eggshells around this. I myself have observed, even today, the subtle office racial nuances at play even in 2013...). I also appreciated Ivar Jacobson's first-time presentation introducing the SEMAT kernel, although I really had background knowledge of this concept from earlier this year - I think most people left that talk wondering more about the wider implications of SEMAT kernel and effects of "formalising" a way of working or thinking when it comes to producing software...

I have briefly summarized my take-away and notes here:

• Software Design in the 21st Century - Martin Fowler
• Agile and SEMAT - Perfect Partners - Ivar Jacobson
• To be Agile or Not to be Agile - JSE Corporate Perspective - Ana Forssman
• Doing Agile in a brutally waterfall enterprise environment - Derrydean Dadzie
• Enterprise Agile Adoption with Parental Guidance - Riaan du Toit
• Beyond Apartheid & Democracy - Aslam Khan
• Beyond hype and dogma: How South African teams increased their value - Martin Cronje
• Measuring Throughput - Danie Roux
• Software Development Revolution in Africa - Ernest Mnkandla
• Questions and Answer Forum

Agile Africa 2013 Conference - Day One

I attended the inaugural Agile Africa Conference in Johannesburg yesterday, hosted by the JCSE & ThoughtWorks, sponsored by an interesting array of partners. A two-day conference hosting guest speakers from all over, quite an impressive list of speakers.

I attended more out of curiosity to gauge the talent in Africa/South Africa, as I've been quite skeptical about the breadth, depth and appreciation of software engineering practices in Africa, due to my limited exposure to the local industry after having spent ten years in Europe working with Agile from way back in 2004/5, coming back to work SA felt like I'd been set back by the way software was done 15-20 years ago. Not only that, it seems corporate structures of large companies in SA are still set in this mindset that is so out of date with current modern day practices...

In my two years of being in SA, I've touched base with a few consultants and quite honestly I was not overly impressed. I felt that people were just jumping on the Agile bandwagon, without fully appreciating what adopting Agile really meant, the impact on company organisation, culture and productivity. My limited experience & interaction with these Agile Trainers also led me to believe it was just the next gravy train, people went on Agile courses, got certified as Scrum Masters / Coaches and then went out to offer expert training without being in the trenches themselves, seeing the transformation of teams through this process of Agile Adaptation / Evolution, basically training us without any substance/depth.

So I was looking forward to getting an insight into the Agile community in South Africa - and boy, was I pleasantly surprised! The turnout for the conference was impressive, the Alex Theatre in Braamfontein was almost full to capacity - there is indeed lots of interest in Johannesburg alone. The line of local speakers impressive (respect!). The presentations were world-class, engaging, interactive, inspiring, motivating - but most of all, they echoed the values that I've shared on Agile all along, renewing my own energy and passion, since the last two years have been quite a struggle to get a company to move from one level of Agile maturity to the next level, in the context of a major product delivery, in what's become quite a difficult and rather ambiguous Agile journey.

What I've also taken away is that the speakers haven't really conveyed truly enlightening insights to me, that I myself have been in Agile for 8+ years, and I've built up my own body of knowledge (through my own work experience, training & readings) that I think is possibly worth sharing, especially with companies working with Large-Scale Software Systems in Digital TV projects. The white papers I've written have been my first attempt at distilling this knowledge, but I realise now (well, this has been on my mind for a long time now) that I need to do more, since quite frankly, the medium isn't really working for me. My white papers are long and tedious to read, often my readers suffer from Communication Saturation. I've made strides at shortening my posts, using visualisations, etc - but now I think I should really put myself out there, ask for feedback on my writings from these esteemed speakers, and also take the leap into presenting my own case studies at similar conferences! Time will tell...

I've summarized my personal key take-aways from the first day here:

• Successful Agile Transitions: Patterns to ensure success (or failure) - Mitch Lacey
• Agile for Creative Industries - Trevor Kimenye
• Agile Testing Mindset - Sam Laing & Karen Greaves
• Teams and Sticky Notes versus Organisational Agility! - Arrie van der Dussen
• The Process Increments Method: Case studies from Egypt - Amr Noaman Abdel-Hamid

The Projects Manager as a Shepherd

It is now eight years and counting that I've been involved in managing Software & Systems Projects as Projects / Programme / Portfolio Manager, enough time I suppose, to have built up at least some experiences that might count as pearls of wisdom, that I could impart on fellow young Project Manager newbies entering the role; or to young engineers doubting the usefulness & effectiveness of the various Projects Manager roles.
I started off as an EIT (Engineer-in-Training), following the usual technical ladder reaching the point of Senior/Principal Engineer. I could have stayed on the technical path (I enjoyed writing code, wrote some really good code too [it's a great feeling when a consultant comes to you after your code has been in circulation for three years saying "your code is one of the most beautiful pieces of code I had to extend, it was really well written, a pleasure to maintain"]) but I always intended to learn everything about the Software business and so couldn't see myself inclined to writing code, debugging, doing maintenance, support or integration forever. I also did not want to become a traditional Line Manager that gets to look after lots of people (although I could) doing appraisals, managing performance and dealing with Administration & HR issues, not to mention Office Politicking. So I delved into Technical Project Management, just around the time that Agile/XP practices were gaining strength and coming of age.

I felt that Project Management touched on a variety of aspects of the business of Software, Management, Business, Execution & Delivery - so I wanted to experience it. The role includes managing stakeholders up and down the business, engaging with engineers directly, doing demos and exhibitions, dealing with customers and third parties, setting up contracts, commercials and strategic planning, interacting with sales/marketing and also contained (to my pleasant surprise) a lot of coaching, mentoring and up-skilling project team members -- so pretty much touching on all areas of the business, a little more exciting than being just a Software Development Manager IMHO...

Quite recently, a young graduate newbie once called me a Pseudo-Technical Manager, inferring to someone who has dabbled in Technical/Engineering and just wasn't cut-out for the Techie nitty-gritty stuff. A young Project Manager would have been outraged at this insult of this arrogant young-fool, child of the Y generation - challenging the new PM in the first week of the job! Alas, I just smiled at him for his passion in being an engineer and for his naivete in not understanding the bigger picture. I could have pointed this graduate software engineer to all the code I've written and products delivered that was light years ahead of the current company's portfolio - but instead I let it lay (He will grow in time, and besides, PMs have a thick skin - we don't get offended or take things personally)...

Projects Managers, whether they come from a Technical (Software Background) or not (Construction, Mining, Psychology, Finance), regardless of the new buzzwords that come and go, are here to stay because the role offers value. Plain-and-simple. Scrum Master, Product Owner, Product Manager or not, there is still a need for a role to provide the Map, a Path to get to the Destination, clearing the Path, removing Obstacles and ultimately Executing & Delivering the Project through its People. Although, having said that, I have been biased, perhaps a little prejudiced against Project Managers assigned to a Software/Systems Project that have really never experienced the art of Software/Systems Engineering before, first hand, in the trenches. That is, I used to say "If you ain't written code before and delivered real software products, then you're ain't managing my project". In the same vein, I was also not convinced that you can be a CTO or CEO of a Technical Organisation, having never been technical yourself (a post for another day...). But through my own personal growth I have come to work and appreciate a Projects Managers from variety of backgrounds, even those of whom have never written a line of code, deserving of my appreciation...

Coming back to point, a Projects Manager is a much needed (but sometimes undervalued) role in an organisation. One of my first PM mentors stressed upon me that a PM is really a Service Provider: PMs provide a service to the project team/engineers. That concept did indeed take some time to warm up with me, as will most PM newbies experience. The first inclination of a young PM is to be misled or disillusioned with power, that "with great power comes great responsibility", "I am Project Manager now, I call the shots, I drive the plan, crack the whip!", "I set my meetings around my calendar, people must just commit because I commit", etc, etc. On the other hand, another early mentor gave me this advice "If you ain't pissing people off, you're not doing your job as a PM. A PM asks difficult questions, follows-through, pisses people off. I measure your progress by the number of complaints coming back from people - you're being a hard-ass PM!"

Young Project Managers must start off slowly, beware of jumping into the role with the wrong attitude as if you're in charge, as a PM, you have to earn your stripes, work with and through people to gain respect, build relationships and ultimately become influential and a driver, through action and taking the lead. A PM must build up a level of Emotional Intelligence (something which I at first was wary of), understand the nuances of Cultural Dynamics and Neuroscience, working with and through people...Building relationships and effective communication skills are probably the two most important soft skills you will need as a PM...

So this Project-Manager-as-a-Service spiel stuck with me for a long time, and continues to do so today. Recently, I was playing with this metaphor that a Project Manager is really a Shepherd, as in a shepherd of a herd of livestock/cattle. I googled for a bit for similar metaphors but couldn't find anything that related to the message I had in my head. So I decided to write about it here as a first attempt, hopefully it will grow and mature over time depending on public feedback.

• The Project Manager as a Shepherd metaphor
• Books that have helped me in my own PM Journey

Agile Architecture - Roadworks Analogy

I've had some interesting, somewhat frustrating conversations recently, around the roles, responsibilities and expectations of a Systems Architect. Refer to a previous post on how I broke down the various architecture roles for Digital TV Systems Projects.

What triggered the recent discussions is around understanding how architecture fits in with the Agile process. Before people who think themselves Agile Aficionados (Agileficionados) singing praise that architecture can be done just-in-time, please remember the context which I refer to: Embedded Set-Top-Box or Backend Systems Engineering.

We are not in the business of writing web applications, smart phone apps, tablets, or any desktop application, or even a web site -- where these environments are usually proven, solid and stable that fosters rapid application development cycles.

Digital TV projects, especially Set-Top-Box projects are a little more complicated than iterating through web page developments, switching versions easily through redirecting URLs, eliciting real world customer feedback for alpha/beta sites or apps, etc. STB projects generally involved multiple vendors or software suppliers, each with its own development processes. Testing is often done in a limited lab environment, whilst real world customer feedback demands an end-to-end engineering system to be deployed in a live broadcasting environment (which is currently in active use, generating revenues for the PayTV operator, so engineers better be damn sure they're not going to disrupt business operations, etc).

So some STB project teams are indeed embracing Agile (or at least trying to be Agile in the context of their component-worlds). I have written about the difficulty of doing Agile in these projects that I'm not going to repeat here. How then does one deal with the concept of architecture in Agile? Do we stick to Just-in-time, just enough or do we embrace a little structure and discipline, in keeping with Mile-wide, Inch-thick view of the future, but not waste time with too futuristic features??

For Digital TV projects, I prefer the mile-wide, inch thick approach, where we're looking at least a year ahead into the future, to develop new features and enhancing the roadmap. This is especially true for long lead features, like Recommendations, Targeted Advertising, Home Networking, or even designing the next evolution of your Middleware or EPG. And I believe that Architecture, can still be done incrementally, in an Agile-manner, although with a little discipline called foresight and pragmatic proactivity. To support this picture, I created the Paving-the-Road analogy:

What it takes to cut the mustard in STB Systems Integration

I have had this topic sitting on my backlog since I first activated this blog, now is a good time to start this topic off since recently it has become quite a topic of conversation in the workplace. I will refine this post over time, however, the first installation is just to share my thoughts around the role of STB Systems Integration, its importance and touch on the traits/skillsets as pre-requisites for the role, i.e. What I lookout for in people when hiring a Systems Integration Engineer.

First of all, one needs to understand the context of being a Systems Integrator. In the context of Set-Top-Box software, there are a couple variations to the concept of integration. The core building blocks for this software stack are generally split into the following components: Hardware, Drivers/Operating System, Middleware, Conditional Access Client, EPG Application, Virtual Machine Engine and Interactive Applications. In terms of variations of SI, it essentially falls into two categories, not so different to "White Box" & "Black Box" concepts used in testing - however, it does have a direct impact on the level of influence & amount of topic-specific knowledge the SI team has or is empowered with, thus having a direct impact on the outcome of the project.

Ultimately, the PayTV Operator owns the full stack that makes up the product. The components are often provided by multiple software vendors. Assembling these components into a workable stack is left up to the task of a Systems Integrator (SI). SI acts as the mediator, the aggregator, the arbitrator - assembler of  components, puts the build together, runs smoke & sanity tests, including full functional testing, investigates defects and assigns such to offending components. SI controls the flow of releases, plans the release schedules and is ultimately accountable for generating the launch software, thus getting it to deployment / operational space.

In a previous post on SI is King, I stressed the importance of SI. I still maintain that the role of SI is in fact the most important role in a STB project, SI has high visibility and high impact, the gateway to success and the gatekeeper for maintaining the integrity of the software. Without a strong SI team, your project is really dead-in-the-water, project will thrash for months on end and will likely result in a project doomed for failure.

In terms of growth potential for engineers, I personally believe, through my own hard-won experience as an engineer myself, having been involved in various engineering roles; as well as from my experience in managing projects across several development & integration teams, that Systems Integration (SI) is the ultimate place to be in, in terms of career progression and should be a natural development path for engineers seeking the next level from software development...

The effectiveness of the SI team is governed by the teams ability to execute on expert technical knowledge thus implementing best practices, streamlining processes, operating as an efficient well-oiled machine. More importantly, the strength of SI is determined by the calibre of its contingent Engineers, the strength of its workforce.

In this post, I will share my thoughts around the following areas:

• Basic Qualifying Criteria
• Technical Skills & Competencies
• Behaviors: Personal / People skills, including Work Ethic
• SI Manager Expectations
• My own sample of Interview Questions for SI Manager Role

It doesn't really matter on the context of Black Box or White Box, however, I do feel that Black Box SI is often counter-productive and is really at odds with the intent of true SI, but in some projects or commercial agreements, this is sometimes unavoidable. SI engineers who shine in Black Box SI would probably excel even more if they had access to source code, build tools, etc. In the whole however, when it comes to requirements for SI engineers, they should be equipped to work equally well in any context.

Programme / Systems Readiness Heat Maps

In the past I've written about the make up of the Digital TV (DTV) ecosystem or "value chain" making it quite clear that the system in itself is a complicated mix of different systems, often provided by more than one vendor. To recap, you can follow-up on these posts:

• DTV End-to-End Architecture
• Programme Organisation for DTV Programs
• Worlds of System Integration

It is not very common for DTV projects to impact the end-to-end value chain, where implementing a new feature or major product launch touches upon just about every element of the system, but it does happen. It is complicated, but it can be managed. It takes an investment in effort, diligence, rigour, sense-of-strong-will, and patience to work with the different teams, to coherently come together in delivering the overall plan.

In this post, I share what I've come to find quite a useful tool: the Programme/Systems Readiness Heat Map.

It is well-known that human beings think best in pictures. As the well known works of Edward Tufte teach us, that visualisations are a powerful way of communication. A visualisation, if done correctly or smartly, can accurately reflect or communicate the state of "things" or "tells a story" so that almost anyone can just, from looking at the picture, get the message.

So, as a Programme or Systems Projects Manager, dealing with complicated technology components, and time lines are highly parallelled and intertwined, when it comes close to the final delivery stages of the project, what is the best way for you to communicate the overall status of readiness of the system?

Sure, one can use a series of PowerPoint slides, representing the status of each Work Package, focusing on the key delivery criteria for each component - this is usually what most PMs do anyway in reporting overall Project Progress via status reports. Your SteerCo have very little patience to wade through 30 slides of PowerPoint tables...

So, what the Heat Map does is simple: On a single piece of paper (okay, maybe the size is A3!), the heat map shows the state of the entire System / Program, focusing on the key criteria for each component that determines the fit-for-purpose state that your SteerCo & Executive can use as input into making the decisions for approving final deployment: Go Live. Essentially the Heat Map is represented as an n x m matrix, with n number of components to track, m the number of metrics/criteria that must be met to guarantee successful implementation to deploy. Each entry is given a value or an associated Heat Colour (Red, Amber, Green), that when all entries are filled in, the reader can quickly ascertain the state of readiness (are we hot or cool or in between?)

I will talk you through a generic template that I've used in my own programs...

PayTV Interactive Applications is dying slowly, but surely

Interactive PayTV is dying slowly, or was it dead on arrival? Imagine these scenarios:

• OK TV, Have any of my favourite shows been recorded lately?
• OK TV, Find me something nice to watch - I am feeling good
• OK TV, I've had a crap day at work today, put something on that'll make me unwind and relax
• OK TV, What are my friends watching now?
• OK TV, What show is trending on the social scene?
• OK TV, What's selling hot on Box Office today? Show me the trailer, skip the ads please.
• OK TV, Let me see what's happening on my hangout?
• OK TV, Give me the top five recommendations today
• OK TV, What's the latest news headlines?
• OK TV, What's happening with Syria today? Connect me to any Live News Stream?
• OK TV, What's making waves on YouTube recently?
• OK TV, Tell me the latest stats on the Formula 1 (Cricket, Football) scene
• OK TV, What's the weather like at my mom's place?
• OK TV, Any weather warnings I should worry about?
• OK TV, Are there any live sports events happening in my area? Which is the closest?
• OK TV, I'm in the mood for Star Trek - make it so!
• etc

Get the picture? This is what interactivity supposed to be all about! TV is a passive medium, lean back, relax & watch TV. I don't want to read wades and wades of text on the screen! I've had a long day behind my PC, been coding all day or working with spreadsheets, the last thing I want is to navigate through some tiresome and clunky menu tree to find the information I want, then I must be forced to get up from my couch because the font & text is so small that I need to stand at least 3 feet away to read the screen! More work, more effort for me - I don't want to be interacting with the remote control to do stuff - I want to interact with my TV, not control my TV!

Yet, most of the PayTV operators have maintained to still support the original concept of Interactive TV - the famous Red Button, then the infamous loading time for the application, then interact with an eighties-style user interface. Whilst there are ways to workaround and improve the classic "Please wait...application is loading..." time, the original concepts have not changed that much.  There's a lot of chatter around Interactive TV 3.0 or second screen, etc - about the next wave of interactivity, that people would rather interact with a second-screen device (smartphone / tablet / etc), but this is not what my post is about.

Mostly, this post is part rant, and part explanation about the current predicament PayTV Operators find themselves. I've already a solution in my head that I think is the way forward, but it is probably disruptive to keep even the most open, objective PayTV operator running away at lightning speed! Just take a look at this absolutely cool demo shown recently at GoogleIO:

This is the future! I think Google is on the right track - Years ago, going back to 2005/2006, I was pushing my company to explore Voice & Text-To-Speech integration into Set-Top-Boxes. When we looked at Recommendations back in 2004/2005, we considered voice -- but hats off to Google, they've done it with Search, and I'm sure the next wave is enhancing their Interactive TV offering.

My elevator pitch: Times are changing! PayTV Operators must stick to their core business: Content provision, distribution, content protection & core information services. Partner with third parties to bring true interactivity to the user experience & platform. Focus more on Open Systems, don't control the whole world / ecosystem. Create strategic partnerships with Google (or other), creating a bridge between a closed PayTV world & the Open Social Apps world... 

This post is organised as follows:

• Brief story about Interactive TV and Typical Architecture Model
• Application Stores Overview - What PayTV Operators need to consider
• My view of a possible Future Platform / Architecture

System Integration is King - Release Manager, Build Meister-Gatekeeper

I spent a lot of energy in the last week trying to (and succeeded in) convincing some senior managers about processes, that in my humble opinion, are tried-and-tested, well-established norms in the management of software: the art of Software Release Management - taking control of component releases, especially w.r.t. approving or rejecting bug fixes. It has been good practice in terms of establishing myself as a consultant-expert on all things related to Set-Top-Box Software Development & Integration. I spend a lot of my time promoting best practices and trying to win & influence people over, something I have to do to maintain focus and direction of the overall programme plan...

The point I want to make is that, in a STB delivery project, the System Integration (SI) team is really the GateKeeper, with full authority and accountability for producing a stable, functional release. This means that SI have every right to manage component releases coming in, especially with reviewing & approving bug-fixes. If SI are not happy with a component release, for example, the component team may have fixed more bugs than was required, or fixed far fewer bugs than was requested, or made any other changes that wasn't specifically requested by the SI team, then SI can quite easily reject the component's release.

I have written in the past on the following topics:

• Template to manage STBSI Release Campaigns
• Release Schedule template to manage Component Vendors
• Modeling the Climb to launch in STB Projects
• Process for fast tracking issues, reduce the backlog on SI
• Tracking Defects using Predictive Analysis
• Effective Defect Management Processes required in STB Delivery Projects

All of these areas become the focus of a STB project as we get closer and closer to launch mode. Recapping the template that most STB projects follow:

• Develop / Integrate Features >>> Functionally Complete >>> Closed User Group >>> Wider Field Trials >>> Launch Candidate 1 >>> ... >>> Launch Candidate X >>> Clean Run >>> Deployment Build >>> Launch

As the system matures to reach each of these milestones, the defects are expected to get less and less. Obviously no product gets launched with zero defects! All projects can do is ensure as best as they can that almost all high impact, high severity defects (a.k.a. Showstoppers or P0s) are dealt with to the best of their ability; and are prepared for the inevitable influx of new Showstoppers from in-field, real world usage of the product - it is the reality, unavoidable. Very rarely do products launch having met all original quality engineering requirements, after all, this is an entertainment device, not a life critical device - so we can relax a little on quality objectives! :-)

So how should you control releases leading up to launch?? It's simple really, not complicated, and really not rocket science...

So you think writing a Set-Top-Box Middleware is easy? Think again!

So you came across this technology called Digital TV and stumbled across the various (open) standards (MPEG, DVB, MHP, etc) and thought it cool for you to write your own software stack? Or you've been writing Set Top Box Software for other customers (namely PayTV operators) and decided you have enough knowledge to venture out on your own and sell your own software stack? Or you're a

PayTV operator, sick and tired of being locked into expensive and time-consuming Middleware vendors that you decide it is time to write your own software stack, be in control of your destiny and deliver cool products to market faster, cheaper and on-time?
Or you figured there is a quick and easy way of making some money, since just about every country in the world is going to switch off analog TV and turn to digital, where you can provide a very cheap and cost-effective software stack?

If you've answered yes to at least one of these questions, then you should be asking yourself: Just how serious are you about Middleware?  Just how serious are you about the EPG User Experience?

Just how serious are you about competing in this marketplace? Is there any room for you to compete in this arena? Where  do you see your software stack in the marketplace: low-tier, middle-tier or top tier? Who are your competitors? Are you big enough, flexible-enough, savvy-enough, innovative-enough or unique-enough to displace your competitors? Or do you even wish to displace your competitors? What relationships do you have with the big technology players? What is the compelling value proposition that you're offering that your competitor's aren't? Do you stick to one large customer that can finance your operation? Do you seek out other customers to monetize on your product's offering? Do you keep a common product that can service multiple customers at relatively low cost? Do you feel the market is big enough to support many divergent offerings? Are you a small fish in a small pond, big fish in a small pond, or are you swimming with the sharks and you just don't know it yet? Do you rely on hope that everything goes well, or have mountains of cash so as not to notice you're burning fast, but don't realise it? Have you been sold vaporware, demoware, software that's far from being fit-for-purpose for real world mass production use?? Etc, Etc, Etc...

I have met developers, architects, software managers and product owners who think that writing a STB Middleware stack is easy, a piece of cake, all you need is just four really good developers...to which I smile and think in my mind, oh how naive they are! Yes, for sure, the software architecture can make sense on paper, heck you can even hack a proof-of-concept together in about three months or less that proves a simple Zapper or PVR stack. But that's just it, it's a hack, a Proof-of-Concept (POC) - nothing close to getting a real world product into the marketplace.

My rambling in this post covers the following:

• Typical Challenges for New Middleware Players
• Securing support from Chipset Vendors
• Choosing the Application / Interactive Engine / API for EPG
• Integrating with Conditional Access Vendors
• Competitive Costing / Pricing Model
• Overall Middleware Strategy
• Advice for PayTV Operators
• Conclusions - Middleware/EPG business is hard work
• Last Point: If I entered the Middleware/EPG Market today, what would I do?

Pragmatic Set-Top-Box QA Testing - Don't just trust Requirements-to-Test Coverage scripts

This post might be a little edgy - I think it's important to understand alternative perspectives around Set-Top-Box (STB) testing, how it is uniquely different to other forms of IT Testing; to understand the dynamics of STB testing that promotes agility and flexibility, whilst simultaneously understanding the risks associated with pragmatic testing.

The Headline:
DON'T BLINDLY TRUST REQUIREMENTS-TO-TEST COVERAGE MAPS

Last year, I wrote quite an in-depth paper on Effective Defect & Quality Management in typical DTV projects. It covered many topics, and touched briefly on the aspect of Project Metrics reporting. This post expands on the subject of QA Metrics tracking, focusing on how this reporting can help change the direction of the project, and instigate changes in focus to the overall QA efforts, particularly around Set-Top-Box QA testing. I advocate the project will change QA focus as stability is achieved with Requirement-to-Test coverage maps, to include more Exploratory, Risk-based & User-based testing.

Worlds of QA Testing in Digital TV Systems Projects

I have previously written about the Digital TV ecosystem with its complexities and challenges in defining the Architecture & Systems Integration spaces. In this post I will share some thoughts around the QA (Quality Assurance) / Testing problem space, as well as generally accepted strategies for solving these problems.

What really triggered this post (which is centred around E2E QA) was a recent conversation with a QA Manager who in passing commented that he's facing a challenge with the "End-to-End QA" (E2E QA) team, in that it takes up to six weeks to complete a full QA cycle. Now some might think this is expected, as E2E QA is the last mile of QA and should take as long as it takes. My response to this is that it depends...

It depends on which phase your project is: is it still in early development / integration testing - where components are being put together for the first time to test out a feature across the value chain? Or is the project well ahead in its iterations having already executed much of the E2E testing? It also depends on the scope and layers of testing the systems and sub-systems before it gets to E2E Testing. Has a version of the system already been promoted to live? If so, what are the deltas between the already live, deployed system compared to the current system under test?

[--- Side note:

This post is one of many to follow that will encompass some of the following topics:

• Worlds of testing / QA
• Is ATP / E2E testing the longest cycle than other system test cycles - or should it be the shortest?
• High level block diagrams of architecture / interfaces / system integration
• Visualisation of technical skills required for various aspects of testing skills required at various areas of the environment: Scale of technical - heat map
• E2E QA is highly technical and should be considered as part of Integration, not a separate QA function
• Which came first: the chicken or the egg? Headend first, then STB, or do both together - big bang?
• How to solve complex systems integration/test in DTV?
• What is end-to-end testing, what areas should you focus on?
• What is required from a Systems ATP?
• What kind of QA team structure is generally accepted practice? How does Agile fit in with this?
• Can E2E Testing be executed using Agile - iterate on features End-to-End?

Side note --- end ---]

Digital TV Summit Day Three...

The summit ended today with a market analysis on the opportunities for PayTV in Africa, covering most of aspects of the end-to-end value chain, from content procurement to delivery and consumption. I found today's session especially interesting and refreshing, a great complement to the book I am currently reading, The Business of Media Distribution: Monetizing Film, TV and Video Content in an Online World, providing an overview into what the business of PayTV actually entails. We learnt from past experiences of people who've worked in several countries in launching PayTV operations in-country, had an overview of a core component of the system (i.e. Content Management System), touched on Next-Generation platforms, and culminated in a demo on a low-cost (sub $100), but advanced Set-Top-Box, exploring the opportunities that await the African market.

My notes cover the following:

• Day 2: Case Study Namibia Spectrum Allocation
• Overview of Africa Markets
• Lessons from Past (failed) African Projects
• Content & Operational Considerations
• Overview of Media Asset Management Systems
• Next Generation Platforms
  
• Overview of Monitoring Systems
  

Digital TV Summit Day Two...

Here's my write up from Day 2 of the SA Digital TV Summit held in Bryanston, Johannesburg. The second day's Agenda went much deeper that the first day, although there was a no-show from the SABC on expanding the topic of Digital Migration. Nevertheless, I learnt quite a bit today as well as refreshing some other topics I don't get to focus on in my day-to-day work. We covered the following areas:

• SA DTT STB Subsidy Rollout Plan
• The Connected TV - Research Feedback
• Insights from Discovery Network
• Update on Namibia's DTT Migration
• Impromptu Discussion on DTT STB Readiness
• Content Delivery Networks Overview

Digital TV Summit Day One...

The inaugural Digital TV Summit of South Africa takes place this week in Bryanston, Johannesburg. I registered almost immediately on receiving the invite back in 2012, since one of the key speakers was to be Collins Khumalo, CEO of Multichoice South Africa. I was looking forward to seeing Collins in action, as my way of assessing how Multichoice contributes to the wider cause of Digital TV in South Africa. Unfortunately, the agenda was changed last minute, some key speakers, including Collins couldn't make it. I was also interested in networking to get an idea of the landscape of this market in South Africa/Africa...

Anyway, I am still attending the conference and will share what I've learnt from the presentations, on this blog - starting with Day One, which gently introduced the conference as being biased towards the topic of Digital Migration / DTT (Digital Terrestrial Transmission) as being one of the biggest event, if not, the biggest event to happen to the TV Broadcast Industry since its inception back in 1974...

Day One covered the following (Click on the links to jump to the topic):

• Digital broadcasting trends in Sub-Saharan Africa
• DTT SwitchOver Discussion
• Looking at Internet Protocol TV (IPTV) and its future in Africa
• Sentech's DTT Infrastructure Readiness Update