QA Manager Job Spec

As a Software & Systems Engineering Management Consultant, I spend a lot of my time helping senior management sort out their department processes, run audits around international benchmarking, teach and coach topics that I'm quite familiar with, help with crafting roles & responsibilities for project & department structures, help set up implementation strategies, and remain on board to see these recommendations & changes through to completion...

Recently, I helped a client rationalise its Quality Assurance (QA) Engineering organisation. I have written about QA/Testing in the past, most of those posts revolve around the experiences from this particular client. You can check out these two resources for my background & knowledge of the Quality Engineering QA/QC disciplines:

• All QA blog posts on this site
• My most read QA white paper 

One of the triggers that drove this client to rethink its QA structure was some work that I did last year, around benchmarking what other organisations do in similar projects, and the outcome from that investigation identified an overload in the QA space, with lots of teams doing similar activities, resulting in duplication. The ratio of QA-to-Developers seemed unnaturally high. I personally knew this from the very beginning, but had to learn the hard way of taking a company through its own journey of self-discovery, that whilst an expert might see flaws and holes on day one, the customer is usually so set in its ways-of-working, that they are blinded by the inefficiencies because the projects do deliver anyway.

Anyway, I helped identify a need for a QA Manager, and helped create a job specification around this - which is what I'd like to share with you, in case you are looking for something similar. In my view, this is still a journey, the QA manager maybe an interim step, as an improvement to embedding some quality disciplines within the organisation, my end goal however, is to lead this client to a Lean Mindset, which takes the best of Agile/Lean processes, and drives quality engineering principles & disciplines right up the value chain, to the source - which is around architecture, design & development...we're not yet ready for that yet, so a QA Manager intervention is as good-a-starting-point-as-any....

P.S. My source of info for drawing up this job spec was the IEEE SWEBOK (I once studied the IEEE CSDP although didn't want to write the exam because I'm currently against any form of certifications, although I'm still a member of IEEE & ACM) as well as my own previous job experience...

 

JOB SPECIFICATION:: QUALITY ASSURANCE MANAGER

ENGINEERING: DECODER QUALITY

 

CONTEXT

[THE COMPANY] has rationalized its various Testing roles in the department and has introduced a new role focused on driving Software Quality Assurance Practice across the business. The reporting department line (CEQA) is overall responsible for applying Software Quality Management (SQM) that applies to all perspectives of decoder software processes, products and resources. SQM defines processes, process owners, and requirements for those processes, measurements of the process and its outputs, and feedback channels.

 

SQM processes consist of many activities. Specifically, with respect to [THE COMPANY], this is broken down into the following areas:

• Software / System Quality Assurance
• Software & Systems Testing (Quality Control / QC)
• Automation Tools & Methods
• Real-world home testing (Field trials)

    

Collectively, the above areas are separate lines reporting to an overall Head: Quality. These SQM streams however are interconnected and share the underlying foundation of quality principles, which all these groups need to adhere.

This role is about a Quality Assurance Manager position, which is based on three pillars:

• Measurement & Metrics
• Standards, Reviews & Audits
• Project-based role as QA Program Manager / QA Director

SOFTWARE QUALITY ASSURANCE (SQA) VISION

SQA processes provide assurance that the software products and processes in the project life cycle conform to their specified requirements by planning, enacting, and performing a set of activities to provide adequate confidence that quality is being built into the software at all levels of the software development lifecycle (SDLC).

The role of SQA with respect to process is to ensure that planned processes are appropriate and later implemented according to plan, and that relevant measurement processes are provided to the organization.

The SQA plan defines the means that will be used to ensure that software developed for a specific product satisfies the user's requirements and is of the highest quality possible within project & business constraints.

SQA takes into account the culture of the development teams, identifies processes, standards and practices, and conventions governing the project, highlighting how they will be checked and monitored to ensure adequacy, compliance, and ultimately the decision that the product is fit-for-purpose from a QA standpoint (checks and balances).

SQA also identifies measures, statistical techniques, quality metrics including procedures for problem reporting and corrective action, techniques & methods including tools, training, reporting & relevant documentation.

SQA also contributes to defining acceptance criteria, as well as reporting and management activities which are critical to software quality.

SQA will also contain a review and audit facility that may involve, but not be limited to the following areas: Management reviews, Technical reviews, Inspections, Walk-throughs & Audits. This can only be implemented once the department has reached a level of maturity and stability of processes (i.e. standards adherence) that have proven to deliver business value, such that it becomes the charter & practice of all projects going forward, and hence warrants a review & audit activity.

WHAT ARE WE LOOKING FOR IN A QUALITY ASSURANCE MANAGER?

I was once an ESTJ

I am continuing to dig through my records of psychometric & personality test reports, this one goes back to when I was a Software Engineer of three years, I'd just lost my job through redundancy, Ireland was going through some pretty bad times, and I was faced with a lot of uncertainty: do I return back home to South Africa, find a job in Ireland, or try my hand in the UK?? 

As part of the redundancy package, included a one week intense course on Career Planning, CV/Resume writing, Video Interviews and Personality testing. At the time, this would have been my second experience with personality tests, the first was right after university, where I showed all signs of an analytical, thinking, careful engineer. In the two years since, I had left my home country, settled in a brand new world, got to meet new cultures, and see a different side of software engineering - that it changed my outlook of life and work quite a bit.

I strongly feel that a person is on a path to discovery, despite being born with certain traits, that life and experiences will change a person's outlook (although we're told that our real core tenets are what we born with, and fall back to these by instinct or default), thus I have some reservations with these personality tests, because I just don't like being placed in a box. I prefer to create my own path, and adapt to the situations as needed. However, these tests are interesting, and useful to trigger one to reflect...

So this goes back to 2002, thirteen years ago, when I did the Myers-Briggs profile questionnaire, here's the results of that time:

In 2002, I showed signs of ESTJ

So looking back almost thirteen years ago, it's interesting to note that I remained a Software Engineer for another four years, before breaking out into leadership roles as development project manager, senior project manager, and more recently senior program & technical manager.

However, I am no longer that traditional and conservative, as I'm constantly learning and adapting my processes, style and working tool-set, although I do sometimes fall-back to my own traditions that are known to work, deliver value...

I am still quite the organisational, control freak...everything has it's place. I'm so glad that it's actually a Lean discipline to keep the workspace organised, clean & efficient :-)

Culture: Working with the Israelis

I'm in the process of rounding up all my reports on my self from various psychometric tools & frameworks that I had to use in the course of my professional life, and share them openly on this blog. I did make a start some time ago with these two posts from a few years ago:

• BBC Interview Psychometric Report
• Multichoice Interview using CPP (report still to share)

This week, I experienced another framework called the "Enneagram" which I also will aim to share on this blog.

As I dug through my old records, I found one particular course that brought back some fond memories: Working with the Israelis :-) 

Working with Israelis, a One Pager Cheat Sheet

So I worked with a lot of Israelis, all levels: engineers-to-directors, and it was quite exciting and a thrill, personally & professionally. I really didn't let any personal/religious biases get in the way of working professionally, and actually developed some really good relationships that I'm happy to call them my friends...where there is code, there are no barriers....all you need is code :-)

So if you ever confronted with working with an Israeli, here is a handy table to help get you to understanding the experience:

Cheat Sheet

Cross-Cultural Awareness is becoming increasingly important in a Connected World

In the years 2003-2011, I worked with a company that was truly global. We had software teams all around the world: North America, Canada, United Kingdom, Paris, Denmark, Australia, Israel, India, Korea & China. The company went on a global training drive to address the issue of working with the various cultures. The company partnered with the folks from TMC CulturalNavigator,  an online system that we could use to learn about the people from all parts of the world, however, specifically tailored for the regions we operated in. 

Each person in the company had to complete the online questionnaire (like all these psychometric questions) to gauge the type of person you are. The results were mostly online for people to access, as it was useful information to have to hand especially when you're meeting a colleague for the first time, or have a high-profile meeting to attend and you want to get a sense of the people, etc, etc.

I applied the tools quite often, found it extremely useful and handy to have around...

In addition to the online tool, we had coaches come in and talk around this system, which is called Cultural Orientations Index - I will write about my own results from 2007 in a follow-up post, the COI contains the following dimensions:

I will expand on this in follow-up posts...

Street vendors and Talent houses

WARNING: This is probably one of those posts that just might not work, but anyway, this has been playing on my mind today that I had to write about it, so here goes!

The sell

If you're living in South Africa, or you spent some time here, you would have definitely come across our resourceful street vendors that hang around at major intersection traffic lights (robots), offering some really tempting fruits on sale. It is convenient, on-the-go, and price-competitive, reasonably safe, and most of the time, the stuff on sale looks really good. So to save yourself some time from stopping at a green grocer, you grab what's on offer, haggle a bit, pay and drive off. Seller and buyer are happy.

Sometimes though, and this is something that's becoming increasingly popular, is your street guy propping up his boxes, making it seem you're getting a good deal, the box has got the best samples on top, and tucked behind those good ones, are the not-so-good, slightly older fruit. Sometimes, the box has been pushed from the bottom to give the perception of volume, but sadly you're getting much less than you expected. In some bad cases, you get deceived by finding some really old rotting stuff (not cool)...

Hidden gems

A thought struck me today, that this is similar to the shows I've seen from software service providers, especially the ones that outsource, off-shore resources. They put on a good show, showcase their best talent at the initial meet-and-greet, CV/Resumes ripe for the taking - you sign the contract, agree on a medium-to-long-term support contract, thinking you're not only getting value-for-money, but also getting ripe fruit for the taking!

Alas, after a few months into the engagement, you realise that the good fruit was just a show, that the wider team is a box of dull tasting fruit, that you were sold more packaging than what you bargained for. Bodies are thrown at your project, giving the perception of a strong workforce, but you're unable to see any real value, delivery happens at snail-pace. Just what kind of fruit did you end up buying?? 

You're left with a sour taste in your mouth, you make a mental note not to be fooled by the alluring front of such street vendors. You will do your homework next time, ask the right questions, and continuously monitor the progress and commitment that your outsourced vendor had initially promised.

Too often, development managers are faced with just seeing the contract through, working with a mediocre team, fronted by a good tech lead if you're lucky...and kicking yourself for not taking extra time out to seek out to the rightfully established reputable greengrocer, who takes the time to source just the right quality fresh produce you really wanted, even if you have to pay a premium or wait in long queues!

Be ware of up-and-coming Talent houses. Take time to develop the relationships, and make sure you look past those lovely shining front-facing, mouth watering fruit....

As I've become my own independent consultancy, I am always mindful of overselling my expertise, mindful about putting on a show & being seen for a fraud, and take pains to delight my clients with work (and talent) that I can be proud to associate my brand with...

Non Functional Requirements and Sluggishness

There is a lot of talk about whether writing code, or creating software, is really an art, a craft, or is it a highly disciplined, software engineering discipline. This post is not about this debate, although I do wish to make my stance that great software is just as much it's a craft as is a software / systems engineering discipline. One should not discount the analysis, rigour that should go into the design of any software system, be it relatively simple 4th/5th tier high level applications built on very easy frameworks (e.g. WebApps, Smartphone & Tablet frameworks), or very low-level, core, native software components (e.g. OS kernel, drivers, engine services).

Take the Set Top Box (STB) for example, a consumer device that is usually the central and focal point of the living room, people expect the thing to just work, to perform as well as any gadget in the household, being predictable, reliable and always delivering a decent user experience.

How then does one go about delivering performance in a STB software stack? Is it the job of the PayTV operator to specify in detail the NFR (Non-Functional Requirements)? Or is it the job of the software vendors (Middleware / Drivers) to have their own internal product performance specifications and benchmark their components against their competitors? Does the PayTV Operator benchmark their vendors based on the performance of their legacy decoders in the market, for example: this new product must be faster than all infield decoders by a factor of 20%. Or is their a collaboration between the technical owners (i.e. architects) to reach mutually agreed targets?

Take the problem of Sluggishness. According to good old Google,

sluggish

ˈslʌɡɪʃ/

adjective

1. slow-moving or inactive.

   "a sluggish stream"

   synonyms:

   inactive, quiet, slow, slow-moving, slack, flat, depressed, stagnant,static "the sluggish global economy"

   • lacking energy or alertness.

     "Alex woke late feeling tired and sluggish"

     synonyms:

     lethargic, listless, lacking in energy, unenergetic, lifeless, inert,inactive, slow, torpid, dull, languid, apathetic, passive, unresponsive,weary, tired, fatigued, sleepy, half asleep, drowsy, heavy-eyed,enervated, somnolent
   • slow to respond or make progress.

     "the car had been sluggish all morning"

When we're field testing STBs, over long periods of time (or sometimes after not-so-long-usage), people & customers report:

• The box got very sluggish, needed to reboot
• The STB isn't responsive to remote presses, takes 4-5 seconds to react 
• Search takes too long and I can't do anything else but wait for search to complete
• Channel change is slow compared to my other decoder
• When using the program guide or navigating through the menus, the box gets stuck for a few seconds as if it's processing some other background activity - it's very irritating to my experience

This feedback, whilst extremely useful to the product team, is very hard for engineers to characterise without having access to system logs, to trace through the events that led up to the slowdown in the performance that resulted in degraded user experience.

In my view, based on my own experience of writing STB applications and managing complex systems projects, I believe that unless we quantify in fair amount of detail what sluggishness means, then it's a cop out for passing on the buck to other parties: either the product owners didn't take time to functionally spec out the requirements, or the application is using API services that it doesn't have control of...

In the remaining part of this post, I will touch on an example of a previous project of how we handled the problem of sluggishness, and this was through a process of rigorous Non-Functional Requirements focused on Performance Specifications...