Tuesday, February 26, 2013

Blu-ray - a real improvement ?

Blu-Ray could be the last physical video media before everything goes on line, and has been seen as a major improvement over the previous generation, the DVD. But is it a real improvement and did the designers really do a good job ? My answer is a big NO if you look at it from a quality perspective.

The first thing to take into account when developing a product is to collect the proper requirements (= the "voice of the customer" in quality terminology). It seems that this phase has not been done properly if at all.

What are the key requirements and the improvements expected over the DVD generation ? and what are the disturbing factors having a zero added value as far as the customer is concerned ?

- improved image quality, HD resolution : good job
- improved sound quality, more possibilities : good job
- more reliable media (scratch resistance, life time) : good job
BUT
- general performance of the system (reaction time, startup time) : ridiculously worse (a HUGE problem has been created where there was not problem: what would people say if after they start their engine, they cannot move the car unless they wait for five minutes)
- usability : an incredible step backwards. A complete nonsense user interface with an incredible number of manipulations instead of a single button press on the first DVD remote controllers.
- still a not so compatible system (very few PCs have a BD player, many cars have DVD players, no BD players, Apple does not support the format).
- regional codes: the distributors still don't understand anything about this market. This is an open insult to the user. You can legally buy a disc in a country, move to another (there are more and more world travellers) and you can throw away everything you bought. They just created another reason to STIMULATE piracy.
- languages: while at the beginning of the DVD, you had 8 spoken languages and 16 subtitles on a disc, nowadays you can be lucky if you find the right subtitles and the right language in a country far from the country where you live. This is really becoming ridiculous in Europe. If I live in Belgium, and want the original version (in english), english subtitles and the possibility to have french audio or dutch subtitles (the languages you would expect in Belgium) I often need to buy the discs in Germany, with the box in German.  This is true for DVD and BD by the way. This is in contradiction with the PRINCIPLE of DVD and BD, providing multilingual possibilities. And it is a nonsense for the distributors, who have to manage a huge variety of unnecessary versions while one single version for all of Europe would be easy.
- most discs do not allow to restart from a saved position. You start watching a movie. You stop. You want to see the rest the day after, NO WAY. You must restart and skip to the place where you left (if you remember it). A major step backwards again.

I don't know if I am the only one to think that way, but to me the BD, while being a reasonable commercial success (I personally like the pictures and I have a lot of BDs), is among the major engineering failures of the last twenty years: an extremely badly designed system, probably introduced by incompetent organizations in a hurry.

Friday, February 22, 2013

Testing Maturity Grid Concept


The testing maturity grid concept.

This is a concept I first encountered doing product development and that I found very useful to determine if a product development was ready for release to the market. The same principles or similar principles can be applied specifically to software development, whatever the type of development is. This can be typical IT application development, or embedded software or anything else.

The basic principle is to display the number of  problems/issues/bugs in a project in a grid form. The "look" of the grid will tell you in a very simple way if you are ready with your project.

The grid has to axes, the first one being the step in the bug workflow and the other dimension is the severity of  the bug.

The different typical categories are:

Bug workflow (inspired by JIRA, one of the widely used issue tracking systems):
- Opened
Meaning the bug has been entered by a beta user, a developer, a tester. The bug has not yet been analyzed or allocated (can evolve to in progress status, or be rejected, counted as duplicate, accepted, deferred or non reproducible)
- In progress
After first analysis, the bug has been allocated and is currently being solved but the final solution has not been implemented yet
- Resolved
The bug has been solved by the person it has been allocated to, it has been typically verified by the solver
- Closed
The solution has been validated by an independent tester or product owner
- Other end state: those are states which are equivalent to a closed state from a bug tracking point of view, in this category (they can be recorded separately), you can find states like: rejected (not an issue), duplicate (already recorded), non reproducible (not an issue ?), accepted (we/or the customers don't care), deferred/delayed (optional for a new release/version of the application - irrelevant to this instantiation of this project)

Bug severity
- Safety/security issues
The product cannot be used by actual users, in most cases even for testing purposes. For embedded software, it can imply safety issues up to life threatening issues (e.g. brakes not working in a car), for IT software, it can be related to major security issues (e.g. gaining access to strictly confidential information, legal consequences...)
- Unsellable
A required functionality is not available or not functioning properly making the usage of (part of) the delivered product impossible (e.g. a banking application where payments don't work).
- Major
A problem a normal user would find critical or even unacceptable but not making the application completely unusable (e.g. the software sometimes hangs but can be restarted)
- Minor
Even a critical user might tolerate the issue, it is in no way blocking to use the application or the product (e.g. a screen layout is not exactly as it should)

The maturity criteria also depend on the project phase
- Inception
- Elaboration
- Realization
- Transition

The grid

Severity->
Status
Safety
Security
Unsellable
Major
Minor
Opened
10
50
75
In progress
45
15
10
Resolved
20
10
5
Closed
2
10
10
5
Other
End state
1
20
5
15

If first limited prototypes are available to show a limited functionality, they cannot even show safety or security issues. Nothing can be released even to developers or internal project testing with bugs in the red area. The orange area can be acceptable during the realization phase, all types of issues  but the safety/security ones can be discovered and addressed.
After the realization phase, the product/application is being released to the final testing group (could be system testing, functional testing, beta testing), typically in a so-called acceptance environment. In that case problems can still be in the yellow zone. When the product is released to actual users/customers, typically in a so-called production environment, only the green area can be accepted.

The numbers indicated in the grid are the number of bugs/issues in a defined category/state, the example shown gives a status during the implementation (realization) phase.

Other information

The above is a specific interpretation of the concept: each organization can determine for itself the different steps in the workflow, the severity range and also the release and quality criteria for any phase of a project. Criteria might also depend on the domain area (medical devices, car industry, telecommunications, IT, public institutions...)

Another example of the same ideas can be found here: