Recently, the Quality of the IT services were frequently talked about. Users were yelling about how bad the services are. Crisis showed up everyday. People were running around with the same word in their mouth, “Quality”. A magic word. But how to get it? Let’s take a look at the ITIL way.

What’s Quality?

“Quality is the degree to which a set of inherent characteristics fulfils requirements”  – From ISO 9000

“Service Quality is about ensuring customers get what they want” – From Managing Service Quality

We tend to think quality is just the number of defects and production issues. That’s a misleading thinking because it didn’t take in consideration the “customer” factor.

Actually, “Quality” is an interactive term. It’s all about satisfying and exceeding Customer Expectations. Customer is in the center of the Quality Universe.

Here are some common mistakes we made in the history:

  • We deliver the highest quality software in the industry because we use the latest and coolest technologies!
  • Customers don’t know what they want. We know better. We should make decision for them.
  • They already gave us the functional specs. That’s all we need to deliver what they want.

In our past experience, those usually backfired immediately when the software was delivered to Production environment.

The coolest technology may not be the easiest to use and may cost too much to build or may not be stable enough. Our customer may not like the decision we made for them. They are the one using the software, not us. The old functional specs may go out of date immediately after they are written. What we delivered is not useful anymore.

Just imagine that we go to a restaurant, the waitress doesn’t even ask what we want and just throw us some dishes that uses the most costly and coolest materials, or some dishes she thinks we will like, or the same dishes we ordered two years ago. What will we think? Is she out of her mind?

We have been constantly doing that for years. When customer complained, we felt they were like idiots and were even more convinced they don’t know what a “high quality” product is.

If we seriously want to improve the quality of our services, we need to start with listening to our customer.

Manage Customer Expectations

Let’s put ourselves into our customer’s shoes. How do our customers evaluate the service quality? We can again recall our experience with our favorite restaurant and how we become the frequent visitors.

  • When we first visit the restaurant, we found the food and all the services (parking, friendliness of the waiter, music, atmosphere, look and taste of the food, correct bill, etc.) all exceed our expectations. We were so satisfied and decided to come back.
  • When we came back the second time, we got the same level of services. Everything was great. We fell into love and came back again and again.
  • After we became frequent visitors, the price of the dishes didn’t go up for us. Instead, we even got frequent-dinner discounts. Well, the prices sometime went up because everything else went up, we understood.

That’s exactly what our customer expected from us:

  • Can the service we provided satisfy their expectations?
  • Can they get the same service time and time again?
  • Is the service provided by a reasonable cost?

We can further translate those into IT terms.

For Software Development Services, they expect us:

  • Deliver on schedule, as we promised
  • Deliver all the functions specified in the requirements, correctly
  • Deliver the project under the budget

However, we are not only delivering software to our customers. We are actually delivering “Business Application Services” to them. They not only expect the software works, but also they expect:

  • The application is available at the time needed
  • The performance of the application is good enough to support the business
  • The application will act normally even at the busiest hours.
  • The application can follow up with the growth of the business. No matter how fast we add clients, the application should act normally.
  • In case of disaster, the application service will be resumed as soon as possible with no data lost
  • When some issues happen, they will be addressed in a reasonable period of time with the satisfaction of the customer.
  • The same issue won’t come again and again. That’s very annoying.
  • The maintenance cost for the application is reasonable.

It’s very easy to simply ignore some or all of those non-functional aspects of the quality. We are happy and proud when we deliver all the functions on time with “no defect”. But very often the celebration party was interrupted with annoying issues falling into those categories like performance, availability and capacity. If we didn’t consider them in our design and implementation, we were doomed to have angry and unsatisfied customers, no matter how many new functions we put in and how hard we worked overtime to guarantee ontime delivery.


If you cannot measure it, you cannot manage it
If you cannot measure it, you cannot improve it
If you cannot measure it, you probably don’t care about it
If you cannot influence it, you don’t need to measure it

We now know that Quality is all about Customer Expectations and we know that besides the correctness of the functionalities, the non-functional aspects are as important to customer satisfaction. But, how do we improve the customer satisfaction (a.k.a the Quality)? The first thing we desperately need is Metrics, the quantitative measurements.

Here is an example. Say, we decided to improve the Availability of one of our mission-critical application service. How do we know the current level of Availability, the ideal availability and if we are improving it or ruining it? Instead of relying on the “feeling” of our customer (or ourselves), we need some numbers on the paper to speak for itself. We either see the Availability number goes up or the Unavailability number goes down week after week, month after month before we (and our customers) can be sure “it’s improving“. But, what number?

We need a mathematic model to define Availability or Unavailability. Here is the one most commonly used by the industry.

Small Availability

We define the “System Incident Interval” as the time period between two consequent incidents that make the system unavailable. It can be further broken down into “Down time (fix time)” and “Normal Operation Time”.

We can define the following metrics to measure the system availability:

  • MTTR (Mean Time to Repair): Average Down Time including Detect Time and Resolve Time.
  • MTBF (Mean Time Between Failures): Average Normal Operation Time
  • MTBSI (Mean Time Between System Incidents): Average time between two incidents
  • Availability Ratio: MTBF/MTBSI*100%

If we keep tracking those metrics in a regular basis, we will have a much better understanding of issues that are affecting the availability of our application services.

We will know if the nature of our issues is small ones that happen frequently and can be fixed quickly or big one that rarely happens but once it happened, it takes a long time to fix. Based on that nature, we can implement different preventative strategies.

Best of all, having those numbers around help set the correct impression. It’s not about “feelings” anymore. We will have some common ground to stand while communicating to our customer and have something concrete to work on.


An important factor to the Quality (remember, it’s all about customer impression) is consistency. If the service level varies significantly from time to time, it’s hard to construct trust and confidence in the customer.

For example, you went to the restaurant at Monday and you were served by Nicky. It was a beautiful experience and you totally enjoyed it. When you went back at Wednesday with higher expectation, unfortunately Nicky was on vacation. You were served by Bill. It was totally different and much worse experience. You left so angrily. Would you go back at Friday?

The same concept can be applied to IT services. Delivering the extremely high level service once, followed by several below-average service, will badly hurt your relationship with the customer. You can only earn the trust of your customer by constantly delivering high quality services.

Now, the question is how to maintain the consistency of your service level. How can you make the service level consistent even the service is delivered by totally different people, no matter it’s Nicky or Bill or Laura?

The key is process.

“A process is a specific ordering of work activities across time and space, with a beginning and an end, and clearly defined inputs and outputs: a structure for action. … Taking a process approach implies adopting the customer’s point of view. Processes are the structure by which an organization does what is necessary to produce value for its customers.”

– Thomas Davenport, “Process Innovation”

Only when the process is clearly defined and strictly followed, the consistent results can be expected. This is especially true for Services because Services are the one time interaction with the customer. In most of the time, we don’t have the time and opportunity to rollback or redo the Service.

By following a predefined and well-tuned process, the “human” factor can be greatly reduced. Both the customer and us, the service provider, will have more confidence in the quality.

Just by clearly defining and documenting the process, we open the door to improve the process and the quality of the outcome. Without a defined process, it’s hard to even repeat the service delivered last time, not to mention improving it.

Having said that, we shouldn’t underestimate the influence of “human” factors. After all, any good process has to be carried out by human. Familiarities, morale and skill level will make big difference.


Quality is all about customer expectations. We must consistently delivery high quality services that exceed customer expectations. To do that, we need to quantitatively measure the key criteria of the service, taking into consideration of non-functional criteria, and implement the right process to guarantee the consistency.