Rothstein Associates Inc.

QOS:
MEASUREMENT AND EVALUATION OF TELECOMMUNICATIONS QUALITY OF
SERVICE
by William C. Hardy

“If you need to get to grips with QoS, this is the place to start.

“Quality of Service (QoS) is continuously growing in importance in the telecommunications
industry because competition is growing fiercer by the day. By drawing on 30 years of
experience, William C. Hardy explains how to examine specific tools and techniques that he
has developed for the measurement and evaluation of QoS and understand the underlying
analysis perspectives and methodologies.

- Details the basic concepts of QoS, together with the methodologies for
organizing,
structuring, and carrying out analyses of QoS from scratch.
- Describes the attributes of the telecommunications service that determine user
perception of quality in non-technical terms.
- Discusses specific measures, measurement techniques and evaluation criteria
for
all
of the factors that affect user perception of QoS.
- Addresses user concerns including:
- Will I be able to get to the service when I want to use it?
- How long does it take before I know a connection is being set up?
- How good will voice sound over a connection? Includes valuable tips for QoS
analysis and the perspectives vital for describing QoS in ways that are useful and
operationally meaningful.

“Whether you have a limited technical background or are a telecommunications professional
this simple and straightforward approach will be an essential tool to understanding QoS.”

- - - - - - - -
-

“...provides a straightforward and very accessible approach to measurement and evaluation
of
QoS in telecommunications networks strongly recommended for all people, either
experienced professionals or graduates, involved in the area of networking...” - IEEE
Communications Magazine, February 2002.

- - - - - - - -
-

EXCERPT FROM THE INTRODUCTION

“The purpose of this book is to define and describe a family of measures of quality of
telecommunications services that have been demonstrated in their successful application
over many years to be useful both to telecommunications service users, as a basis for
understanding and assessing possible differences between competing services, and to
service providers, as a means of determining what improvements in service performance
are
needed to assure customer satisfaction. The distinguishing characteristic of these measures
is that they have in every instance been designed to simultaneously achieve two ends:
1. The credible, reliable assessment of the likelihood that users will find a particular
service to be satisfactory; and
2. The determination of how system performance must be changed when that
assessment shows that users are not likely to be satisfied.

“This kind of complementary utility in a measurement scheme is not hard to achieve.
However, it is, in fact, frequently absent in proposed quality of service (QoS) metrics,
because definition and development of particular measures have failed to take into account
both the concerns of the users of telecommunications services and the perspectives of the
engineers and technicians who must design, build, and operate the systems that deliver
those services. It is, therefore, a secondary, but equally important objective of this book to
describe the analytical perspectives and discipline that have reliably guided the development
of the specific measures that are presented here.

“To this end, the material in this book is divided into two parts:

- Part I presents the concepts and perspectives that have guided the development
of
the measures. This section first presents what might be thought of as a theory of
measurement. It begins with an examination of the possible reasons for developing
measures
and proceeds with a formal description of the process by which the measures discussed
here were developed. This part of the book also contains a chapter that briefly defines and
describes basic telecommunications functions and the processes by which those functions
are used to deliver telecommunications services.

- Part II then discusses a complete family of measures of QoS of
telecommunications
services, keyed to the user concerns and different types of telecommunications services
defined in Part I.

“Under this organization of the material, then, Part II comprises the source material that can
be researched for specific measures and applications, while Part I comprises both the
background necessary to follow the development of the particular measures, and the "how
to"
manual for those who may be called upon to develop measures of QoS for new services or
new ways of delivering services.

“This structure allows for a variety of approaches to the material.

“Persons who are conversant with telecommunications services and QoS measurement may
choose to begin with Part II, and then revert to Part I for purposes of understanding the
perspectives that supported development of the measures. Alternatively, a seasoned QoS
analyst might read through Part I and readily acquire an understanding of the analytical
discipline and techniques sufficient for purposes of developing measures for new services
that
are useful both to service users and to telecommunications system operators and engineers.
Finally, persons with lesser background and experience in QoS will find that reading Part I
first to get the grounding in the basics will make it much easier to follow the reasoning that
justifies the selection of the measures described in Part II as being particularly well-suited for
purposes of measuring and analyzing the particular aspect of QoS each describes.

“Whatever the background and experience of the reader, I hope that this book shall clearly
convey, both by force of reasoning and by example, three principles to be applied in defining
and developing measures of QoS:

1. Meaningful measurement of quality of a telecommunications service must begin
with
a consideration of the concerns of the users of that service to develop a set of evaluative
concepts that will guide the definition of measures and measurement schemes,
2 Useful measurement of QoS must be based on measures that can be readily
interpreted by users, but are also clearly related to the performance characteristics of the
systems that deliver the service, and
3. Cost-effective measurement of QoS can be realized only when the means of
quantifying or estimating any measure is consciously selected on the basis of consideration
of both the intended use of the measure and readily available sources of data.”

- - - - - - - -
-

EXCERPT FROM THE FOREWORD

“My involvement in analysis of quality of telecommunications services began almost by
accident in June, 1967, when I started my first full-time job out of graduate school. The job
was with the Operations Evaluation Group of the Center for Naval Analyses. It seems that
what they happened to need the day I reported was someone to fill a slot as a
communications analyst. Since I was there, I was anointed, never mind that I knew
absolutely nothing about telecommunications systems, electrical engineering, or even
electricity, since I had skipped that part of the college physics curriculum, and almost
nothing of my graduate education in mathematics was relevant to understanding Navy
tactical voice and teletype communications over radio frequency channels.

“Because my career started with such a complete lack of practical experience and technical
skills, my analytical efforts have never been marred or impeded by technical expertise or
conventional wisdom. Rather, what I discovered was that all I really needed to do to be
effective as a problem solver in this area was to:
- Imagine myself using the system I was studying;
- Decide what I would be concerned about if I were using it;
- Research the technology of the system to the extent necessary to understand the
mechanisms affecting performance of the system with respect to those concerns; and
- Formalize the relationships between system performance and user perception of
quality of service gleaned from this drill.

“When I did this, everything else needed to solve the problem would readily follow - the user
view would suggest concerns; concerns would suggest measures of quality and
effectiveness; understanding of the mechanisms would suggest measures of performance
and their relationship to measures of quality; measures would suggest quantifiers; quantifiers
would suggest data requirements; and so on, all the way down the analytical chain.
“This book is based on more than 30 years experience in successfully applying this
approach in analyzing issues of quality of service of telecommunications systems to produce
practicable solutions to quality problems. Because of the very basic nature of the approach,
this book is apt to be viewed by some as being short on technical content and long on
formulation of evaluative concepts and generic measures. However, I refuse to apologize for
this, because the perspectives on quality of telecommunications services that I am trying to
lay out here are exactly those that I would want all of my employees to share, were I ever to
become the CEO of a telecommunications company, so that, for example:

- My marketing and sales forces would know how to communicate with customers
in
a way that would demonstrate their understanding of customers' concerns;
- My system engineers would know how to design my networks to satisfy customer
expectations, rather than simply meet industry design standards; • My operations managers
would know the comfortable levels of performance affecting quality of services that must be
achieved and maintained to assure user satisfaction;
- My service technicians would know how to troubleshoot user complaints with the
same competence that they identify, diagnose, and correct technical problems; and
- Everyone involved anywhere in the company would have a very good idea of
exactly
how their day-to-day activities affect user perception of the quality of our services.

“To this end, what I have tried to present here is a treatise on the ways and means of
measuring and evaluating telecommunications services that is simple and straightforward
enough to be appreciated by anyone, but sophisticated enough to be informative and useful
to telecommunications professionals. The only way you can judge whether I have succeeded
is to turn the page...”
- William C. Hardy WorldCom, USA

- - - - - - - -
-

CONTENTS

PREFACE
FOREWORD
INTRODUCTION
PART I BASIC CONCEPTS
1 DEFINITIONS
1.1 Quality of Service
1.1.1 Intrinsic vs. Perceived Quality of Service
1.1.2 Perceived vs. Assessed Quality of Service

2 MEASUREMENT AND EVALUATION
2.1 Function of Measurement and Evaluation
2.1.1 Audience and Utility
2.2 More Definitions
2.2.1 Data vs. Information
2.2.2 Measures vs. Quantifiers
2.2.3 Concerns
2.2.4 Objectives

3 THE ANALYSIS PROCESS
3.1 Phase 1: Formulation
3.1.1 Identify the Audience
3.1.2 Determine Decision-Making Responsibilities
3.1.3 Specify Analysis Objectives
3.1.4 Identify Concerns
3.1.5 Define Measures
3.1.6 Select Quantifiers
3.1.7 Example
3.2 Phase 2: Data Handling
3.2.1 Data Acquisition
3.2.2 All the Statistics You Need to Know to Read this Book
3.2.3 Data Organization
3.2.4 Data Manipulation
3.3 Phase 3: Evaluation

4 TELECOMMUNICATIONS CONCEPTS
4.1 Basic Systems and Processes
4.1.1 Injection/Extraction
4.1.2 Encoding
4.1.3 Routing
4.1.4 Signaling Systems
4.1.5 Switching Systems
4.1.6 Types of Service
4.1.7 Types of Connections
4.1.8 Set Up
4.1.9 Billing Method
4.2 Basic User Concerns with Service Quality
4.3 Preview

PART II EVALUATIVE CONCEPTS, MEASURES, AND QUANTIFIERS

5 OVERVIEW

6 ACCESSIBILITY
6.1 Evaluative Concepts
6.1.1 Examples
6.1.2 Variations with Type of Service
6.2 Intermittently Used Services
6.2.1 Concerns
6.2.2 Generic Measure
6.2.3 Quantifier
6.2.4 Availability vs. Accessibility
6.2.5 Adjustments of OCC Axis Values using Delta[SO,SIlty]
6.2.6 Evaluation
6.3 Continuously Used Services
6.3.1 Concerns
6.3.2 Measure
6.3.3 Quantifiers
6.3.4 Evaluation

7 ROUTING SPEED
7.1 Evaluative Concepts
7.2 Circuit-Switched Services
7.2.1 Concerns
7.2.2 Measure
7.2.3 Quantifiers
7.2.4 Evaluation
7.3 Packet-Switched Services
7.3.1 Concerns
7.3.2 Measures
7.3.3 Quantifiers
7.4 A Note on Data Acquisition

8 CONNECTION RELIABILITY
8.1 Evaluative Concepts
8.2 Concern
8.3 Measure
8.4 Quantifiers
8.4.1 Perceived QoS
8.4.2 Voice
8.4.3 Intrinsic QoS
8.4.4 Answer-Seizure Ratio (ASR)
8.5 Evaluation
8.5.1 Assessment of Likely User Perception of Quality
8.5.2 Assessment of Intrinsic QoS
8.5.3 Diagnosis
8.5.4 Monitoring

9 ROUTING RELIABILITY
9.1 Evaluative Concepts
9.2 Concern
9.3 Measure
9.4 Quantifiers
9.4.1 Perceived QoS
9.4.2 Intrinsic QoS
9.5 Evaluation

10 Connection Quality - Voice
10.1 Background
10.2 Evaluative Concepts
10.3 Concerns
10.4 Measures
10.5 Service Attribute Tests
10.6 Quantifiers
10.6.1 Perceived Connection Quality
10.6.2 Intrinsic Connection Quality
10.6.3 Evaluation

11 CONNECTION QUALITY - DATA
11.1 Evaluative Concepts
11.2 Concern
11.3 Measure
11.4 Quantifiers
11.4.1 Dedicated/Circuit-Switched Set Up
11.4.2 Store-and-Forward Relay
11.5 Evaluation

12 CONNECTION CONTINUITY
12.1 Evaluative Concept
12.2 Concern
12.3 Measure
12.4 Quantifiers
12.4.1 Perceived QoS
12.4.2 Intrinsic QoS
12.5 Evaluation

13 DISCONNECTION RELIABILITY

14 THE OTHER STUFF
14.1 Evaluative Concepts
14.2 Typical concerns
14.3 Service Level Agreements
14.4 Quality vs. Economy Afterword

APPENDIX A
APPENDIX B
APPENDIX C
ABBREVIATIONS
INDEX

- - - - - - - -
-
2002, 230 Pages. Order #DR654.
- - - - - - - -
-

4 Arapaho Rd.

Brookfield, CT 06804-3104

1-888-ROTHSTEin

Telephone: 203.740.7444; 888.768.4783

Fax: 203.740.7401

All bookstore enquiries should be sent to Rothstein Associates at the above address.

Looking for Practical Knowledge?