“The Art of Computer Systems Performance Analysis” is a comprehensive book by Raj Jain on a topic that is often ignored by computer programmers and architects or is approached in a haphazard manner: the analysis of the performance of computer systems. Such an analysis is useful for capacity-planning, selection of hardware or software for procurement, identification of bottlenecks in an under-performing system, etc. To be clear about the intent of this book, it is not a cookbook on building high-performance systems or on optimising code to make it run faster. Even though it was first published nearly 20 years ago, almost all of it still remains relevant and useful. It would perhaps not be hyperbolic on my part to term this book as one of the under-appreciated classics of computer programming.
It is usually quite important to know the performance of a computer system. A high-performance system generally does more work with fewer resources when compared to a low-performance system and thus saves time and money for its users. Without a proper evaluation of its performance, you cannot effectively improve the performance of a system.
The book presents three techniques for performance-evaluation: analytical modelling, simulation and measurement. Each of these techniques is explored in great detail and the author provides several criteria for choosing among them. The book is full of illustrative case-studies and warnings against common traps and pitfalls like misleading benchmarks and inappropriate ratios. The author also presents a refresher on probability theory and statistics, as well as an overview of data-presentation, design of experiments, generation of pseudo-random numbers and queueing theory. As I noted earlier, it is quite comprehensive.
The 36 chapters of the book are spread across the following parts:
- An Overview of Performance Evaluation
- Measurement Techniques and Tools
- Probability Theory and Statistics
- Experimental Design and Analysis
- Simulation
- Queueing Models
Each part is preceded by an introductory overview and is succeeded by recommendations for further reading from the bibliography provided near the end of the book. Each chapter is filled with illustrative examples and is followed by exercises that test your comprehension. I must admit that the quotes before most of the chapters made me smile.
Though the book presents quite a few equations and formulae, a reader with exposure to engineering mathematics at the under-graduate level should not find it too difficult to follow. To be honest, my eyes tended to glaze over some of the formulae-filled text, especially in sections that seemed to be there merely to make the book exhaustive. If you are new to probability and statistics or have largely forgotten about it, I would strongly recommend that you first read a book that delves into the intuition behind the formulae presented in this book. Misunderstood application of statistical techniques is perhaps more damaging than being unaware of them.
As computer programmers we tend to think of performance analysis as, say, measuring the execution time of our programme on a large data-set or the average of running it several times over an input data-set. Such grossly-misdirected efforts have prompted rants like the infamous “Programmers Need to Learn Statistics or I Will Kill Them All” by Zed Shaw. Even if you are not a performance analyst by profession, as an engineer you need to know what to measure, how to measure it, how to analyse the measurements and how to model a system in order to predict its behaviour. This is a book that teaches you a lot of the basics that you need to achieve that goal.