Weighing the intangible: towards a framework for Information Society indices

Authors: Dan M. Grigorovici, Jorge Reina Schement, Richard D. Taylor

The objective of the paper is to make a contribution to a conceptual model of information indicators. As suggested by the recent surge in the development of quantitative measures of "e-readiness" or "e-metrics" in both academic and industry research, macro level indicators of Information and Communication Technologies (ICT) are an essential tool for quantifying the "Digital Divide", thus having a profound role in the development of effective policies to overcome it. A variety of statistics are being currently used to measure Internet access or level of deployment of ICT at various levels, but the underlying theory is non-existent at best, data are often not comparable, the choice of indicators is much too subjective, and the variables chosen, statistics, their methodologies and the logical process of arriving at a choice for one index rather than another do not have common conceptual ground, thus lacking concurrent validity. This fact is obvious when comparing same level rankings (country or industry, etc.) stemming from different indices, as there are no two levels ranked in the same place in different indices. We suggest reasons why this is the case, and ways to overcome the challenge of developing valid e-metrics.

The paper is organized into five sections. First, a survey of the history of Information Society Indicators (ISI) draws on an analysis of the relationship between ICT and development. We argue that the area lacks coherence in that every major index is built upon different values, different definitions of the key terms ("information", "knowledge", etc.), and boundary conditions restricting concurrent validity. It is our contention that this is an effect of the lack of conceptual clarity that stems mainly from the difficulties of "weighing the intangible" when using industrial era measurement instruments. Thus, the first section ends with the conclusion that most of the existing operationalizations lack a comprehensive deductive theory to guide them. The second section analyzes what the different measurement models reviewed bundled into their operationalizations of ISI, thus answering the question of where, how and in what forms are information indicators currently being applied. We argue that a major reason for the lack of a common metatheoretical ground in measuring inter-sector and inter-country comparisons in terms of the impact of information in development, Information Society readiness, etc., is the lack of agreement as to the use of different terms and the values behind them. Put another way, we might measure different things with completely different, and too often with "unweighted" measures. The third section discusses challenges to building an effective model of information metrics by way of a statistical analysis of one of the widely diffused e-metrics (UNDP's "Technology Achievement Index"). Based on results from several regression analyses of the index chosen, we dissociate the variables that could be useful for developing a unified measure of Information Society and report the results. Our "weighing" of key indicators and the variables under them will help draw a conceptual blueprint for future research in the area and answering the question whether a single composite index is at all needed. The fourth section inductively argues for the usefulness of building a new model of ISI that addresses the difficulties analyzed by allowing a more flexible, multi-criterial instrument suited to the necessity of measuring intangibles. We suggest Structural Equation Modeling as a methodology useful in testing current models and sorting variables that cluster together into a new model. We also discuss its potential to issues of application to social development measurements.

In the last section we suggest steps for future research in the area and discuss the utility of building a new model of ISI by taking in the account issues of broad-based development and their subsequent impact on quality of life.