The Global State of Democracy Indices Methodology

Inspired by the Institute’s 20th anniversary in 2015, International IDEA renewed its commitment and vision to be a visible global actor, voice and agenda-setter in the democracy-building field. With this goal in mind, the Institute has initiated a periodic publication, The Global State of Democracy (International IDEA 2017), which analyses key topics related to democratic development. The Report draws on multiple sources of information, including the Global State of Democracy (GSoD) Indices, which support the comprehensive analysis of the global state of democracy.

The overarching objectives of the Global State of Democracy initiative are to (a) assess the global state of democracy by analysing topical issues in democracy; (b) diagnose critical global and regional trends and developments that reflect the current state of democracy around the world; (c) identify opportunities for improving or reforming democracy, paying special attention to diversity, gender and security; (d) draw attention to good practices; and (e) complement global and regional overviews of democratic development with analyses of particular issues that fall within International IDEA’s mandate and areas of expertise.

This methodology document outlines the conceptual distinctions and measurement framework of Version 9 of the GSoD Indices. Chapter 1 provides an overview of the overall objective. Chapter 2 details the GSoD conceptual framework, including comparisons with a precursor project, International IDEA’s State of Democracy (SoD) Assessment Framework. Chapter 3 expands on the empirical indicators selected, including information on sources and the procedures used to aggregate the indicators into indices of democratic features at the attribute and subattribute levels. Chapter 4 presents a comparison with existing measures that attempt to capture relatively similar aspects of democracy at the attribute, subattribute or subcomponent levels. Chapter 5 presents cautionary notes.

The document ends with annexes outlining the GSoD conceptual framework; the attributes, subattributes, assessment questions and empirical indicators; an overview of indicators and sources; a comparison with International IDEA’s SoD Assessment Framework; and an overview of differences compared with previous versions of the GSoD Indices.

The objective of the Global State of Democracy (GSoD) Indices is to provide systematic and nuanced data that captures trends at the global, regional and national levels related to International IDEA’s comprehensive understanding of democracy. The Indices turn a broad range of empirical indicators from various data sets into measures (attributes, subattributes and subcomponents) of different aspects of democracy. They also provide scores for almost all independent countries in the world with more than 250,000 inhabitants for the period 1975–2024. The Indices can assist stakeholders, including policymakers, researchers and civil society actors, in their analyses of trends related to different aspects of democracy, and their identification of priority policy areas. In addition, the quantitative data lend themselves to further uses, such as the comparison of scores across countries and within countries over time for disaggregated aspects of democracy.

Like the original State of Democracy (SoD) Assessment Framework (Beetham et al. 2008), which has primarily been used for qualitative democracy assessments, the aim is to construct a framework with universal applicability. So, rather than creating an overarching democracy index that offers a single score per country, the GSoD Indices provide measurements of distinct aspects of democracy (namely through 30 indices), which are emphasized by one or more major traditions within democratic thought.

Nonetheless, it is important to emphasize that International IDEA’s broad understanding of democracy as popular control over public decision making and political equality is compatible with different formal and informal institutional arrangements. These principles are open to a context-sensitive implementation of universal standards around the world. Within this conceptual framework, it is assumed that a democratic political system can be achieved and organized in a variety of ways, and the principles can be fulfilled to varying degrees. This perspective has informed and influenced the development of a disaggregated measurement framework that provides users with more nuanced information compared to a single ‘mash-up’ index that collapses all the attributes into a single score.

The GSoD data set includes separate, fine-grained indices grouped under four attributes of modern democracy: (a) Representation, (b) Rights, (c) Rule of Law, and (d) Participation. These measures build on an elaborate conceptual framework that is explicitly rooted in International IDEA’s SoD framework and academic works on democratic theory (see Chapter 2). It is, however, necessarily somewhat narrower and has been modified since some of the features captured by the SoD Assessment Framework do not lend themselves to systematic cross-national and longitudinal measurement. See Annex E for more information.

The conceptual framework has guided the selection of relevant and reliable indicators with a high coverage in terms of years and countries. These rely on various types of sources and have been collected from extant data sets compiled by different organizations and researchers. Any interested party can have full and free access to the country-level data for all Indices, downloadable from International IDEA’s website. Almost all the underlying indicators extracted from various data sets are also available (with some exceptions for copyright regulations and other limitations attached to external data sources). The selected indicators are aggregated into nuanced index scores. These scores are supplemented by uncertainty estimates to help users judge whether apparent differences are statistically significant.

Taken together, the GSoD Indices have a number of strengths compared to many extant measures of democracy (see Chapter 4). They are based on a broad understanding of democracy fleshed out in an elaborate conceptual framework, and the different steps in the construction of the Indices are transparent and explicitly justified. The Indices themselves offer nuanced distinctions in the form of interval scale measurement. This means that the scores are graded; and that the numbers express a rank order and the exact differences between the values. In contrast, nominal data only show that some things are different; similarly, ordinal scale measurement ranks phenomena but, as the distances between scores are not known, it is not possible, for instance, to meaningfully calculate the average (without relying on rather demanding assumptions).

The GSoD data set offers four indices at the attribute level and 17 indices at the subattribute level based on 154 input indicators. In addition, the data set contains an ‘intermediate’ contestation index (see Chapter 3) and seven subcomponent indices for two subattributes: Civil Liberties and Political Equality (see Chapter 3 and Annex A). The underlying data are drawn from a variety of high-quality data sets based on different types of sources. For almost all indices, the yearly scores for each country are accompanied by uncertainty estimates that can be used to assess whether differences between countries and within countries over time are statistically significant. The only exceptions are the subattributes based on a single observational indicator (Electoral Participation) or formative aggregations procedures (Inclusive Suffrage, Direct Democracy, and Local Democracy).

Finally, after the release of the first version of the GSoD Indices, the conceptual and measurement framework was updated, partly based on external remarks on, and questions about, the methodology and concrete scores. The conceptual framework was further updated before the release of the seventh version of the data set. These processes led to a number of revisions to the methodology, which are summarized in Annex E.

The point of departure for the conceptual framework of the GSoD Indices is International IDEA’s SoD Assessment Framework (Beetham et al. 2008). Several adjustments to the SoD framework were made in order to establish internal coherence in the conception of attributes of democracy and to enhance the theoretical links between different levels of the framework—that is, the principles, attributes, subattributes, subcomponents and indicators (see Annex D for a detailed discussion). These modifications were also necessary to transform the original SoD framework from an in-country, synchronic, qualitative democracy assessment tool into a systematic instrument for cross-national, diachronic, quantitative measurement of the state of democracy.

This chapter outlines the conceptual framework behind the construction of the GSoD Indices: the democratic principles and the associated attributes and subattributes. It concludes with an overview of the conceptual structure that guides the measurements.

2.1. Two democratic principles

Democracy means rule by the people. The obvious contrast to democracy is autocracy, or rule by a narrow, privileged elite that is not subjected to popular control. Beyond these parameters, however, there is much disagreement about the meaning of democracy. The definition of democratic principles presented here is grounded in considerations about the basic premises underlying the ideal of democratic rule. This explication of the premises makes it possible for others to judge their legitimacy and validity, and their alignment with the principles (Adcock and Collier 2001).

David Beetham, one of the main contributors to the development of the SoD framework, argued that the motivations behind democracy are (a) ‘the idea of human dignity or worth, and its core value is that of human self-determination or autonomy … autonomy is understood collectively, as a sharing in the determination of the rules and policies for the association of which one is a member, and to whose authority one is subject’ (Beetham 1999: 7); and (b) ‘that people are generally the best judges of their own interests’ (Beetham 1999: 13). This emphasis on equal respect for individuals and their capacity for creative self-determination, according to Beetham (1999: 18), underpins democracy as a universal value (see also Buchanan 2004; Fukuyama 1992; Sen 1999). Mutual respect and autonomy are also the premises that suggest we should respect cultural differences to the extent that they are compatible with these premises. Thus, democracy and respect for cultural differences do not contradict each other—their justification has overlapping roots, which means that they have similar claims to legitimacy.

These premises about human dignity and judgement are very similar to those proposed by Dahl (1989), who argued forcefully that all interests should receive the same weight and that virtually all adults are competent to participate in collective decision making regarding their own interests. A number of general arguments have been presented in favour of these underlying premises. First, opponents of autocratic ‘guardianship’ often refer to the lack of reliable alternatives as undemocratic regimes tend to be more repressive and do not generally outperform democracies in terms of creating human development. Second, many people share moral intuitions about just political rule, which tend to include fundamental ideas of equal human dignity and sufficient competence of ordinary citizens to take part in decision making. Third, human beings are generally risk averse and this means that we should prefer democracy because we are generally more secure and to a greater degree know what we get under democratic rule. Fourth, sound scepticism of paternalism tells us that truly benevolent dictators are a rare phenomenon (see e.g. Beetham 1999; Buchanan 2004; Christiano 2011; Dahl 1989; Gould 1988; Held 2006; Rawls 1971).

From these premises, Beetham (1999: 1–13) and International IDEA (Beetham et al. 2008: 20–21) derived popular control and political equality as the two core principles of democracy underlying the SoD framework. Consequently, popular control over public decision making and decision makers, and equality of respect and voice between citizens in the exercise of that control, also underpin the GSoD framework.

That is not to say that these principles only have one suitable institutional embodiment. The core democratic principles are compatible with different, context-sensitive and context-specific institutional set-ups, which means that the principles can be realized in a variety of ways. Democracy is, for example, compatible with various electoral systems (majoritarian, proportional, or mixed), different forms of government (presidentialism, parliamentarian, or mixed), different legal systems (common law, civil law and so on), different types of political parties and party systems, and unitary or federal states. There are also some institutional arrangements, such as absolutist monarchies and military and one-party dictatorships, that are not compatible with the principles and therefore undemocratic, because they do not align with the fulfilment of popular control and political equality.

The first principle concerns what is being distributed (political control over authoritative political decision making) while the second principle concerns how it should be distributed (equally) and implemented (impartially). However, popular control on its own does not mean that all individuals have equal influence over authoritative political decision making. Conversely, political equality alone does not mean that there is any (collective) popular control over decision making in a society. This means that the principles complement each other and that they are both required. In other words, popular control and political equality are necessary and jointly sufficient principles of democracy. Hence, a specification of what democracy is should consider all relevant aspects of both.

A related issue is whether these principles jointly are, indeed, sufficient to capture the democratic ideal. Most importantly, the question arises why political liberty or freedom is not explicitly mentioned as one of the principles. The answer is essentially that particular liberties and freedoms are implied by the two principles. Meaningful popular control and political equality are not possible without respect for fundamental freedoms such as civil and political liberties. Moreover, popular control and political equality mean that there is self-government (directly or through representatives) as opposed to government by internal or external guardians. Accordingly, there is freedom in the sense of living under laws that people have (mostly through political representatives) been part of making, rather than laws imposed from above.

In this way, the democratic principles are based on explicit premises, on the one hand, and correspond to the values that are generally associated with democracy, on the other (Bobbio 1989; Dahl 1989; Hansen 1989; Kelsen 1920; Lauth 2004; Munck 2016). The original SoD framework lists participation, authorization, representation, accountability, transparency, responsiveness and solidarity as key intermediary values (Beetham et al. 2008: 22–24). See Annex D for more details of the original SoD framework and the adjustments made to transform it into a systematic and clear conceptual framework for the GSoD Indices.

2.2. Attributes and subattributes

Since the principles of democracy are general and abstract, they have to be specified and broken down into measurable attributes and subattributes that can be used to develop a transparent and precise measurement framework. As a point of departure, the project used the SoD framework’s somewhat more detailed explication of the principles:

The democratic ideal in and of itself seeks to guarantee equality and basic freedoms; to empower ordinary people; to resolve disagreements through peaceful dialogue; to respect difference; and to bring about political and social renewal without convulsions.
(Beetham et al. 2008: 17)

Accordingly, the GSoD Indices seek to capture five issues: the extent to which there is effective popular control over public decision makers (vertical accountability); the extent to which the citizens hold politically relevant freedoms and power resources; the extent to which executive powers are checked effectively by other powers (horizontal accountability); the extent to which public authorities are impartial and predictable in implementing the law; and the extent to which people have and make use of various opportunities for political participation at different levels.

Thus, the framework (see Beetham 1999: 154–57; Beetham et al. 2008: 27–28) distinguishes between the following four attributes:

1. Representation (free and equal access to political representation);
2. Rights (individual liberties and resources);
3. Rule of Law (predictable and equal enforcement of the law and judicial checks on government power); and
4. Participation (active political involvement by the citizens).

This list of attributes covers the features that are conventionally associated with democracy, primarily representative government. However, it also covers issues often neglected or consciously left out by other attempts to conceptualize democracy. The GSoD conceptual framework draws on the various understandings of democracy generally known as electoral democracy, liberal democracy, social democracy and participatory democracy. It therefore demonstrates partial overlaps with the features emphasized by these different traditions of democratic thought (see Coppedge et al. 2011; Cunningham 2002; Held 2006; Møller and Skaaning 2011, 2013). Annex A is a matrix that shows which attributes and subattributes of the GSoD conceptual framework are shared with each of these traditions. These issues are discussed in more detail in the following sections, where the four attributes are further divided into a number of subattributes.

See Beetham (1999) and Beetham et al. (2008) for a more detailed treatment of the link between the two principles of democracy and these attributes—or, more correctly, a similar set of attributes, since the GSoD framework differs slightly from Beetham’s distinctions and the SoD framework. The mediating values—linking principles to attributes and presented in the SoD framework (see Beetham et al. 2008: 24)—are explicitly or implicitly referred to in the discussion of the attributes. For a brief overview of the use of the original SoD framework see Annex D.

2.3. The four attributes in detail

Attribute 1: Representation

Of the four attributes of democracy, arguably the most essential and least contested is Representation (Beetham 1999: 155, 162–63). It emphasizes contested and inclusive popular elections for legislative and (directly or indirectly elected) executive office (Dahl 1971; see also Alvarez et al. 1996; Boix, Miller and Rosato 2014; Skaaning, Gerring and Bartusevičius 2015; Møller and Skaaning 2011; Munck 2009). Most of the features associated with Representation are covered by the concepts of electoral integrity (see Norris 2014), free and fair elections (see Elklit and Svensson 1997) and electoral democracy (see Diamond 1999).

For this attribute, the framework distinguishes between six subattributes:

1.Representation:

1.1. Credible Elections denotes the extent to which elections for national, representative political office are free from irregularities, such as flaws and biases in the voter registration and campaign processes, voter intimidation and fraudulent counting.

1.2. Inclusive Suffrage denotes the extent to which adult citizens have equal and universal passive and active voting rights.

1.3. Free Political Parties denotes the extent to which political parties are free to form and campaign for political office.

1.4. Elected Government denotes the extent to which national, representative government offices are filled through elections.

1.5. Effective Parliament denotes the extent to which the legislature is capable of overseeing the executive.

1.6. Local Democracy denotes the extent to which citizens can participate in free elections for influential local governments.

Attribute 2: Rights

Rights in the form of liberal and social rights support both fair representation and the vertical mechanism of accountability that the first attribute seeks to achieve. Thus, the relevance of this attribute to democracy is due to the importance of individual human rights for securing both popular control and political equality in practice (Beetham 1999: 33–49, 89–114; Beetham et al. 2008: 27). Without equal protection for negative and positive liberties, the meaningfulness of political equality is undermined (Saward 1998; Dahl 1989; Meyer 2005; Heller 1930; O’Donnell 2007, 2010). In short, ‘democracy is only secure if the conditions for the exercise of the popular will are guaranteed on an ongoing basis, through a protected set of basic freedom rights’ (Beetham 2004: 65).

Scholars disagree about whether to treat fundamental human rights as democratic rights. Proponents of minimalist, purely electoral definitions of democracy, such as Schumpeter (1974; see also Alvarez et al. 1996; Boix, Miller and Rosato 2014), argue that democracy should not be conflated with liberal freedoms, social equality or the ‘good life’ more generally because this leads to conceptual stretching and confusion. This critique applies to all the other attributes apart from Representation. Schumpeter even goes so far as to not require universal suffrage, and to suggest that it is up to the demos itself to decide who should have suffrage.

Then again, the capacity to exercise political rights arguably rests on the presence of due process, and civil rights and liberties (Kelsen 1920; Lauth 2004; Merkel 2004). As Beetham (2004: 61) emphasized, ‘if people are to have any influence or control over public decision making and decision makers, they must be free to communicate and associate with one another, to receive accurate information and express divergent opinions, to enjoy freedom of movement and to be free from arbitrary arrest and imprisonment’. In addition, the protection of fundamental rights relies on personal security and a legal system that has sufficient integrity and capacity to uphold them by prosecuting rights violations and holding fair trials. These issues have historically been associated with the concept of liberal democracy.

Finally, political equality also rests on individuals possessing a basic level of power resources. This criterion clearly goes beyond the dominant view found in the empirical democratization literature, in which social welfare and democracy are treated as analytically distinct phenomena rather than being lumped together (Diamond 1999: 8; Karl 1990: 2; Linz 2000: 57–58). One of the main arguments in favour of this position is that if social rights are characterized as democratic rights, the number of testable research questions is reduced—because some relationships become true by definition (Alvarez et al. 1996: 18). However, the more comprehensive perspective is not necessarily subject to this problem, as long as users of the data are given the opportunity to assess the empirical relationship between different aspects of the same overarching concept.

Moreover, if power resources, such as education, health and income, are not provided, economic and social inequalities are likely to spill over into unequal political influence. To quote Beetham again:

If freedom is a good only because of the value that lies in exercising it, then those who lack the capacity or resources to exercise a given freedom are being denied the enjoyment of it, even though they may not formally be being obstructed. In similar vein, we could say, it is a condition of exercising one’s civil and political rights that one should be alive to do so, and should have the education and, where necessary, the resources to take advantage of them.
(2004: 65; see also Plant 1991: Ch. 7)

People should, therefore, have access to a minimum platform of basic welfare that supports their ability to be politically active and reduces the political advantages of those who are better placed.

Furthermore, rights imply equality: otherwise, they would just be privileges. This means that all rights should be equally guaranteed to all. Thus, discrimination due to economic status, social identity or gender is not in alignment with democratic principles as preferential treatment of particular groups violates the democratic values of human dignity and equal worth. These issues are generally associated with the concept of egalitarian or social democracy.

Taken together, the Rights attribute has significant overlap with the rights and liberties covered by the International Covenant on Civil and Political Rights (with the exception of article 25, which refers to Representation as captured by attribute 1), as well as the International Covenant on Economic, Social and Cultural Rights.

For this attribute, the framework distinguishes between four subattributes:

2.Rights:

2.1. Access to Justice denotes the extent to which the legal system is fair (citizens are not subject to arbitrary arrest or detention and have the right to be under the jurisdiction of—and to seek redress from—competent, independent and impartial tribunals without undue delay).

2.2. Civil Liberties denotes the extent to which civil rights and liberties are respected (citizens enjoy the freedoms of expression, association, religion, movement, and personal integrity and security).

2.3. Basic Welfare denotes the extent to which there is access to fundamental resources and social services (citizens enjoy nutrition, social security, healthcare and education).

2.4. Political Equality denotes the extent to which political equality between social groups and genders has been realized (citizens are not subjected to discrimination and exclusion due to their gender, economic status or social identity).

Attribute 3: Rule of Law

Beyond regular elections, the exercise of political power needs to be subject to defined limits and continuous scrutiny and people should be able to live a secure life without the risk of political violence (Beetham 1999: 155, 163–65; Beetham et al. 2008: 24). If there is not integrity in the public administration and the judiciary is not independent, executive power is more prone to be abused for private gain and to bias in political decision making and implementation (Holmes 1997; Lauth 2004; Merkel 2004; Montesquieu 1989 [1748]; O’Donnell 2007, 2010; Vile 1998). Moreover, the government and public administration more generally ought to implement official public policies in an impartial and predictable manner (Beetham 1999: 165; Beetham et al. 2008: 75–76). If the implementation is unfair and unpredictable, large discrepancies between official laws and policies, on the one hand, and practices, on the other, undermine the fulfilment of democratic principles (Lauth 2004; Merkel 2004; Munck 2016; Habermas 1995; Ross 1952; O’Donnell 2010; Alexander and Welzel 2011).

In other words, vertical accountability through elections should be supplemented by the rule of law. That said, it is important to recognize the potential trade-off between popular sovereignty and the rule of law since the majority will and rule of law do not always go hand in hand (Mill 1996[1859]; Hamilton, Madison and Jay 1995 [1787/1788]; Tocqueville 1988 [1835/1840]). Habermas (1996), however, proposes that popular sovereignty and the rule of law are not only compatible but also mutually constitutive, meaning that institutional restraints serve to enable, rather than limit, effective democracy and vice versa (see also Beetham 1999: Ch. 5; Holmes 1997; Lauth 2004; Merkel 2004).

This attribute is also related to the liberal-democratic tradition in political theory. The responsiveness of representatives to citizens is not sufficient for effective popular control over government: ‘The accountability of all officials, both to the public directly and through the mediating institutions of parliament, the courts ... and other watchdog agencies, is crucial if officials are to act as agents or servants of the people rather than as their masters’ (Beetham et al. 2008: 24).

On the Judicial Independence subattribute, it is relevant to mention the ongoing debate about the democratic legitimacy of judicial review. Some argue that, in its strongest form, it is ‘politically illegitimate, so far as democratic values are concerned: by privileging majority voting among a small number of unelected and unaccountable judges, it disenfranchises ordinary citizens and brushes aside cherished principles of representation and political equality’ (Waldron 2006: 1353; see also Bellamy 2007). Others think that strong judicial review can be justified on democratic grounds and is therefore compatible with democratic values (Lever 2009). In general, judicial independence should support the courts exercising weak judicial review (on the distinction between strong and weak judicial review, see Waldron 2006: 1354–55).

For this attribute, the framework distinguishes between four subattributes:

3.Rule of Law:

3.1. Judicial Independence denotes the extent to which the courts are not subject to undue influence, especially from the executive.

3.2. Absence of Corruption denotes the extent to which the executive, and public administration more broadly, does not abuse office for personal gain.

3.3. Predictable Enforcement denotes the extent to which the executive and public officials enforce laws in a predictable manner.

3.4. Personal Integrity and Security denotes the extent to which bodily integrity is respected and people are free from state and non-state political violence.

Attribute 4: Participation

Democratic institutions tend to be hollow if not filled by active citizens in connection with and between different kinds of elections. In other words, politically involved citizens are considered an important part of democracy (Beetham 1999: 156; Beetham et al. 2008: 28). The more that citizens are allowed to participate at all levels of government and make actual use of these opportunities, through participation in dynamic civil society organizations and elections, the more popular control and responsiveness can be achieved (Barber 1988; Macpherson 1977; Mansbridge 1983; Pateman 1970). This is the main agenda of the participatory democracy tradition. Whether popular participation also increases political equality depends on how representative of the whole population the engaged citizens are. Nonetheless, everything else being equal, high levels of different forms of popular participation will tend to reflect more inclusive and representative involvement than very low levels of popular participation. Without any popular involvement, democratic institutions become empty and meaningless shells, whereas with active participation democracy is vibrant.

For this attribute, the framework distinguishes between three subattributes:

4.Participation:

4.1. Civil Society denotes the extent to which organized, voluntary, self-generating and autonomous social life is institutionally possible.

4.2. Civic Engagement denotes the extent to which people actively engage in civil society organizations and trade unions.

4.3. Electoral Participation denotes the extent to which citizens vote in national legislative and (if applicable) executive elections.

Additional measure: Direct Democracy

In previous versions of the GSoD Indices, Direct Democracy was included as one of the subattributes of Participation. However, exercises of direct democracy have remained relatively rare and stochastic globally, and measuring the extent to which mechanisms of direct democracy are employed has not produced data that are especially useful in our conceptual framework. Versions 7 and later remove Direct Democracy from the conceptual framework, but continue to include the index in the data set for those users who may find it useful.

0.0. Direct Democracy denotes the extent to which citizens can participate in direct popular decision making.

2.4. Summary

Table 2.1 presents overviews of the conceptual framework underlying the GSoD Indices. To summarize, the framework consists of 17 subattributes linked to the four attributes. Each subattribute is associated with an assessment question that guides the selection of relevant empirical indicators. It is important to note that the different conceptions of democracy that are combined in this framework (electoral democracy, liberal democracy, social democracy and participatory democracy) are not considered to be orthogonal or contradictory. Instead, the different understandings and aspects are assumed to be compatible and complementary.

The Global State of Democracy produces indices to capture the main attributes of democracy. Each attribute, in turn, covers three to six subattributes. The subattributes are operationalized using a series of indicators from existing data sets (see Annexes B and C). The goal is to cover the period since the International Covenant on Civil and Political Rights and the International Covenant on Economic, Social and Cultural Rights took effect in 1976. The previous year is included as a reference point. Hence, the period covered thus far is 1975–2024.

This period overlaps with what is often termed ‘the third wave of democratization’ (Huntington 1991), which began with the first free elections in Portugal on 25 April 1975, exactly one year after the start of the Carnation Revolution. This period is particularly important for International IDEA because it serves as the most common reference point for current democratic trends. This is reflected in the fact that many contemporary debates about democratic development and resilience focus on these four decades (e.g. Diamond 2011; Levitsky and Way 2015; Merkel 2010; Møller and Skaaning 2013; Puddington 2011; Schedler 2013). Other reasons for not going further back in time are the higher level of confidence in more recent data on some of the indicators and the fact that there is generally more relevant and extensive data available for recent decades.

The GSoD Indices have global coverage. As the Varieties of Democracy (V-Dem) Database is the largest source of data for the construction of the Indices, the data collection focuses on 174 of the independent countries covered by the V-Dem data set. Semi-sovereign units (for example Somaliland) and microstates with a population of less than 250,000 have been excluded. V-Dem procedures on how to treat units that have split (such as the Soviet Union and Yugoslavia) or merged (such as East Germany and West Germany) have also been adopted (see Coppedge et al. 2021a). The resulting Indices capture democratic development in the vast majority of the countries in the world.

All the indicators are compiled in a single database in a country–year format, which means that a country receives a single score per indicator for a particular year. Indicators not originally available in a country–year format are transformed to fit this format—see the Codebook (Tufis and Hudson 2025a) for details. The aggregation procedures used to construct the various indices at the level of subcomponents, subattributes and attributes are presented below.

3.1. Criteria for indicator selection

The operationalization of the GSoD conceptual framework takes the assessment questions presented in Table 2.1 as a starting point. The most important task is to identify empirical indicators that tap into the features emphasized by the different subattributes. It is important to highlight that International IDEA has not collected new data for this measurement exercise and is therefore exclusively reliant on existing sources.

The main priority of the construction of Indices is a high level of concept–measure consistency; that is, the extent to which the indicators capture the core meaning of the particular concept that is being operationalized (Adcock and Collier 2001; Goertz 2006: Ch. 4; Munck 2009). In addition, the following criteria guide the selection of indicators:

1. Indicators must be produced through transparent and credible data-generating processes.
2. There must be extensive coverage: the indicators should include scores for at least 130 countries from different regions.
3. There must be multiple indicators for each subattribute wherever an adequate observable indicator is not available.
4. The data sets from which the indicators are sourced should be updated regularly.

Moreover, the GSoD Indices attempt to make use of indicators from different data sets based on different types of data and to prioritize data sources that are readily available in a systematic, downloadable format, free of charge. The International Country Risk Guide (ICRG) is the only data set used that is not freely available. Thus, the ICRG scores are used for the construction of index values but not made publicly available on International IDEA’s website. A number of trade-offs mean that not all the criteria are fulfilled in every case, but the construction of the GSoD Indices applies most of them in most cases.

Multiple indicators from different data sets are used because, given high-quality indicators, a cumulative approach to measurement generally improves confidence in the scores. The combined efforts of various data providers make the resulting measures more nuanced and reliable. The use of different indicators enables capture of related, but nonetheless distinct, aspects of the features to be measured. This procedure also tends to reduce the influence of idiosyncratic measurement errors associated with individual indicators. Finally, drawing on several indicators allows an assessment to be made of the level of agreement between them, and this information can be used to calculate uncertainty estimates for the Indices (see Pemstein, Meserve and Melton 2010; Fariss 2014; Linzer and Staton 2015).

Unfortunately, many recent efforts at data collection, such as the Rule of Law Index by the World Justice Project and the Perceptions of Electoral Integrity by the Electoral Integrity Project (EIP), only cover a small number of years and/or have been subjected to changes in the methodology over the years. Their indicators are therefore not used because there would not be sufficient overlap with other sources with longer time-series. Other data sets, such as the National Elections Across Democracy and Autocracy data set (Hyde and Marinov 2012), the Human Rights Protection Scores (Fariss 2014), and the Judicial Independence Index (Linzer and Staton 2015), are not used because they have not been updated recently and in some instances have been discontinued. However, measures from these sources are used when assessing the validity and reliability of the GSoD Indices. More particularly, the correlations between the GSoD Indices and widely used and recognized alternatives were assessed as the Indices were designed, and some of the largest disagreements discussed. The selection of indicators has been an iterative process over the several versions of the GSoD Indices. Potentially relevant indicators were suggested, assessed and sometimes discarded over several rounds by members of the team and the Expert Advisory Board. Particularly after the first and sixth versions were released, an internal evaluation process and external inputs led to a few modifications. The presentation of the selected indicators below discusses examples of discarded indicators and the reasons for their exclusion.

3.2. Different types of source and data sets

The GSoD Indices summarize information from 154 indicators collected from 22 data sets. Some of these indicators, such as the elected office and direct democracy indicators from V-Dem, are composite measures based on several subindicators. The data sets listed in Table 3.1 represent four different types of source data:

1. Expert surveys (ES). In these surveys, country experts assess the situation on a particular issue in a country. This kind of data is provided by V-Dem and the ICRG.
2. Standards-based ‘in-house coding’ (IC). This type of coding is carried out by researchers and/or their assistants based on an evaluative assessment of country-specific information found in reports, academic publications, reference works, news articles, and so on. This kind of data is provided by V-Dem, Polity5, Lexical Index of Electoral Democracy (LIED), Civil Liberty Dataset (CLD), Bjørnskov-Rode regime data (BRRD), Political Terror Scale (PTS) and Media Freedom Data (MFD). Freedom in the World and the BTI are classified as ‘in-house coding’ in the rest of this document, but it should be noted that their internal processes involve both country experts and in-house review and revision, meaning that their coding processes are between these first two categories.
3. Observational data (OD). This is data on directly observable features such as the ratio of women to men in parliament, infant mortality rates and legislative elections. This kind of data is provided by V-Dem, Food and Agriculture Organization of the United Nations (FAO), Global Health Data Exchange (GHDx), World Health Organization (WHO), International Labour Organization (ILO) and the UN Statistics Division.
4. Composite measures (CM). These are based on a number of variables that come from different existing data sets rather than original data collection. This kind of data is provided by V-Dem in the form of an elected officials index, a direct democracy index, and a local government index.

All of these source types and data sets have different strengths and shortcomings (see, e.g. Arndt and Oman 2006; Landman and Carvalho 2009: Ch. 3; UN OHCHR 2012; Raworth 2001; Schedler 2012; Skaaning 2018). For evaluations of specific governance and democracy indicators, see Munck (2009); Ríos-Figueroa and Staton (2014); Skaaning (2009); Møller and Skaaning (2014a); Coppedge et al. (2011); and Coppedge et al. (2017).

The advantage of expert surveys is their utilization of the extensive, country-specific knowledge of scholars, journalists, and so on, to capture features that are not easy to observe directly. One potential disadvantage with such data is that it is difficult to make the different experts apply the same standards in their assessments and to rule out individual biases.

With in-house coding it is generally easier to establish cross-country equivalence in the standards employed, but this data collection procedure is dependent on relatively easy access to relevant information and coder biases can also be an issue. Observational data is less susceptible to coder biases, but this type of data is frequently characterized by systematic biases in coverage and it can be very difficult to find relevant indicators that capture a particular phenomenon well, especially those that are not directly observable such as freedom of expression. The advantage of composite measures is that they can utilize information from several variables to achieve more nuanced and comprehensive measurements of a phenomenon, while the potential drawbacks of such measures are the accumulation of problems associated with the different variables and implausible or fuzzy relationships to the concepts they are expected to capture.

Finally, representative surveys of the general population were judged less useful for a number of reasons. These include limited coverage in terms of years and countries, the dissimilar standards generally applied by respondents (both within and across countries and time periods), the large differences in terms of nuanced knowledge about the general dynamics and performance of political institutions and the circumstances where citizens might be afraid to express their honest understanding of the lay of the land. Indicators based on surveys of the general public were therefore not used. In contrast, surveys of the mass public are used to construct the Democracy Barometer, the Worldwide Governance Indicators, the Economist Intelligence Unit (EIU) Democracy Index, and the World Justice Project (WJP) Rule of Law Index.

About the data sources

The V-Dem indicators are mainly based on scores provided by multiple, independent coders (usually a minimum of five per indicator) who are guided by elaborate coding guidelines, while some of the more factual or less judgement-based indicators are coded in-house. The scores from the country experts, which involve extensive bridge and lateral coding in order to strengthen comparability, are aggregated into point estimates and confidence bounds by a measurement model based on Bayesian item response theory (IRT) modelling techniques. Bridge coding means that an expert has coded more than one country for all years; lateral coding means that an expert has coded several countries for one year. The measurement model assesses inter-coder reliability and helps to reduce the impact of individual bias and increase the cross-country equivalence of the indicator scores in a systematic way (see Coppedge et al. 2021b, 2023; Marquardt and Pemstein 2017).

The scores included in the ICRG constructed by Political Risk Services (PRS) are assigned based on answers to a series of pre-set questions for each component by a staff member with special country expertise (Howell 2012).

In-house coding is used to construct the indicators included in the Bjørnskov-Rode Regime Data (Bjørnskov and Rode 2018); BTI (Bertelsmann Stiftung 2024); CLD (Skaaning 2008; Møller and Skaaning 2014b); Freedom in the World (Freedom House 2025); Freedom on the Net (Freedom House 2024); LIED (Skaaning, Gerring and Bartusevičius 2015); MFD (Whitten-Woodring and Van Belle 2014; 2017); Polity5 (Marshall and Gurr 2020); and PTS (Wood and Gibney 2010; Gibney et al. 2020).

The CLD, BRRD, LIED and MFD data sets draw on information found in written sources, such as electoral observation reports, human rights reports, academic volumes and articles, and information from news media. The relevant information is then transformed into indicator scores following the specific guidelines used for each data set. All of these data sets are supplemented by systematic inter-coder reliability tests that generally indicate very high reproducibility.

The remaining indicators rely on observational data compiled by V-Dem, ILO, World Bank, WHO, the FAO, and Institute for Health Metrics and Evaluation (IHME).

3.3. Linking indicators to attributes and subattributes

Annexes B and C present overviews of the indicators considered the most suitable to operationalize the democratic attributes and subattributes, employing the above-mentioned selection criteria. The tables contain brief descriptions of the indicators and the types of sources they are based on, as well as the name of the data set or data provider. The coverage in terms of years and countries, the adjustments made to some of the indicators and other details are described in the Codebook (Tufis and Hudson 2025a).

Indicators for attribute 1: Representation

Credible Elections

Ten indicators are included to capture the Credible Elections subattribute based on in-house coding and/or expert surveys from BTI, Freedom House, V-Dem and LIED. All the selected indicators tap into the quality of elections. One of the V-Dem indicators and the LIED indicator reflect the presence of free elections more generally, whereas the other V-Dem indicators and those from BTI and Freedom House capture more specific aspects of elections, including the fairness of the electoral laws, electoral management body (EMB) autonomy and capacity, government intimidation and other irregularities. The election-specific V-Dem indicators have been revised to fit the country–year format of the GSoD data set and all indicators have been set to 0 (based on the legislative and executive elections indicators from LIED) if elections were not on track due to coups, conflict and so on. As noted above, indicators from the EIP are not used as they are only available for recent years. For the same reasons, indicators from Judith Kelley’s (2012) Quality of Elections Data, Bishop and Hoeffler’s (2016) Free and Fair Elections Database, and the BMR political regime indicator (Boix, Miller and Rosato 2014) were not used to measure this or other subattributes.

Inclusive Suffrage

To measure Inclusive Suffrage, two V-Dem indicators are used. One of them designates the percentage of enfranchised adult citizens. The indicator only captures the formal regulations for citizens, however, and not the extent to which non-citizens can vote or the extent to which some people might informally be restricted in casting their votes. Once suffrage has formally been granted, the indicator does not capture whether it disappears in practice in the event of a coup, suspension/abolition of the constitution or a military regime that does not hold elections, unless a new constitution formally establishes a non-electoral regime or suffrage restrictions. To rectify this, the indicator is set to 0 when elections are not on track due to some kind of interruption (based on the legislative and executive elections indicators from LIED). This also applies to the other indicator, which refers to another important aspect of Inclusive Suffrage, namely, irregularities in voter registration. Indicators on the actual distribution of political power across social classes, social identity groups and gender are used to operationalize social rights.

Free Political Parties

Seven indicators from Freedom House, LIED, Polity and V-Dem, partly based on expert surveys and partly in-house coded, are used to measure how free political parties are. All of them reflect whether political parties more generally, and opposition parties in particular, can organize freely and stand in elections. The election-based V-Dem indicator was adjusted to fit the country–year format and together with the LIED indicator set to 0 based on two indicators from LIED (executive elections and legislative elections) if elections were not on track.

Elected Government

Eight indicators from BRRD, Freedom House, LIED, Polity and V-Dem are used to estimate an index of Elected Government. The V-Dem indicator, developed by Jan Teorell, is a composite measure based on expert coded data, in-house coded data and observational data. It captures whether the chief executive is elected (directly or indirectly) through popular elections, and whether there is a parliament with elected members. The in-house coded BRRD, Freedom House, LIED and Polity indicators capture whether political power is formally and in practice vested in contested elected offices. The two lowest values of one of the Polity indicators (openness of executive recruitment) were collapsed because they both refer to non-electoral practices. Some of the indicators have a rather formal focus, which means that they do not fully take the quality of elections into account or capture the extent to which reserved domains (or tutelary powers) and undue influence by non-elected groups might in practice restrict the effective power of elected officials to govern (see Valenzuela 1992; Merkel 2004).

Effective Parliament

Three indicators from the V-Dem experts’ survey tap fairly directly into the effectiveness of parliament by capturing the presence of opposition parties and whether the legislature carries out investigations and questioning of officials. Another V-Dem indicator on executive oversight, and the executive constraints indicator from Polity have a broader focus but are also included as they capture relevant aspects of horizontal checks on executive power.

Local Democracy

V-Dem is the only provider of a comprehensive, cross-national data set on subnational elections that meets our broader data requirements. The local government index indicates whether the local government is elected and whether it is empowered in relation to the central government, while another indicator assesses the freedom and fairness of subnational elections.

Indicators for attribute 2: Rights

Access to Justice

Concerning Access to Justice, V-Dem offers four expert-coded variables that go beyond the independence of the courts. That feature constitutes the core of the Judicial Independence subattribute under the Rule of Law attribute (see below). Two of them are based on questions that ask directly whether access to justice is secure and effective for men and women. The others are more concrete as they are based on questions relating to judicial corruption and the removal of judges for misconduct. The V-Dem indicators are supplemented with an in-house coded CLD indicator of the right to a fair trial, an effective protection of civil rights indicator from BTI, and an indicator of due process rights from Freedom House.

Civil Liberties

A larger number of indicators lend themselves to the measurement of two other subattributes of Rights: Civil Liberties and Political Equality. They are initially linked to eight subcomponents, each of which reflects core concepts in the human rights literature. The construction of these subcomponent indices enables data users to carry out more focused and disaggregated analyses using measures that have stronger conceptual coherence than highly aggregated indices. Moreover, some of these subcomponent indices help capture some of the issues emphasized in the work of International IDEA in a clearer and more specific way. These so-called cross-cutting themes are gender, diversity and conflict sensitivity.

The five subcomponents under the Civil Liberties attribute are: Freedom of Expression; Freedom of the Press; Freedom of Association and Assembly; Freedom of Religion; and Freedom of Movement.

The first subcomponent, Freedom of Expression, is measured using three indicators based on expert surveys from V-Dem and seven in-house coded indicators from BTI, CLD, and Freedom House. The questions underlying the BTI, CLD, and some of the Freedom House variables are rather broad, whereas the V-Dem indicators are more specific and refer to the right to openly discuss political issues and express political opinions outside the mass media. Two of them distinguish between freedom of expression for men and for women. Three of the questions from Freedom House (Freedom on the Net) add considerations of the broader information environment to this subcomponent, with particular attention to access to information online.

The second subcomponent, Freedom of the Press, is built from nine indicators. Seven of these measures come from V-Dem and measure the extent to which the news media are diverse, honest, critical of the government, and free from censorship (from the government or self-imposed). We add to this a broader indicator of media freedom from MFD, and an indicator on the freedom and independence of the media from Freedom House.

Eight indicators from four sources are used to measure Freedom of Association and Assembly. One of them refers directly to freedom of association. One indicator each from V-Dem and Freedom House focuses on the freedom of peaceful assembly. Two indicators from Freedom House deal with associational rights for non-governmental organizations and trade unions. The indicators offered by BTI and CLD cover freedom of assembly as well as freedom of association, where association refers to both civil society organizations and political parties. Due to their broad focus, they fit better here than under the political party freedom subattribute, which is already captured by many other indicators.

Since a number of distinct indicators with broad coverage are available for religious freedom, a separate subcomponent index was also constructed for this feature. V-Dem offers two general indicators on religious freedom based on expert surveys. These were supplemented by similarly broad in-house coded variables from CLD and Freedom House.

Freedom of Movement is captured by general, in-house coded indicators from CLD and Freedom House and three more specific, expert-coded indicators from V-Dem that distinguish between foreign and domestic movement, and provide separate assessments of the latter feature for men and women.

Basic Welfare

The provision of basic welfare is measured using a number of standard observable human development indicators: infant mortality rate (CME), life expectancy at birth (WPP), healthy life expectancy at age 60 (WHO), supply of kilocalories per person per day (FAO), and average years of schooling (IHME). The project generally refrained from using data sets with significant amounts of missing values. In the few cases where interpolation was used to fill some of the gaps, there were good theoretical and empirical reasons to expect them to be trended and not to fluctuate a great deal. Interpolation means that new data points are constructed for missing data points within the range of known data points.

In addition, two expert-based indicators from V-Dem were included to assess whether everyone in a given society has equal access to basic education and healthcare. All of these reflect the extent to which the basic needs of the population are being met.

Political Equality

The Political Equality subattribute was divided into three subcomponents: Social Group Equality, Economic Equality and Gender Equality. Five V-Dem expert-coded indicators that reflect social equality are used to measure social group equality with regard to civil liberties and political power distribution. These are supplemented by indicators of equal treatment and political equality from BTI and Freedom House.

The Economic Equality subcomponent aggregates four V-Dem indicators that specify dimensions of political exclusion based on economic and geographical characteristics. The index also includes a measure of socio-economic barriers (from BTI) and the Gini coefficient for household disposable (post-tax, post-transfer) income (from SWIID).

The Gender Equality subcomponent is measured through two expert-coded indicators from V-Dem, on power distribution by gender and female participation in civil society organizations, and five observational indicators, on the ratio of female to male mean years of schooling (IHME), the proportion of lower chamber legislators who are female (V-Dem), the gender disaggregated labour force participation rate (women-men, ILO), the share of managerial positions held by women (ILO), and gender disaggregated control of financial accounts (women-men, World Bank). To this we added an index of exclusion by gender (V-Dem), and in-house coded measures of women’s empowerment (Global Gender Gap Report).

Indicators for attribute 3: Rule of Law

Judicial Independence

Since our framework places Judicial Independence under the attribute concerning the Rule of Law, it was important to supplement the two judicial independence indicators (high courts and lower courts) from V-Dem with two V-Dem indicators on government compliance with the courts. In addition, measures of judicial independence from BTI and Freedom House are included. Finally, a measure of the separation of powers from BTI is added to include a broader concept of judicial independence. Other extant measures of judicial independence either have rather low coverage or have been discontinued.

Absence of Corruption

Although many data sets now provide indicators on corruption, only a few go back more than one or two decades or distinguish between different types of corruption. Four V-Dem indicators explicitly refer to corruption in the government as broadly understood; that is, the executive and public administration more generally but excluding the courts and parliament. These are used along with another expert-coded but broader indicator of government corruption from ICRG, and in-house coded measures from BTI and Freedom House.

Predictable Enforcement

To measure the related feature of Predictable Enforcement, five expert-coded V-Dem indicators on the executive’s respect for constitutional provisions, the presence of transparent laws with predictable enforcement, and rule-abiding in the public sector, and the practices of appointment in the state administration and armed forces are used. They stand out as the most relevant, together with an indicator from ICRG (also expert-coded), which assesses the strength and expertise of the bureaucracy and an indicator from Freedom House that measures the openness and transparency in government. To round out the measurement of the extent to which the government enforces laws in consistent ways, we added a measure of law and order from ICRG, and measures of administrative capacity and the monopoly on the use of force from BTI. More directly observable indicators either have relatively low coverage (e.g. the World Bank’s statistical capacity measure) or do not approximate the concept sufficiently.

Personal Integrity and Security

To operationalize Personal Integrity and Security, three indicators were used to capture different types of violations, such as torture and political and extra-judicial disappearances and killings. These indicators come from V-Dem and PTS. In order to capture personal security more broadly, a general indicator on political violence from ICRG is also included. It pertains to different types of conflict and violence and distinguishes between various levels. Furthermore, it is standards-based rather than events-based, which makes it more suitable for integration into the GSoD data set in a meaningful way than other conflict indicators with a broad scope. Similarly, we include indicators from Freedom House that measure illegitimate uses of force and economic exploitation. Another graded and standards-based conflict indicator, the societal major episodes of political violence measure (see Marshall 2016), was considered. Likewise, we considered a couple of civil conflict indicators from the Uppsala Conflict Data Program. However, dimensionality analyses showed that these indicators were somewhat out of sync with the other indicators or did not easily fit the country–year format of the GSoD data set.

Indicators for attribute 4: Participation

Civil Society

The measurement of Civil Society relies on six indicators. Three of these are V-Dem indicators based on expert surveys that consider the extent to which the legal and political context supports civil society organizations and activities. To these we have added indicators of the strength of interest groups and social capital from BTI, and the infrastructurally focused e-Participation Index from the UN.

Civic Engagement

A complementary measure of Civic Engagement is constructed from three expert survey indicators of the extent to which people engage in political and non-political associations, and trade unions, from V-Dem. We also include an indicator of civil society traditions from BTI, and a measure of mobilization for democracy from V-Dem. Unfortunately, potentially relevant indicators based on mass surveys, asking people about their actual involvement in civic activism, are hard to combine across surveys, which, moreover, have rather limited coverage in terms of years and countries.

Electoral Participation

The turnout of the voting age population in national elections is the single indicator used to capture electoral participation. These observational data are collected by International IDEA and capture the concept of interest in a direct way, and one indicator is therefore sufficient to measure this subattribute. Turnout as a percentage of the voting age population is used instead of turnout as a percentage of the registered voters because it is more directly comparable across countries. Registration may be relatively higher or lower between countries for many reasons, some of which are captured by the Inclusive Suffrage index.

Additional index

Direct Democracy

Outside of our conceptual framework, the GSoD Indices include a measure of Direct Democracy. V-Dem offers the only comprehensive data set in the form of the direct democracy index developed by David Altman (2016). It is based on observable variables on the formal opportunities for and actual use of different instruments of direct democracy at the national level. However, it seems pertinent to take into account whether mechanisms of direct democracy are available and used in a context where elections are generally respected as the main source of political power. To do so, we also use the Credible Elections subattribute described above as a multiplicative adjustment for the baseline level of direct democracy.

3.4. Aggregation

Deciding on the rules for aggregating the selected indicators is another key issue of index construction. This is the stage where the theoretical links between attributes and subattributes, as well as between subattributes and indicators, are translated into corresponding aggregation formulas. This section addresses a number of issues related to the task of combining the selected indicators in ways that mirror the concepts of interest. The GSoD conceptual framework is based on the assumption that the more the principles are fulfilled, the more democratic a political system is. Thus, the achievement of these principles—and each of the attributes and subattributes derived from them—is not conceived as an either/or matter, but rather as a matter of degree (cf. Collier and Adcock 1999). The measurement procedure used—that is, the construction of indices with relative, fine-grained scales with uncertainty estimates but without substantive thresholds—aligns better with this perspective than crisp distinctions.

Reflective and formative models

One of the most important questions to be asked in relation to combining different measures is whether a reflective or a formative aggregation model best captures the relationships between the indicators and the concept of interest. These general aggregation models are conceptually and substantively different (Bollen and Lennox 1991; Coltman et al. 2008).

In a reflective model, the latent variable is understood as the common cause of the indicators used to measure it. Hence, causation runs from the latent concept to the indicators. Changes in the latent trait (not directly observed) are therefore expected to cause a change in the indicator scores, but not vice versa. In a reflective model, indicators are partially interchangeable. This means that leaving any of them out of the model should not have a major impact on the meaning of the concept of interest. To illustrate, different indicators of basic welfare provision, such as the infant mortality rate, life expectancy, kilocalories per person per day and mean years of schooling, would be expected to be highly correlated with and to reflect a common latent factor (i.e. Basic Welfare). Removing one indicator from the model is not likely to change the interpretation of the core concept too much.

The assumptions behind a formative model are different. A composite variable is posited as the summary of the relevant variation in a set of indicators that are understood as constitutive of a particular concept. In other words, a composite variable is composed of variables that are individually important for the meaning of the concept. In this case, causation flows from the indicators to the composite variable. In the formative model, the indicators are understood as definitional, meaning that excluding one or more of them will fundamentally alter the meaning of the concept that is to be captured. To illustrate, contestation (or competitive elections) and inclusive suffrage are often conceived as the two essential features of representative government (see Dahl 1989; Coppedge, Alvarez and Maldonado 2008). However, they are not necessarily highly correlated with each other. Today, many countries have universal adult suffrage but not much contestation, and historically many countries had a high degree of contestation but highly restrictive voting rights. Moreover, only including indicators that capture either suffrage or contestation would critically alter the core concept that is being measured. Measuring one aspect cannot substitute for the measurement of the other aspect.

Both reflective aggregation models and formative aggregation models are used to combine the various indicators into composite GSoD Indices. When indicators of the theoretical constructs are understood as reflecting a common underlying variable and/or generally show very high levels of covariation—as indicated by factor loadings above 0.6—the aggregation procedure chosen is an item response theory (IRT) model, or Bayesian factor analysis (BFA). The distinction between formative and reflective indicators is not always easy to implement in practice and, arguably, in some cases formative indicators are treated as reflective indicators. However, in the case of very high correlations, the choice of aggregation rule generally makes less of a difference. Moreover, in many cases there is only a weak and inconsistent theoretical basis to inform the development of formative aggregation rules. Finally, the use of a similar (reflective) aggregation procedure for many of the indices reduces the complexity of the GSoD Indices. More generally, the suggested aggregation rules are justifiable but not necessarily conclusive—and users of the GSoD data set will be able to aggregate the indicators and indices in different ways if they have alternative ideas on how to match the data with their concepts of interest.

IRT modelling is used at the lowest level of aggregation (the subattribute or subcomponent level) if there is a significant amount of missing data (more than 5 per cent) in any of the indicators used to reflect the concept in question. According to Coppedge et al. (2021a: 21–22), ‘The underpinnings of these measurement models are straightforward: they use patterns of cross-rater [cross-indicator] (dis)agreement to estimate variations in reliability and systematic bias. In turn, these techniques make use of the bias and reliability estimates to adjust estimates of the latent—that is, only indirectly observed—concept’.

The use of IRT modelling techniques has a number of potential benefits. First, it allows the use of multiple indicators of the same latent concept ‘to identify and correct for measurement error, and to quantify confidence in the reliability of our estimates’ (Coppedge et al. 2021a: 11). The distribution of scores across indicators used to capture particular concepts in particular country–years provides valuable information on how much confidence can be had in each data point. If, for instance, there is a lot of disagreement between indicators about the credibility of national elections in a particular country–year, then the uncertainty about the point estimate will be high. Such uncertainty is reflected in the relatively large range of the confidence interval. (The confidence interval demarcates uncertainty as signified by the upper and lower bounds of the interval around the point estimate in which the measurement model places a fixed percentage of the probability mass for each country–year score.) Overlaps between confidence levels for index scores within a country over time or between countries at the same time indicate that the differences in point estimates are not significant. Caution is therefore required in interpreting such differences as substantial.

Second, lack of overlap in the coverage of indicators does not result in missing values in estimates for the affected country–years, as would be the case if using factor analysis. This is an important feature because although all the selected indicators have very good coverage and therefore overlap significantly, quite a few of them do not offer complete time series and/or do not cover all countries. The use of full information maximum likelihood IRT models means that all the relevant information from the indicators can be used. The gaps in some indicators are then reflected in the uncertainty estimates but if none of the indicators provide data for a given country–year, no estimate is calculated for this country–year. The latter also reflect the level of agreement between indicator scores—or the extent to which they are correlated.

On the specific type of IRT model to use, a number of different procedures have been proposed in the literature for similar situations. Pemstein, Meserve and Melton (2010) suggested one of the current options in connection with their computation of their Unified Democracy Scores (UDS)—a combination of many existing measures of democracy into a single, fine-grained estimate for each country, with uncertainty estimates. A similar option is proposed by Fariss (2014) and Fariss and Schankenberg (2014) in their work on Human Rights Protection Scores. Unfortunately, these models are highly demanding in terms of computational power. Indeed, as the authors note themselves, these kinds of models demand access to a supercomputer in order to construct just one index. A Bayesian IRT approach would therefore not be feasible given the time constraints and computational resources.

A related but much less demanding model was therefore used based on the multidimensional IRT approach implemented in the mirt package for the R statistical software by Philip Chalmers (2012).

Using mirt to compute the scores has several advantages. First, it can compute scores for all country–years—even where there is incomplete overlap in the coverage of indicators—using the FIML (Full Information Maximum Likelihood) approach. Second, mirt computes standard errors for the scores, which allows the construction of confidence intervals around the estimates. Third, estimation via mirt is not too demanding in terms of computational power. It can be done on a desktop computer and does not require access to specialized computing infrastructure. Thus, the indices are not only easier to construct but also easier to replicate. The mirt package has already been used to replicate UDS scores. The findings showed that scores computed using mirt are ‘essentially identical to those produced by PMM’s [Pemstein, Meserve and Melton’s] more sophisticated Bayesian procedure’ (Marquez 2016: 4).

The IRT model requires that the indicators measured on an interval scale must be recoded using an ordinal scale. While this rescaling obviously led to some loss of information for some variables, all the IRT methods discussed above use the same approach to rescaling interval indicators. As Pemstein, Meserve and Melton (2010: 433; see also Marquardt and Pemstein 2017) emphasize, this procedure is both more conservative and more empirically valid.

As a rule-of-thumb, ordinal variables are used without recoding. Interval scale variables that have an approximately normal distribution are recoded to ordinal scales using cut-offs at regular intervals (5 percentiles) on the original scales. For those V-Dem sourced variables that are heavily skewed, V-Dem’s own ordinal versions of the estimated values are used. In addition, in those cases where the category of an ordinal variable had less than 1 per cent of the observations, these observations are merged into an adjacent category. For further information on recoding see the Codebook (Tufis and Hudson 2025a).

If there is virtually perfect overlap in the measures to be combined (due to there being few missing data points), BFA becomes a more viable option. Compared to the IRT models, this method generally has the advantage that the measures included do not have to be ordinal variables. This means that variation is not lost and that the estimates (factor scores) of the underlying concept are more fine-grained.

Factor analysis uses information about covariation patterns between indicators to collapse several correlated, observed indicators into fewer underlying variables called factors. Simply put, the resulting factors reduce complexity by capturing variation that is common to several observed variables. Sticking to the Basic Welfare example, information on infant mortality rates, life expectancy, kilocalories per person per day and mean years of schooling can be understood by a single factor because all the indicators reflect a common underlying phenomenon (as indicated by high factor loadings and bivariate correlation coefficients), which in this case can be interpreted as the general provision of basic welfare.

BFA, like IRT models, provides point estimates for the latent dimension as well as confidence intervals, but it does so only for country–years with uniform indicator coverage. BFA was therefore used to combine subcomponent scores with subattribute scores and thereafter subattribute scores with attribute scores when applicable; that is, if the measures were expected to reflect the same latent concept in the framework and when the indicators/indices to be aggregated showed strong correlations. When indicators are understood as constitutive components of the concept of interest, the indicators are not necessarily expected to be highly correlated, which makes the use of a formative approach more plausible.

In such cases, the aggregation procedure should be based on the answers to at least two questions. The first concerns whether the attributes (or subattributes) interact. If not, then a high score on one attribute is insulated from a low score on another. If they do, then a low score on one attribute tends to drag down the score of the other. The second question concerns whether the attributes are substitutable. If so, then a low score on one can be compensated for by a high score on the other, although this should not be allowed in the case of a non-compensatory relationship. A middle option in the form of partial substitutability is also a possibility (Munck 2009: 70–71). Two formative models were used in the GSoD framework where it was judged that a particular version of this procedure was more appropriate than purely reflective procedures. Hence, a formative model is used to combine the contestation index with an inclusiveness indicator to create the Representation index; and a formative model is used to aggregate indicators related to the presence and the freeness and fairness of subnational elections.

Aggregation of indicators into GSoD Indices

Many of the selected indicators are expected to cluster in meaningful ways and to tap into a limited number of overarching concepts. These expectations rest on theoretical grounds and the findings of previous dimensionality analyses of these and related indicators, which show that many of them are highly correlated and reflect common latent attributes (see, e.g. Skaaning 2009; Møller and Skaaning 2014a, 2014b; Teorell et al. 2016). An independent assessment of empirical dimensionality was implemented by correlating the indicators selected to capture the respective subattributes (or subcomponents in relation to Civil Liberties and Political Equality) and run BFAs of the same groups of indicators.

Cronbach’s Alpha (CA) coefficients were calculated to assess scale reliability, or how closely related (internally consistent) a set of indicators is as a group. Both the bivariate correlations and the factor loadings were generally very high and thus, where this was expected, supported the indicators’ reflected common latent attributes. Moreover, the CA values indicated very strong scalability. Indicators with factor loadings lower than 0.6 were generally not included. In addition, indicators were not combined into an index if the CA value was lower than 0.8, unless they were judged as crucial because they captured an important nuance not otherwise covered or represented a different source than all other indicators combined in an index. Such exceptions to these criteria are explicitly mentioned in the main text.

Representation

Beginning with the first subattribute of Representation, Credible Elections, the indicators are very highly correlated, and the CA is 0.94. Since not all the indicators cover the full range of years included in the GSoD Indices, IRT was used to aggregate them. In a preliminary step, all the country–years without an electoral regime (according to the executive and legislative elections indicators from LIED) had their input variables recoded to the minimum value (zero). Regarding Inclusive Suffrage, the indicators were combined by taking a weighted average, where suffrage counts twice as much as voter registration since overt suffrage exclusions are more fundamental and tend to have more significant impact on who is allowed to vote than problems with voter registration that mainly have an impact at the margins even where problems are severe. The Free Political Parties indicators showed a high CA (0.88) but as they are characterized by gaps (in the Polity indicator) and incomplete coverage (BTI and Freedom House), IRT modelling was used. The Elected Government indicators similarly showed strong unidimensionality, with a CA of 0.94. As with Free Political Parties missing data in the Polity indicators and incomplete coverage in data from BTI and Freedom House meant that, once again, IRT modelling was used to aggregate the relevant indicators. Effective Parliament has fewer data sources, but gaps in Polity data require the use of IRT here, though CA remains strong (0.95). Finally, the Local Democracy index is created through multiplication of the two V-Dem source indices relating to this subattribute.

The Inclusive Suffrage index was combined with the five other subattribute indices to construct an overall Representation index, but only after the construction of an ‘intermediate’ index based on the other subattribute indices. Inspired by Dahl’s (1971, 1989; see also Coppedge, Alvarez and Maldonado 2008; Miller 2015) theoretical distinction between two attributes of representative government—contestation and inclusion—a first step uses the factor scores from a BFA to construct a contestation index (CA=0.96). Thereafter, a formative aggregation procedure combines the contestation index with the Inclusive Suffrage measure. Although contestation and inclusion are not highly correlated, they are both necessary preconditions for representative government. Accordingly, the Representation index is based on a multiplication of the suffrage scores and the normalized scores for the contestation index.

Rights

All the indicators linked to Access to Justice show strong unidimensionality and scalability (CA=0.87). Since some of the indicators have incomplete coverage, the indicators were combined using IRT. Regarding the second subattribute of Rights, Civil Liberties, the indicators for Freedom of Expression, Freedom of the Press, Freedom of Assembly and Association, Freedom of Religion, and Freedom of Movement are highly correlated and express high scalability (CA between 0.89 and 0.96). While the V-Dem indicators and the CLD indicators have near to full coverage there is incomplete coverage in indicators from BTI and Freedom House. In addition, the ICRG conflict indicator has a significant proportion of missing observations. All five subcomponents were therefore constructed using IRT modelling. In the next step BFA was used to reduce the highly correlated subcomponents into a single index score for the Civil Liberties subattribute (CA=0.97).

The indicators linked to Basic Welfare also reflect a common underlying attribute and strong scalability (CA=0.94). Thus, they were aggregated using a reflective model. More specifically, an IRT model was used because several indicators (notably the BTI indicator) do not have complete temporal or geographic coverage.

Turning to the Political Equality subattribute, many indicators of Social Group Equality had high factor loadings. Due to incomplete coverage in the indicators from BTI and Freedom House, which are considered too valuable to exclude because they capture important aspects not sufficiently captured by the other indicators, IRT was used to construct the social group equality index (CA=0.89). The Economic Equality index similarly has incomplete coverage in the indicator from BTI, and in the Gini coefficients. Therefore, IRT was used to aggregate this index (CA=0.89). Some of the Gender Equality indicators, primarily those on gender gaps in labour force participation, managerial positions, and control of financial accounts, had similarly significant gaps. These indicators showed relatively low factor loadings of 0.54, 0.50 and 0.51, respectively. Nonetheless, they tap into highly relevant features and the CA was still 0.86, so they were all used to construct the gender equality index using the IRT model. The indices for Social Group Equality, Economic Equality and Gender Equality showed a very high CA (0.94). They were combined into an overall Political Equality index using BFA.

A high covariation was found between the four Rights subattributes (factor loadings of between 0.72 and 0.95). Against this backdrop, and the widespread understanding that these core human rights go hand-in-hand—as stated in the Vienna Declaration (UN Office of the High Commissioner for Human Rights 1993; see also Whelan 2010)—BFA was used to create a Rights index (CA=0.92).

Rule of Law

Four reflective indices were constructed under the Rule of Law attribute: Judicial Independence (CA=0.92), Absence of Corruption (CA=0.92), Predictable Enforcement (CA=0.92), and Personal Integrity and Security (CA=0.82). IRT was used to estimate the values of each of these subattributes due to significant missing data or incomplete coverage in some of the indicators. The unidimensionality of the indicators grouped with Absence of Corruption and Predictable Enforcement was somewhat less pronounced—especially due to the lower correlations between the V-Dem indicators, on the one hand, and the indicators provided by the ICRG, on the other. Nonetheless, all the factor loadings within these indices are still at least 0.70 and the respective scalability values were high (as noted above). The association between the different Rule of Law subattributes is also high: the lowest bivariate correlation is 0.76 and the lowest factor loading in a single-dimensional BFA model is 0.87. These indices capture closely related concepts so it is natural that they show a strong association. Hence, a Rule of Law index was constructed based on BFA scores (CA=0.95).

Participation

With regard to the Participation subattributes, no aggregation was needed for Electoral Participation (since only one indicator is being used). The other two subattributes suffer from incomplete coverage in the indicators from BTI and are therefore estimated through IRT models. The six indicators on Civil Society (three from V-Dem, two from BTI, and the UN e-Participation Index) are of quite different types, but clearly tap into a common latent dimension and they were therefore aggregated into an index (CA=0.82). Finally, the Civic Engagement subattribute includes three highly correlated measures of engagement from V-Dem, a measure of civil society traditions from BTI, and a less-well correlated but theoretically important measure of mobilization for democracy from V-Dem (CA=0.75). The CA would be a little higher without the measure of mobilization for democracy (0.79), but it rounds out the concept beyond what is measured by the other indicators, and therefore the loss in precision in the estimates is worth accepting for the sake of having a more complete measure of the concept.

It was less straightforward to create a single index for Participation compared with the other attributes. The three subattributes are all theoretically related and do empirically tap into a single concept of democratic Participation. However, the CA is relatively weaker than we would like at 0.72. This is due to the weaker association that Electoral Participation has with the other two subattributes. Furthermore, gaps in the data for Electoral Participation required the use of an IRT model for this index. On balance, however, we consider the centrality of Electoral Participation to democracy to be a strong enough reason to accept the relatively weaker empirical association between these subattributes and move forward with the single attribute-level index.

3.5. Summary

Table 3.2 summarizes the aggregation procedures used in the construction of indices at the different levels. All the indices at the different levels have been normalized to range from 0 (lowest achievement) to 1 (highest achievement). A score of 0 refers to the worst performance in the entire sample of country–years covered by a particular index, while a score of 1 refers to the best country–year performance in the sample.

For a number of indices, however, 0 has an absolute meaning in addition to a relative meaning. The voter turnout index has substantively meaningful minimums and maximums. More particularly, 0 refers to the full absence of Inclusive Suffrage or voter turnout, while 1 refers to universal adult suffrage and 100 per cent voter turnout. The subattribute indices capturing Credible Elections, Elected Government, Direct Democracy and Local Democracy also have substantively meaningful minimum values that refer to agreement between all the indicators about the total absence of their respective features.

For most indices, the yearly scores for each country are accompanied by uncertainty estimates, which can be used to assess whether differences between countries and within countries over time are significant. These uncertainty estimates are in the form of confidence intervals (margins of error) and reflect the statistically likely range for the country–year index scores based on the indicators used. The GSoD Indices confidence levels refer to one standard deviation below and above the estimated score. This means that about 68 per cent of the ‘true’ values would be found within these intervals.

Confidence intervals are only available for indices based on multiple indicators. The more the underlying indicators are in agreement regarding the scoring (high–low) on a particular aspect of democracy, the narrower the confidence levels are. If the confidence levels overlap when comparing the scores for two or more countries on the same GSoD index, the difference between the scores is not statistically significant. Similarly, overlapping confidence intervals for different years when comparing the scores of one country for a particular GSoD index also indicate that the difference is statistically insignificant. More generally, short-term fluctuations—especially very recent ones—are hard to capture well and should be interpreted with caution, while it is usually possible to be certain about longer-term trends.

3.6. Interpreting the indices

The GSoD Indices are interval level indicators that enable a nuanced measurement of democracy. Since an interval-level measure provides a potentially infinite number of scores differing from adjacent scores, it is not possible to link each individual score to nuanced verbal assessments capturing the empirical reality in a country. To translate the scores into such assessments, numerical thresholds are needed and useful. Such thresholds help situate a country’s specific score within ranges corresponding to meaningful verbal labels. Since each index is rescaled to vary between 0 and 1, International IDEA has decided to define the scores of 0.4 and 0.7 as thresholds that distinguish levels of performance on attributes.

If a country’s score exceeds 0.7, its performance is labelled ‘high’. Scores below 0.4 correspond to ‘low’ performance. Scores between 0.4 and 0.7 classify a country’s performance as ‘mid-range’. These numerical thresholds distinguish few broad categories and are identical across attributes. The advantage of such a distinction is that it creates relatively simple, but still meaningful, classifications.

Since 0.4 and 0.7 are absolute thresholds rather than percentiles, they correspond better to the notion of distinct, crisp categories implied by a classification. Absolute thresholds are not completely unaffected by the empirical distribution of scores. The normalization of index scales and the distribution-based recoding of country scores required for IRT modelling imply that a data set update may modify the scores of countries if, for example, the maximum or minimum scores of an indicator change.

In contrast with percentile-based thresholds, absolute thresholds make a country’s classification less dependent on the performance of other countries. However, the thresholds should be applied with caution since they create hard distinctions between countries that may have only small score differences and may resemble each other strongly.

This chapter presents a brief comparison with existing measures that attempt to capture relatively similar aspects of democracy at the attribute, subattribute or subcomponent levels (see Coppedge et al. 2017; Skaaning 2018). For detailed evaluations of the advantages and disadvantages of these data sets see, among others, Arndt and Oman (2006), Coppedge et al. (2011); Landman and Carvalho (2009); Møller and Skaaning (2014a); Munck (2009); UN OHCHR (2012); and Skaaning (2009). These measures are taken from eight large-scale data sets on democracy, governance and human rights (see Table 4.1).

Compared to the GSoD Indices, it is clear from Table 4.2 that the information provided is quite similar, in terms of the coverage, to Freedom House; and that Polity and V-Dem cover substantively more years. The other data sets in the overview do not go back to the beginning of the third wave of democratization or even the end of the Cold War, however, which makes their ability to capture trends rather limited.

The overview also demonstrates considerable variation in the kind of data sources on which the data sets are based. Half of them only use one type—either in-house coded indicators or expert surveys—while the others use two, three or even four types. Expert surveys are used most and public opinion surveys least. With three exceptions, the data sets rely on indicators from just a single data provider: their own data collection. In addition, far from all of them complement their scores with uncertainty estimates, although all of them at least partly rely on judgement-based data collection procedures.

On the listed parameters, the GSoD measures are most similar to the V-Dem measures, as they are based on in-house coded and expert-coded data as well as observational data but not public opinion surveys; and they provide fine-grained scores that are supplemented by uncertainty estimates. The major difference is that V-Dem builds on its own original data collection, whereas the GSoD Indices make use of extant indicators from various data providers.

It is standard practice to evaluate new measures by correlating them with other, more established measures. If the extant measures are valid, high correlations (at all levels of aggregation) with extant measures from the data sets listed above indicate that the GSoD Indices are also valid. If the extant measures are based on similar or the same information, high correlations also indicate that the GSoD Indices are reliable. Almost all the correlations are either high or very high (above 0.7). However, all the correlations, including the high ones, should be interpreted with caution, since none of the extant measures are perfect and many of them capture slightly different concepts compared to the GSoD Indices.

The GSoD Indices can be used to assess cross-country differences and similarities and to identify trends at the country, regional and global levels over time. Users are advised not to collapse the scores for the individual attributes into one single democracy index as a disaggregated perspective provides more nuanced information and because such an exercise needs to be grounded in careful theoretical reflections.

Furthermore, it is not recommended that the Indices be used to carry out impact assessments of specific policy reforms or democracy promotion initiatives. Despite disaggregation, they are often too abstract to be useful for suggesting concrete policy reforms, which should rather be informed by detailed and context-specific evaluations of opportunities and constraints. In relation to the main data sources that the GSoD Indices draw on, the release of version 15 of the V-Dem data (2025) was followed by a cautionary note:

The V-Dem Methodology assumes five or more coders for the “contemporary” period starting from 1900, originally coded to 2012. With the updates covering 2013–2024 it has for a few country–variable combinations been impossible to achieve that target. From analysis, we have found that this at times can result in significant changes in point estimates as a consequence of self-selected attrition of Country Experts, rather than actual changes in the latent state of a given country.
(Coppedge et al. 2025)

Therefore, caution should be exercised when drawing conclusions for the period 2013–2024 using the GSoD Indices that rely heavily on data from the V-Dem expert survey. In practice, this problem tends to be more pronounced for democracies in North America and North and West Europe, where the scores for some indicators and countries have tended to be dragged down towards the global mean for methodological rather than substantial reasons. Although these changes are generally not statistically significant, they could signal a downward trend but they could also be a methodological artefact.