Benchmarking is a helpful tool often used by organizations to compare their business processes and practices with competitors or best in class, giving them something external against which to measure performance. As the Senate Administration continues to develop its performance measurement framework, directors are encouraged to include external benchmarks in their performance indicators to provide comparative measured. Benchmarks are categorized in multiple ways, which can be further characterized as qualitative or quantitative. This article will provide examples of qualitative and quantitative benchmarks in the parliamentary context.
To establish qualitative benchmarks for improvement purposes, one must first identify what the elements of success or effectiveness look like, i.e., a profile of best practices against which to compare. Quantitative benchmarking looks mainly at the numbers or ratios of an operation. To improve a process or operation, it is best to have measures that indicate where improvements can be made, so both types of benchmarks often prove useful.
International Efforts at Qualitative Benchmarking of Legislatures
Qualitative benchmarking on the democratic well-being of legislatures is being used by various institutions worldwide to gauge strengths and weaknesses and to create strategic plans for democratic reform. In 2009, strategic planners from the House of Commons and the Senate were invited by our respective Clerks to prepare an initial self-assessment against benchmarks created by the Commonwealth Parliamentary Association (CPA)1. There were 87 benchmarks to assess, relating to everything from the rules of the legislature to non-partisanship of staff, and an evaluation was straightforward in most instances. These evaluations were then validated by our respective procedural advisors in the Chamber Operations and Table Research Offices of the Senate and the House of Commons. Some benchmarks, either too political or too subjective, were left for the parliamentarians to assess. Ethical governance was one such section: ethics are the foundation of everything that we do, and the tone starts at the top. Staff thus deferred the ethical governance ratings to parliamentarians themselves. Parliamentarians discussed the benchmarks and rated them individually, taking averages to determine a single rating for each benchmark.
Some months after contributing to this exercise, I was asked by the Commonwealth Parliamentary Association to share our experience with the self-assessment exercise at the International Conference on Benchmarking and Self-Assessment for Democratic Parliaments, hosted by the World Bank Institute in Paris. At least ten organizations made presentations on their benchmarking experiences, either as creators and administrators of the benchmarks or as participants in benchmarking exercises.2
The various sets of benchmarks that a country may choose from to self-evaluate all contain a significant degree of overlap. For instance, the benchmarks used by the CPA and the Assemblée parlementaire de la francophonie (APF) employ nearly identical wording in some cases. However, benchmarks can also be tailored to assess targeted characteristics. The CPA, for example, has developed a separate set of benchmarks for application to Pacific Island Democratic Legislatures, which cover additional concerns such as sufficient remuneration and adequate resources for the drafting of legislation. Among the many that exist, certain sets of benchmarks may concentrate more fully on particular democratic aspects, making them conceivably more attractive depending on whether a legislature may score better on one set as opposed to another.
Several sets of benchmarks were silent on the external sources for election funding or for parliamentary activities, which seems an obvious and important benchmark on the “democraticness” of a legislature. The varieties of benchmarks available are explained as being the result of the many countries involved and the unique history and culture of each. A presenter from the Parliamentary Centre expressed what seemed the widely held opinion that it was difficult to come up with a single set of benchmarks ideal to all. This seemed to overlook the significant overlap, which in itself may indicate that a core set of internationally accepted benchmarks could be developed. One conference participant asked at the end of my presentation on the Canadian self-assessment experience whether or not we had “shopped around” for the most amenable set of benchmarks, to which of course the answer was “no”.
While countries of all sizes and at all stages of development were invited to participate, the countries targeted for these benchmarking self-assessments tend to be smaller nations in receipt of development assistance. Speakers from US Aid and the National Democratic Institute claimed that a nation’s democratic well-being, as may be assessed using the benchmarks, was inextricably linked to its economic well-being although no specific evidence was presented to substantiate this. Canada’s self-assessment for the CPA was applauded, in particular by the Americans present, and it appeared that we might have been the only industrialized nation or “middle power” that had complied with the request to self-assess. We had set a wonderful example to smaller nations.3 But beyond the setting of an example, what was the value of the exercise?
Canada’s Benchmarking Self-Assessment for the CPA: A case study
The CPA benchmarks included general aspects such as candidate eligibility, immunity, and remuneration; as well as sections on procedure and sessions; and more specialized benchmarks on committees, political parties and party groups, parliamentary staff, the legislative function, the oversight function, the representational function, and accessibility. The CPA did not impose an assessment rating system, so we developed a simple 5-point scale, with 5 indicating that our Parliament fully meets the benchmark, a rating of 4 as partially meeting the benchmark, of 3 as currently developing processes to implement the benchmark, 2 as reviewing the potential application of the benchmark, and 1 as no current plan to meet the benchmark. All ratings of less than 5 were accompanied by an explanation.
While the majority of the 87 benchmarks rated clearly at 5, there were 10 occasions where one or the other of our two federal houses did not fully meet the standard, meriting a rating of 4. The benchmarks sometimes made reference to the existence of a formal codification of the standard or principle, and in some instances where Canada was fully compliant in spirit, but no codification existed, a rating of 4 was attached. An example of this was the benchmark, “Special measures to encourage the political participation of marginalized groups shall be narrowly drawn to accomplish precisely defined, and time-limited, objectives.” This is not something undertaken by Parliament or Elections Canada, but rather by individual political parties, which have much to gain in Canada’s cultural mosaic by applying this standard.
A rating of 4 also resulted when one or the other house was not in compliance with a benchmark. For example, the Senate Speaker is not elected by peers following rules of procedure, but is appointed by the Governor General on the advice of the Prime Minister. Another 4 rating noted that neither Senate nor Commons committees can summon officials from the executive branch, although the practice is that they usually agree to appear.
No ratings of 3 or 2 were assessed, although there were two flagrant occasions of a 1, indicating little to no plan to comply with the benchmark. The non-compliant ratings of 1 are of particular interest, and merit further explanation. The first rating of 1 was perhaps indicative of the age of our constitution, which provided full powers to the Senate – an unelected body – to reject a money bill. This, in fact, has never happened. The other rating of 1 reflected that our Parliament has no right to override an executive veto, while in fact the Crown’s veto powers have never been exercised in practice at the federal level.
The array of benchmarks where the Parliament of Canada scored less than full compliance did indeed create a checklist for any serious debate of modernization or democratic reform. This is not to say that we do not enjoy a healthy democracy, only that the CPA benchmarks provided a summary of best practices against which certain shortcomings became evident. As noted in the worthwhile introduction to the IPU publication Evaluating Parliament: A Self-Assessment Toolkit for Parliaments, “Democratization is not a one-off event, but a continuing process, in both recent and long-established democracies.” All legislatures should periodically review the international benchmarks for democratic legislatures as an opportunity for improvement and growth.
Quantifying Qualitative Benchmarks
The CPA did not deliberately attempt to quantify its benchmarks, perhaps to avoid the potential pitfalls of assigning values to qualities and to reassure the participating parliaments that the assessment results really were for their own, internal use. For comparative purposes, however, it may be preferable to devise a rating system for a set of benchmarks, as the Inter-Parliamentary Union has done. The IPU Toolkit requires assessors to make a value judgment on each question, by rating the legislature from “5 = very high/very good” to “1 = very low/very poor”. The benefit of quantifying these ratings is that evaluators may be able to discern shifts of opinion from one interest or stakeholder group to the next and, in some cases, determine the reasons behind them, allowing for a fuller assessment.
Some of the pitfalls of quantifying qualitative benchmarks include selecting the right criteria; ensuring that there are no gaps, particularly if benchmarks are rated as part of a composite; setting a value scale that accounts for significant partial compliance with a benchmark (i.e. yes/no insufficient); and determining an appropriate weighting of the benchmarks. Professor M. Steven Fish of the University of California at Berkeley states that, “The strength of the national legislature may be a—or even the—institutional key to democratization.”4 He and Professor Kroenig developed a Parliamentary Powers Index based on 32 benchmarks, by which they scored national legislatures (postcommunist nations in this study) “simply by dividing the number of affirmative answers by the total number of questions.” A score of one would denote compliance with all criteria. These scores, in turn, were compared with Freedom House survey scores ranging from “most free” to “least free”. The correlation between the scores was established, although which was causal (if either) remained inconclusive.5
The “Fish-Kroenig Legislative Powers Survey” includes a number of benchmarks that seem to beg a challenge, such as:
- “13. The legislature’s laws are supreme and not subject to judicial review” (not suitable in the land of the Charter);
- 17. Members of the legislature are immune from arrest and/or criminal prosecution” (popular in France and Italy);
- 19. The legislature alone, without the involvement of any other agencies, can change the constitution” (Russia comes to mind).
The fact that there are only 32 benchmarks suggests that there may be serious gaps. Also problematic is the equal weighting of the benchmarks, which means that “25. The chairman of the central bank is appointed by the legislature,” or “28. Each legislator has a personal secretary,” scores the same as “12. Laws passed by the legislature are veto-proof”. Furthermore, the lack of ability to assign value for partial compliance would leave Canada, for example, with a score of zero for such benchmarks as 13, 17 and 19. While there is value in the exercise, one might question the validity of the quantification of the results.
Qualitative Benchmarks on Other Aspects of Legislatures
The importance of our legislatures to democracy was the focus of the Paris conference on benchmarking democratic legislatures. The excellent assessment criteria used by the participants of the conference should not, however, limit our thinking about other possible qualitative benchmarks for democratic legislatures. The benchmarks identified perhaps overlooked some of the most fundamental principles of healthy democracy, such as the freedom of legislators to represent the electors effectively, without the threat of party discipline, or the importance of their constituency work.
In recent years, Canadian academics studied our democratic performance in a significant, five-year project entitled the “Canadian Democratic Audit”, which examined “several key decision-making bodies, including legislatures, the courts, and cabinets and governments … [using] the benchmarks of participation, inclusiveness and responsiveness to structure their assessment of a particular institution or component of Canadian democracy.”6 A contributor to the audit, David C. Docherty of Sir Wilfrid Laurier University has asked “Are we comfortable that we are being properly, and democratically, represented? Apparently not, for legislatures at the provincial and national levels seem to be wallowing in new lows of public approval ratings.”7
Docherty thoroughly reviewed these three benchmarks of participation, inclusiveness and responsiveness, and considered that, “The lack of internal responsiveness currently appears to be at an all-time low Not only are decisions becoming more centralized but central actors perceive no need to consult or respond”. On inclusiveness, he noted that informal barriers affect who is elected to office in Canada, and that while there is an “overall trend toward a more inclusive Parliament … the increasing proportion of minority MPs is far lower than the increasing proportion of minority Canadians.” Regarding participation, Docherty’s audit discussed the lack of opportunity for members to use their talents in Canadian legislatures, concluding among other recommendations that continued committee reform will provide members with the best opportunity for improved participation. These three benchmarks, then, have the potential to tell an important story on how democratic our legislatures are. The need to consider the three benchmarks that Docherty assessed has since been validated by the results of our last federal election in October 2008, when voter turnout fell to an all-time low of 58.8 percent. The Conference Board of Canada and others have noted that low voter turnout implies that the democratic system may not be reflecting the interests of all citizens. While the Canadian Democratic Audit chose broad benchmarks under which much data can be rolled up, they serve as strong guiding principles to determine the health of our democracy.
Other benchmarks that warrant consideration should be developed for parliamentary support functions, such as procedural support and legislative drafting, as well as administrative services. Parliamentarians and officials do now, in fact, engage in informal forms of benchmarking whenever we share best practices with colleagues from other legislatures at home and abroad. Our natural inclination to use comparison to improve performance becomes evident when we consider the opportunities we provide ourselves to share information and best practices through networks such as the interparliamentary assemblies, and at the procedural/management level through the Society of Clerks-at-the-Table in Commonwealth Parliaments, the Canadian Clerks-at-the-Table Society, or the Canadian Association of Parliamentary Administrators.
Quantitative Benchmarking of Legislatures
The best examples of quantitative benchmarking of legislatures in Canada are to be found in the work of Robert J. Fleming. Dating back to 1979 when Mr. Fleming was the Director of Administration at Ontario’s Legislative Assembly, he compiled “A Comparative Study of Administrative Structures of Canadian Legislatures” that covered the administrative organization of the provincial and territorial legislatures. He also provided the details of members’ compensation and services, including the various allowances they receive: mailing and telephone privileges; support services; and so on. Mr. Fleming’s early study did not provide any analysis of the data, but one can draw conclusions from it regarding which legislatures were best resourced.
In his publication Canadian Legislatures 1992: Issues, Structures and Costs, Mr. Fleming includes various essays in Parts I and II with a wealth of comparative data in Part III, much of which constitutes quantitative benchmarks, including such interesting calculations as the per capita costs of legislatures and comparisons of members’ remuneration. Interestingly, the least expensive legislature was (and continues to be, see Table 1), the Senate at $1.61 per capita, with the next lowest in 1992 being the House of Commons at $8.49 per capita. In the singular environment of legislatures, this type of quantitative benchmarking could serve to highlight any gross anomalies where legislatures may be costing far more than their counterparts, which could merit further investigation.8
Quantitative benchmarking does have the potential to tell interesting stories. To illustrate, the following tables provides a few numbers that may lead to some preliminary observations, some of which would need to be validated with further research.
Table 1 shows a number of interesting facts about Canada’s legislatures. The Senate – Canada’s only appointed chamber – remains its most economical at a per capita cost of only $2.79 (therefore ranked 15th in last column) and spends only one fifth per capita the amount of its federal partner, the House of Commons (this must be compared in the context of the latter having almost three times the number of members, and that MPs must maintain constituency-based offices). The House of Commons now ranks behind the Ontario Legislature for lowest per capita spending. Highest per capita spending results in the territories, and is most likely related to the size and difficulties of transportation in remote areas with low populations (higher costs per passenger). A further correlation from Table 1 data is that budgets increase with population size and (not included) GDP.
|Table 1 – Expenditures by Legislatures 2009-2010|
|Legislature||Budget||Members||Spending per Member||Population||Spending per Citizen|
|House of Commons||$440,312,000 (1)||308||$1,429,584||33,327,300||$13.21|
|British Columbia||$74,279,000 (5)||85||$873,871||4,383,800||$16.94|
|Nova Scotia||$22,077, 000 (9)||52||$424,558||936,600||$23.57|
|Newfoundland and Labrador||$16,062,100 (10)||48||$334,627||506,400||$31.72|
|Northwest Territories||$15,405,000 (11)||19||$810,789||43,700||$352.52|
|New Brunswick||$13,236,000 (12)||55||$240,655||747,100||$17.71|
|Prince Edward Island||$4,487,900 (15)||27||$166,219||139,500||$32.17|
By dividing the number of legislators by the respective population they represent, we see that the smaller the provincial population, the more representative members are of their individual citizens because the member:citizen ratio grows inversely. Further, Table 2 shows that voter participation is highest in provinces and territories with low populations. In conclusion, a lower member:citizen ratio seems to correlate to increased responsiveness and higher voter turnout, which would support Docherty’s argument on the importance of responsiveness to democratic health. Only Saskatchewan, Manitoba and Prince Edward Island have increased their voter participation in their last elections.
|Table 2 – Voter Participation in Federal, Provincial and Territorial Elections|
|Legislature||Members||Population||Member: Citizen Ratio||Voter Turnout||Turnout (+/-)|
|House of Commons||308||33,327,300||1:108,206||58.8%||-5.9%|
|Newfoundland and Labrador||48||506,400||1:10,550||60.20%||-12.32%|
|Prince Edward Island||27||139,500||1:5,167||83.84%||+0.57%|
* Currently the only elected senator, the Honourable Bert Brown of Alberta was selected in the Senate Nominee Election held by the province of Alberta on November 22, 2004. Voter turnout in this election was 44.2%; however, 9.7% of participating electors declined their ballot, and 9.6% of the ballots cast were rejected. The valid votes cast represent a turnout of 35.7% (http://www.elections.ab.ca/Public%20Website/589.htm).
** Data on turnout for the 1998 Senate Nominee Election held by the Province of Alberta on October 19, 1998 is not available on the Elections Alberta website.
Voter turnout based on figures from Elections Canada and corresponding provincial and territorial election offices.
Other interesting figures to track, for which we were unable to find sufficient consistent data for all provinces and territories, might include employees per legislature (may or may not include political staff), to calculate the staff:member ratio; average travel expenses per member in relation to average constituency size in square kilometers; or average office expenditures per member. To illustrate how these figures might be used, our research showed that some legislatures average less than one staff person per member, while others are averaging as many as six staffers per member, not including political staff. Such data could be used by understaffed legislatures to make a business case for more resources.
Parliament may be a historic institution, but to remain relevant and healthy in the 21st century, its practices and systems must keep pace with those of the society it serves and, of almost equal importance, must be recognized as doing so. Benchmarking as a tool for assessment and renewal can contribute to a restoration of confidence in the stewardship of our parliamentary institutions. Benchmarking as a tool to assess how legislatures compare and set goals to build their democracies is nothing less than inspiring. As a well-established democracy, Canada’s Parliament should not fail to challenge itself to improve. The Minister of State for Democratic Reform and the internal procedural committees of our legislatures could supplement their agendas and set new, meaningful goals using benchmarks to contribute to their critical analysis. At the administrative level, a framework of benchmarks should be established, shared and integrated into our management practices
1 . Benchmarks for Democratic Legislatures, A Study Group Report, Commonwealth Parliamentary Association, December 2006. www.cpahq.org
2. Development agencies such as the World Bank Institute, the United Nations Development Program and the National Democratic Institute are working closely with interparliamentary assemblies such as the Commonwealth Parliamentary Association (CPA), the Inter-Parliamentary Union (IPU), the Assemblée parlementaire de la francophonie (APF), the Southern African Development Community Parliamentary Forum (SADC PF), and Parlatino (of Latin America) to develop democratic benchmarks for use by their respective member institutions. Some of these agencies further assist legislatures with the articulation and implementation of strategic plans for democratic reform based on the benchmarking assessments.
3. It merits mention that other nations, especially those working with international organizations, undergo far more rigorous benchmarking assessments than the one conducted in Canada’s federal Parliament, often surveying multiple stakeholder groups to determine varying degrees of perception on performance against the benchmarks, such as the public, the media, parliamentary staff and parliamentarians themselves.
4. Steven M. Fish, Stronger Legislatures, Stronger Democracies, Journal on Democracy; Vol. 17, No. 1, January 2006; National Endowment for Democracy and the Johns Hopkins University Press.
5. Professors Fish and Kroenig are also co-authors of The Handbook of National Legislatures: A Global Survey. New York: Cambridge University Press, 2009. This book assesses the national legislatures of 158 countries against the 32 benchmarks in the Fish-Kroenig Legislative Powers Survey.
6. William Cross, The Canadian Democratic Audit; Carleton University; Paper prepared for the 20th IPSA World Congress, Fukuoka, 10-13 July 2006.
7. David Docherty, Legislatures; Canadian Democratic Audit Series, UBC Press; 2005; p. 3.
8. Robert J. Fleming, Canadian Legislatures 1992: Issues, Structures and Costs, Global Press, 1992.