1

The Common Assessment FrameworkThe Common Assessment FrameworkCAFCAF

Patrick StaesPatrick StaesEuropean Institute of Public AdministrationEuropean Institute of Public Administration

CAF Resource CentreCAF Resource Centre

22asas Jornadas de calidad y mejora de las Jornadas de calidad y mejora de lasadministraciones padministraciones púúblicasblicas

Pamplona, 3 and 4 November 2004

2

ContentContent

1. TQM and the Public sector1. TQM and the Public sector

2. The Common Assessment Framework2. The Common Assessment Framework

3. The process of self assessment3. The process of self assessment

4. Lessons learned4. Lessons learned

5. Applications so far5. Applications so far

6. Further developments6. Further developments

3

1.Total 1.Total QualityQuality Management Management andand thethepublic sectorpublic sector

Permanent mobilization of all the resourcesPermanent mobilization of all the resources(especially the personnel) to improve in a(especially the personnel) to improve in acontinuous way:continuous way:all the aspects of the functioning of anall the aspects of the functioning of an

organisationorganisation

the quality of goods and servicesthe quality of goods and services

the satisfaction of its stakeholdersthe satisfaction of its stakeholders

its integration into the environmentits integration into the environment

4

TaxpayerQuality return

PeopleQuality of life

Citizen/client

Quality of products

and services

Trinity

of the totalquality

Trinity

Total

Quality

Environment

In the publicsector

5

Satisfaction of thecitizens/clients

Satisfaction ofthe people

Satisfaction of theelected

Motivatedcollaborators

Excellent services Less direct complaints, parliamentaryquestions, interventions of the

ombudservices

Higher effectiveness ofpublic policies

Investments in people Investments in favourof the productivity

6

The TQM models for the private sectorThe TQM models for the private sector

Deming PrizeDeming Prize for Quality Management (Japan, for Quality Management (Japan,1970): award: 2x yield than the average1970): award: 2x yield than the average

Malcolm BaldrigeMalcolm Baldrige Award (USA, 1987): award: Award (USA, 1987): award:5x yield than the average5x yield than the average

European Quality Award (Europe, 1992)European Quality Award (Europe, 1992)

EFQM model EFQM model

European Foundation for Quality ManagementEuropean Foundation for Quality Management

7

The TQM models for the Public Sector in EuropeThe TQM models for the Public Sector in Europe

EFQM – Public sector: since 1998: Public sector Award

2000 : Inland revenue, Accounts office Cumbernauld (UK, Schotland)

2001: St Mary College Northern Ireland : 2QC Copenhagen 2002

2002: Customs and Tax Region aarthus

2003: Runshaw College (UK)

Speyer model

Common Assessment Framework (CAF)

8

2. 2. TheThe CommonCommon Assessment Assessment FrameworkFramework

TheThe principlesprinciples ofof Total Quality Management Total Quality Management

fromfrom thethe EuropeanEuropean FoundationFoundation for Quality Management for Quality Management(EFQM )(EFQM )

fromfrom thethe Speyer Speyer InstituteInstitute ( Germany ) ( Germany )

designeddesigned by by thethe I P S G I P S G ((InnovativeInnovative Public Services Group) Public Services Group)

TheThe expert group expert group reportingreporting to to thethe DGs DGs responsibleresponsible

for for thethe public administration in public administration in thethe MS MS ofof thethe EU EU

together withtogether with E I P A E I P A ((EuropeanEuropean InstituteInstitute for Public Administration) for Public Administration)http:\\www.http:\\www.eipaeipa..nlnl

resulting in resulting in thethe C A F C A F

9

The launching of CAF

LaunchedLaunched at the at the 1st 1st EuropeanEuropean Quality Conference, Quality Conference, MayMay 2000, Lisbon2000, Lisbon

PresentationPresentation of the CAF 2002 of the CAF 2002 versionversion at the at the 2nd 2nd EuropeanEuropean Quality Quality

Conference, Conference, OctoberOctober 2002, 2002, CopenhagenCopenhagen

DiscussedDiscussed at the at the 3d 3d EuropeanEuropean Quality Conference Quality Conference, September 2004, September 2004

Rotterdam Rotterdam

UsedUsed in different in different EuropeanEuropean CountriesCountries in the context of in the context of theirtheir nationalnational

Quality ConferencesQuality Conferences (Belgium, Denmark, Finland, Norway, Germany, (Belgium, Denmark, Finland, Norway, Germany,

Portugal, Italy etc.)Portugal, Italy etc.)

10

Objectives of the CAFObjectives of the CAF

Introduction to the TQM, especially adapted for theIntroduction to the TQM, especially adapted for thePublic SectorPublic Sector

Serve as a Serve as a « « bridgebridge » » between the various models between the various modelsin usein use

Facilitate benchmarking (comparative study of theFacilitate benchmarking (comparative study of theperformances and exchange of good practices)performances and exchange of good practices)

Self-evaluation of an public organisation (entitySelf-evaluation of an public organisation (entitywith sufficient autonomy: e.g. business unit)with sufficient autonomy: e.g. business unit)

11

The CAF ModelENABLERS RESULTS

INNOVATION AND LEARNING

HRM

Strategy andplanning

Partnershipsand resources

Process and change

management

Customer/citizen-oriented

results

People results

Society results

Key performance

resultsLeadership

The CAF ModelENABLERS RESULTS

INNOVATION AND LEARNING

HRM

Strategy andplanning

Partnershipsand resources

Process and change

management

Customer/citizen-oriented

results

People results

Society results

Key performance

resultsLeadership

12

The collaborators

The citizen/client

The society

dependvery much The leadership

The strategy and the planning

The Human ResourcesManagement

The partnerships and theResources

The processes

The results obtained by

The harmonious interaction between all these elements delivers excellentresults in the field of the key performances of the organisation

Logics of the model

on the role played by

The CAF model shows that

13

LEADERSHIPLEADERSHIP

STRATEGY&

PLANNING

STRATEGY&

PLANNING

HUMANRESOURCES

HUMANRESOURCES

PARTNERSHIPS&

RESOURCES

PARTNERSHIPS&

RESOURCES

PROCESSESPROCESSES

3 to 6 sub criteria

Fields of attention:The exemples

(not compulsory)

= Good Practices

1.21.3

1

2

3

4

5

1.4

1.11.2

CAF Factors LeversCRITERIA

14

1. Leadership

1.1. Give direction to the organisation: develop and communicate vision,

mission and values

1.4. Manage the relations with the politicians

and other stakeholders

1.3. Motivate and support the people in the organisation

and act as a role model

1.2. Develop and implement asystem for managing the

organisation

How the leaders Results 6-7-8-9

CAF - criterion 1

15

2. Strategy &

Planning

2.2. Develop, review and update strategy and planning

2.3. Implement strategy and planning in the whole organisation

2.1. Gather information relating to present and future needs of stakeholders

9. Key performanceResults

CAF - criterion 2

16

CAF - - criterion 3 3

3. HRM

3.3 Involve employees by developing dialogue and empowerment

3.2 Identify, develop, and use competencies of the employees, aligning individual, team

and organisational targets and goals

3.1. Plan, manage and improve human resources with regard to

strategy and planning

7. People results

17

CAF - criterion 4

4. Partnerships andresources

4.2. Develop and implement

partnerships with the customer/citizen

4.1. Develop and implement key partnership relations

4.3. Manage knowledge

4.4. Manage finances

4.5. Manage technology

9. Key performanceresults

4.6. Manage buildings and assets

6.Citizen/customer oriented results

18

5. Process andchange management

5.1. Identify, design, manage and improve processes

5.2. Develop and deliver servicesand products by involving

the customer/citizen

5.3. Plan and manage modernisation and innovation

9. Key performanceresults

CAF - criterion 5

19

TheThe assessmentassessment ofof thethe enablersenablers

Based on the Based on the PDCA PDCA cyclecycle::

Plan-Do-Check-Act: Deming circlePlan-Do-Check-Act: Deming circle

Principe of continuous improvementPrincipe of continuous improvementPlanPlan: : project project phasephase

DoDo: : execution execution phasephase

CheckCheck: : controlcontrol phasephase

Act:Act: action-, adaptation- en action-, adaptation- en correction correction phasephase

20

Philosophy of continuous improvementPhilosophy of continuous improvement

Q

t

Excellence

PDCA

Perpetuation of the achieved level

(quality system) e.g. ISO 9000

Continuous step by stepimprovement

21

3. Human Resources

Management

7. People

results

1.Leadership

2. Strategy

& planning

5. Process &

change management

6. Customer/

Citizen oriented results

9. Key

performance results

4.

Partnerships &

Resources

8. Society

results

Enablers Results

Innovation and learning

PDCA in the CAF model

PLAN

DOCHECK

ACT

22

EnablersEnablers assessmentassessment panel panel

0No evidence or only anecdotal evidence of anapproach1An approach is plannedPLAN2An approach is planned andimplementedDO3An approach is planned,implemented and reviewedCHECK4An approach is planned,implemented and reviewed onthe basis of benchmarking dataand adjusted accordinglyACT5An approach is planned, implemented,reviewed on the basis of benchmarkingdata, adjusted and fully integrated intothe organisation PA D C

23

CAF Results

KEY PERFORMANCE RESULTS

KEY PERFORMANCE RESULTS

SOCIETY RESULTSSOCIETY RESULTS

CUSTOMER/CITIZENoriented RESULTS

CUSTOMER/CITIZENoriented RESULTS

PEOPLE RESULTSPEOPLE RESULTS

• Perceptions• Indicators

Sub criteria

21

CRITERIA

• Goal Achievement• Finances

24

6. Citizen/customeroriented results

6.1. Results ofcustomer/citizen

satisfactionmeasurements

6.2. Indicators ofcustomer/citizen-

orientedmeasurements

CAF - criterion 6

25

CriterionCriterion 6 - Customer/ 6 - Customer/citizencitizen::WhatWhat to measure to measure and howand how??

Internal indicatorsInternal indicatorsQuestionnaires, panelsQuestionnaires, panels

Number of suggestions that wereNumber of suggestions that weretaken into accounttaken into account

InvolmentInvolment

Number of mistakes, respect forNumber of mistakes, respect forstandardsstandards

Products and servicesProducts and services

Opening hoursOpening hoursAccessibilityAccessibility

e.g.: numbers of complaints,,e.g.: numbers of complaints,,production timesproduction times

Global imageGlobal image

6.2. Indirect measurements6.2. Indirect measurements((produced quality))

6.1. Direct measurements of6.1. Direct measurements ofsatisfaction satisfaction (perception ofquality) ) ««Are you satisfied?Are you satisfied?»»

26

7. People results

7.1. Results of peoplesatisfaction and

motivation measurements

7.2. Indicators of peopleresults

CAF - criterion 7

27

InvolvementInvolvement

Development of carrier andDevelopment of carrier andcompetencescompetences

Labour conditionsLabour conditions

Management and managementManagement and managementsystemssystems

E.g.: number of complaints,E.g.: number of complaints,

absenteeism, rotation of people,absenteeism, rotation of people,

strikes, results of evaluations,strikes, results of evaluations,

degree of participation at trainings,degree of participation at trainings,

degree of response in surveys,degree of response in surveys,

number of suggestions that arenumber of suggestions that are

taken into account, degree oftaken into account, degree of

participation at social eventsparticipation at social events

Global imageGlobal image

7.2. Indirect measurements7.2. Indirect measurements((produced quality))

7.1. Direct measurements of7.1. Direct measurements ofsatisfaction satisfaction (perception ofquality) ) ««Are you satisfied?Are you satisfied?»»

CriterionCriterion 7: People: 7: People:WhatWhat to measure to measure and howand how??

28

8. Society results

8.1. Results of societalperformance

8.2. Results ofenvironmental performance

CAF - criterion 8

29

Criterion 8Criterion 8 : : DefinitionDefinition

Impact of the organisation on the society,Impact of the organisation on the society,

independently of the key activities in theindependently of the key activities in the

context of the missions or the officialcontext of the missions or the official

mandatemandate

Perception measurement + internal indicatorsPerception measurement + internal indicators

Local, national and international communityLocal, national and international community

30

Criterion 8: examplesCriterion 8: examples

8.1 The results of the societal performances8.2 The results of the environmental performances

MediaMediaWaterWaterHealthHealth

AwardsAwardsWasteWasteVoluntary workVoluntary work

AssociationAssociationIntegritiesIntegritiesPollutionPollutionEducationEducation

ArtsArtsCompensationCompensationof mistakesof mistakes

TransportTransportProject onProject ondevelopmentdevelopment

SportSportDiscriminationDiscriminationEnergyEnergyLabourLabour

CultureCultureEthicsEthicsEnvironEnvironSocialSocial

31

9. Key performance results

9.1. Goal achievement 9.2. Financial performance

CAF - criterion 9

0No results are measured1Key results are measured and shownegative or stable trends2Results show modest progress3Results show substantial progress4Excellent results are achieved andpositive comparisons to own targets aremade5Excellent results are achieved, positivecomparisons to own targets are made and positivebenchmarks against relevant organisations aremade.

Results assessment panels

33

1

32

45

Leadership

Strategy& planning

Processes

Partnerships & resources

Your profile !

HRM

Society results

Customer/citizenoriented results

Key performanceresults

People results

34

The following steps areThe following steps are

strongly recommendedstrongly recommended

Around these steps, variables areAround these steps, variables are

possible taking into account: culture,possible taking into account: culture,

maturity level of the organisation, ...maturity level of the organisation, ...

3.The process of selfassessment

35

1. Organise the self assessment process1. Organise the self assessment process

Engagement of the direction (aim,Engagement of the direction (aim,

acceptance of the results, implementationacceptance of the results, implementation

of the action plan)of the action plan)

Inform the personnelInform the personnel

Designate a CAF project leaderDesignate a CAF project leader

36

2. Compose the self 2. Compose the self assessmentassessment group (SAG)group (SAG)

Representative of the organisation: units,Representative of the organisation: units,

ages, responsibility levelages, responsibility level

Not necessarily the big boss!Not necessarily the big boss!

A well informed group individually A well informed group individually

Critical, constructive, credible individuals Critical, constructive, credible individuals

Personal skills rather than professionalPersonal skills rather than professional

37

3. Inform and form 3. Inform and form self self assessmentassessment group (group (SAG)SAG)

Presentation of the process by the leader:Presentation of the process by the leader:

aim, usefulness, action plan, necessaryaim, usefulness, action plan, necessary

honest expressionhonest expression

Training of the Training of the SAGSAG to the application to the application

of the CAFof the CAF

The information searchThe information search

38

4. Individual 4. Individual scoringscoring

Each member allots a score for eachEach member allots a score for each

sub criterion of the CAFsub criterion of the CAF

Justify with strong points and fieldsJustify with strong points and fields

of improvements (key words)of improvements (key words)

Being based on their knowledge andBeing based on their knowledge and

experienceexperience

39

5. Consensus5. Consensus

Research for a consensus in the group forResearch for a consensus in the group foreach score and the justificationseach score and the justifications

Alternative: statistical average: advisedAlternative: statistical average: advisedagainst because of loss of dialogue andagainst because of loss of dialogue andexchange of points of viewexchange of points of view

Important role of the presidentImportant role of the president

Discussion based on evidences or evenDiscussion based on evidences or evenexamples of the CAFexamples of the CAF

40

6.Variants in the self 6.Variants in the self assessmentassessment process process

External appraisers in the External appraisers in the SAG:SAG: consultants, consultants,

representatives of representatives of otherother public public

administrationsadministrations

Stakeholders in the Stakeholders in the SAG:SAG: authority, authority,

citizen/customer...citizen/customer...

2 2 SAG:SAG: personnel and direction personnel and direction

Validation of the results by an externalValidation of the results by an external

appraiser (e.g. for a selection)appraiser (e.g. for a selection)

41

7. How to exploit the results of the self7. How to exploit the results of the selfassessmentassessment

Diagnosis: +/-Diagnosis: +/- Identify fields of improvement as a result Identify fields of improvement as a result ofof the application of the CAF (weakest scores) the application of the CAF (weakest scores) Fix priorities: the most urgent improvementsFix priorities: the most urgent improvements

to be undertakento be undertakenHow?How? by crossing the strategic weighting of theby crossing the strategic weighting of the

criteria and the scores obtainedcriteria and the scores obtained Identifying "quick wins"Identifying "quick wins"

42

Plan d'actions - priorités CAF

0

0,5

1

1,5

2

2,5

3

3,5

4

4,5

5

0 1 2 3 4 5

Poids des critères

Sco

rin

g C

AF

5a,2

4b,1

5c1

3,2

4a,1

5,2

4. Less important and bad scores:

leave temporarily

3. less important and good scores:

divest?

2. Important and good:

status quo?

1. Important and bad scores:

priorities

2,2

5b,3

4b,4

4b,2

43

Comparative analysis of the resultsComparative analysis of the results

With other organisations which applied theWith other organisations which applied theCAFCAF

Situate the organisation in relation to aSituate the organisation in relation to anational or European average.national or European average.

Currently, possible European comparison onCurrently, possible European comparison onhttp://http://www.eipa.nlwww.eipa.nl

National databasesNational databases

e.g. Belgium : http://e.g. Belgium : http://www.publicquality.bewww.publicquality.be

44

The CAF Model

LeadershipKey

PerformanceResults

Process and

ChangeManagement

Human Resources

Management

Strategy & Planning

External Partnerships.& Resources

Impact onSociety

Customer/Citizen-Oriented

Results

People Results

ENABLERS RESULTS

INNOVATION AND LEARNING

ISO 9000People Surveys

Citizen/CustomerSurveys

Investors inPeople

Budgeting,Accounting

Audits

CRM

Citizen orServiceCharters

PerformanceManagement

MissionDevelopment

Benchmarking

QualityCircles

BSCPD

ISO 14000

Info Acts

UserManuals

Action plan

45

8. Advantages of self 8. Advantages of self assessmentassessment

More effective than an external audit:More effective than an external audit:

good awareness of what one discoversgood awareness of what one discovers

oneselfoneself

Not very expensiveNot very expensive

The personnel involvedThe personnel involved

46

Learn andCriticise

Look to thefuture

Problem-solving

Values in TQM

General values

Open Com-munication

Dealing withuncertainty

Dialogue

ManagingConflicts

Culture of Trust

PositiveImage of people

47

4. The 4. The lessonslessons learnedlearned

TotalTotal supportsupport of the management of the management

SA SA groupgroup rreprepreesentativesentative of the of the organisationorganisation

WorkWork fastfast: 4 : 4 halfhalf daysdays ofof meetings meetings

EvidenceEvidence to to justifyjustify thethe scores scores

TryTry to to workwork by consensus by consensus

Translate Translate intointo prioritiespriorities forfor improvementimprovement actionsactions

StrengthenStrengthen communicationcommunication onon the the exerciseexercise and the and the

improvementimprovement actionsactions

RepeatRepeat periodicallyperiodically the the exerciseexercise toto measuremeasure the the progressprogress

48

5. 5. ApplicationsApplications so farso far

Translated into 15 languagesTranslated into 15 languages

+/- 500 applications at the end of 2003+/- 500 applications at the end of 2003

Training CAF: e.g. EIPA, SPEYER, Belgium, NorwayTraining CAF: e.g. EIPA, SPEYER, Belgium, Norway

CAF data bank: e.g. EIPA, SPEYER, BelgiumCAF data bank: e.g. EIPA, SPEYER, Belgium

Chinese CAF pilot projectsChinese CAF pilot projects

Academic interest in United States : Univ.Academic interest in United States : Univ.

Massachusetts 2005Massachusetts 2005

49

6. Further developments6. Further developments

CAF one of the priorities of the MS, especially of the new MemberCAF one of the priorities of the MS, especially of the new Member

statesstates

CAF action plan 2005-2006CAF action plan 2005-2006

CAF Resource Centre EIPA: development of tools, guidelines, training,CAF Resource Centre EIPA: development of tools, guidelines, training,

website, databank, best practices, website, databank, best practices, CAFCAF networking networking

CAF experts group: Vienna, March 2004; Maastricht, October 2004CAF experts group: Vienna, March 2004; Maastricht, October 2004

CAF Users Event : Rome, November 2003; Luxembourg, June 2005CAF Users Event : Rome, November 2003; Luxembourg, June 2005

Benchmarking based on CAFBenchmarking based on CAF