Description

The design and planning of an evaluation methodology can commence with the mapping of key concepts and ideas. Different methods could be used to provide the map - use of pictures, concept maps, or concept 'webs'.

Concept mapping of impacts is used to define the effects that are to be evaluated and, in cases where there are multiple objectives and where these have not yet been firmly established, the associated indicators. It may also be very useful if the set of objectives lack precision. The tool is suited to evaluation in a partnership context because it is based on the aggregation of individual points of view for the purpose of reaching consensus between the partners.

The technique can be based on the use of various sources of information which could be used to undertake the mapping such as policy documentation, historical records, previous research/evaluation, scoping exercises, interviews, briefings etc.

The technique involves a structured process, and can be applied to individuals, stakeholders working in facilitated groups, or through the application of statistical (multivariate) methods to aggregate views from different stakeholders.

Concept mapping has been developed as a strategic planning tool, and is also sometimes used in market analysis and product development situations in industry.

The purpose of the technique

Mapping concepts and ideas can help in the formulation of research questions. Because it can involve groups it is a useful tool for promoting the participation of stakeholders in evaluation design. Importantly, when applied correctly the technique can make sure all views are captured in the debate. At the same time it is a technique which helps to deal with complexity and develop concensus in groups.

When applied to the evaluation of socio-economic development, the usual purpose of this method is to establish main parameters upon which the evaluation should or could focus. Its aim is to ensure that an evaluation will be comprehensive in its design. The aim is also to ensure that the design of the evaluation does not consider aspects that are irrelevant to the policy or programme area or topic. The maps can also be used as a 'touchstone' through the evaluation to keep it on track. If the mapping focuses on expectations around the outcomes and impacts of programmes, then the process can have an important function in the identification of indicators. Because the technique rates and clusters topics, it can also provide the basis for structuring the evaluation reporting.

When concept mapping is used for the purpose of mapping the impacts expected from a public intervention, a picture can be built up which takes into account the representations of all the protagonists (through clarification/grading). The systematic nature of the approach helps focus the work and when applied to groups can avoid fruitless digressions, and assist visualisation of the similarity or difference between impacts. It is possible to conduct collective reflections in large heterogeneous groups (up to 200 persons) and to structure and grade the main impacts expected from an intervention.

Circumstances in which it is applied

Concept or issue mapping is relevant to virtually all situations when an evaluation is planned as it enables resources for evaluation design to be more efficiently focused. It also facilitates stakeholder participation.

The technique is appropriate at different stages:

  • in an ex ante context, to determine the main objectives and priorities;
  • in an ex post context, the impact mapping technique is used to draw up the evaluation reference system (effectiveness criteria and impact indicators).

This technique is particularly relevant when the evaluability of a programme has to be enhanced in a partnership context involving several decision makers/funding bodies, because it provides a common evaluation reference system, despite differences in objectives that may exist between the partners. It is also useful where there is a lack of clarity and precision on the programme objectives and the expected impacts, and a lack of pre-defined impact indicators. Since its implementation is relatively cumbersome, it is advisable to limit the use of impact mapping to complex programmes with many expected impacts.

The concept mapping technique is particularly useful in circumstances where the programme being evaluated is multi-sectoral.

The benefits of Issue Mapping?

  • Maps decision and detailed rationale behind decision-making; maps the thinking process of the group
  • Concentrates on pros and cons to an idea, encourages and explores all view
  • Promotes greater shared understanding of the problem at hand
  • Represents and clarifies diverse points of view, conflicting interpretations and goals, inconsistent information and other forms of complexity
  • Opportunity for all to be heard, contributions acknowledged
  • Keeps participants on topic – they can see the progress of the discussion visually, the bigger picture can be absorbed better and they can appreciate the validity and value of a larger perspective
  • Helps participants come up with better ideas and avoids jumping to simplistic answers and superficial conclusions
  • Promotes deeper reasoning, rigor and crowd wisdom
  • Supporting information (such as documents and images) can easily be attached to map to back up group reasoning
  • Participants can see the effectiveness of mapping and genuinely will try to make the discussion more productive

Purpose

Mapping concepts and ideas can help in the formulation of research questions. Because it can involve groups it is a useful tool for promoting the participation of stakeholders in evaluation design. Importantly, when applied correctly the technique can make sure all views are captured in the debate. At the same time it is a technique which helps to deal with complexity and develop concensus in groups.

State of Development Please Select

Development Contact

Scope

Steady State or Dynamic Unknown

Input & Output Data

Accessibility

Open/Closed Source Please Select

User Information

User Interface Please Select
Ease of Use Please Select

Technical Considerations

Links

http://ec.europa.eu/regional_policy/sources/docgener/evaluation/evalsed/sourcebooks/method_techniques/structuring_evaluations/concept_mapping/index_en.htm

http://issuemappingclass.com/

Associated Case Studies