Monitoring and Evaluation: Evaluation protocols

This document defines a holistic methodology to assess the performance achieved in the SmartEnCity LH cities after the execution of the different actions structured on 7 protocols based in the KPIs proposed in the previous paper "KPIs definition" allowing measuring the objectives established in each city from technical, environmental, economic and social points of view.

This report compiles the holistic methodology developed for the evaluation of the performance of the interventions carried out in the three Lighthouse cities participating in SmartEnCity project. This methodology consists of 7 protocols named as Energy Assessment Protocol, ICT Protocol, Life Cycle Analysis Protocol, Mobility Protocol, Social acceptance Protocol, Citizen Engagement Protocol and Economic Performance Protocol.

Each protocol covers the description of the objectives to be evaluated and the methods to be applied and are represented by a set of KPIs which will be used as tool for quantify the result reached after the execution of the interventions and actions in the cities of Vitoria-Gasteiz, Tartu and Sonderborg. In addition, tailored evaluation plans take part of these protocols in order to describe the specific methods, indicators and procedures to be applied in each city. Tailored protocols have been designed to evaluate the effects of interventions in terms of environmental benefits (e.g. energy reduction, CO2 saving), thermal comfort, social acceptance, economic benefits among others after the collaboration of 16 partners from the three SmartEnCity Lighthouse Cities. Thus, this report has been the result of the involvement of a wide number of partners. Such tight collaboration has allowed agreeing a tailored methodology for the evaluation of the performance of the interventions. Thus, the development of each protocol has been open to include some requirements from each LH but maintaining a common framework of evaluation for a comparison of the results reached in each city once the interventions conclude. For some cases, the assessment will require the comparison of the final performance achieved after the end of the project with the period before the intervention, which is named as baseline.

Find full paper here.


  • Ana Quijano, CARTIF, anaqui(at)
  • Jose Luis Hernández, CARTIF, josher(at)
  • Julia Vicente, CARTIF, julvic(at)
  • Alberto Moral, CARTIF, albmor(at)
  • Marian Gallego, CARTIF, magal(at)