34. Climate Service Score (CSS)

34. Climate Service Score (CSS)

VERSION

V6.0 - 2026.03 — Existing with revisions

INDICATOR CODE

34

TECHNICAL OWNER

PRGR

INDICATOR TYPE

Country Level Outcome Indicator

INDICATOR CLASSIFICATION

Complementary

INDICATOR SCOPE

Programme specific indicator

APPLICABILITY

The selection of this indicator is recommended against the following sub-activities in CSPs logframes. Selection of the below sub-activities will NOT trigger the mandatory selection of this indicator:

1. Forecast-based Anticipatory Actions (FBA)

2. Climate and Weather Risk Information Services (CIS)

UNIT OF MEASUREMENT & ANALYSIS

Percentage of households

DEFINITION

This indicator measures households’ use of climate information provided by climate services to protect or adapt their livelihoods to climatic shocks and stressors.

Climate services: the provision of climate and weather information to assist governments, communities, and households in reducing their vulnerability to climate change impacts by making better decisions. The information needs to be specific to needs, easy to access, to understand and to act upon. Climate services involve the production, translation and dissemination of climate and weather information tailored to end-users’ needs.

RATIONALE

The lives and livelihoods of vulnerable populations are threatened by the impacts of climate variability and severe weather conditions. Some of the losses caused by climate shocks and stressors could be avoided if populations had access to reliable and timely weather information.

Last mile climate services provide households and communities with reliable and timely climate information in form of understandable and actionable messages disseminated through adequate channels (SMS, radio, TV, extensionists…). The information provided through these channels is tailored to the needs of the recipients and focuses on the different climate related elements that may hamper agricultural production (i.e.: start, amount and distribution of rainfall). The information is based on weather forecast for the agricultural season with the intention to support rural population in taking informed decisions to adapt their productive practices to the forecasted changes in rainfall and temperature. In some exceptional cases, climate services can provide early warnings prior to sudden shocks to protect lives and livelihoods (i.e.: flashfloods, cyclones…).

The Climate Services Score measures the utility of weather information provided through last mile climate services to targeted households to make changes to their productive systems in a way to protect or adapt their livelihood to the effects of forecasted change in climatic patterns.

DATA COLLECTION TOOL

The CSS requires the collection of a minimum module of 5 questions (Q1.1. to Q1.5) for its calculation.

In addition, an optional module is suggested to record the specific changes in household and livelihood practices as a result of the climate information received. This module contributes to the narrative explanation of changes observed in the CSS and needs to be disaggregated by cohort/ target group or geographic region as practices put in place depend on beneficiaries’ livelihoods and shock suffered over the recall period.

*Q1.6 can be adapted to the information provided and the type of shocks the information system is designed for.

SAMPLING REQUIREMENTS

Sampling requirements are the same applicable for PDMs or monitoring surveys where the CSS data collection tool is to be included. Detailed guidance on sampling options is available here. Panel sampling and the use of control groups are strongly recommended for the follow-up of this indicator.

INDICATOR CALCULATION FOR REPORTING

The CSS reflects the utility of the climatic services based on beneficiaries’ perception of the relevance and quality of information received. The different elements of climate services are captured by questions Q1.1 to Q1.5:

• Q1.1: Access to climate services

• Q1.2: Relevance of the information

• Q1.3: Timeliness of the information

• Q1.4: Tailoring of information

• Q1.5: Actionability of the information

The CSS is measured as an unweighted average of questions Q1.1 to Q1.5 considering yes=1 and no=0, to produce a score ranging from 0 to 100, with 0 indicating no access to the climate service received and 100 access to good quality climate services.

CSS = [{(Q1.1 + Q1.2 + Q1.3 + Q1.4 +Q1.5)}/5] x 100

A disaggregation of the CSS per terciles can be done to rank the quality of the services into low (CSS<33) medium (33<=CSS<66) and high (CSS>66) categories.

For the spider graph presented in the visualization section, the percentage of respondents having answered ‘yes’ to each question needs to be calculated. To be consistent, the percentage reported on Q1.1 cannot be lower than the percentage reported on other Qs.

Likewise, the total score cannot be higher than Q1 nor lower than the lowest values among other Qs.

Link to SPSS syntax here.

DATA ENTRY AND DISAGGREGATION IN CORPORATE SYSTEMS

Mandatory:

This indicator is always reported by components (Q1 Access, Q2 Relevance, Q3 Timeliness, Q4 Tailoring, Q5 Actionability) and total.

To be consistent the overall value entered into the last column should be between the Min and Max value reported under the same row. Likewise, the total CSS in the last row should be calculated by the system as the unweighted average of the 5 values reported under the same column.

Panel sampling and the use of control groups are strongly recommended. Therefore, it is particularly important when entering information into COMET, that the sampling size and sampling frame of each data collection exercise are entered into the corresponding COMET field of the outcome data entry module.

For each follow-up it is also required to specify the type of applicable weather-related shocks (multiple choice between Floods, Drought, Storm/Cyclone, Heat Wave, Wildfire, other) or other shocks to which the target groups were exposed in the previous year to the data collection round.

For this indicator's purpose, a cohort is defined as a group of beneficiaries that minimally share characteristics such as receiving the same type of WFP assistance/support and during the same period. They can also share other characteristics such as geographic area, vulnerability level, transfer modality, residence status, donor, or cooperating partner.

Optional for reporting outside of COMET:

• Geographical Area

• Transfer modality

Important data entry options: Since SO2 CSP activities commonly combine, sequence and/or layer multiple sub-activities to support the same cohort/target group, this indicator can be reported with all applicable sub-activities. COs are requested to use multiple sub-activities to report on this indicator when its results should be presented as the outcome of multiple sub-activities supporting the same target group.

BASELINE

In line with business rules, baseline values should be established within 3 months before and after the start date of activity implementation. However, it is highly preferable to collect baseline values before the start of activity implementation.

TARGET SETTING

Annual targets:

The annual target should be at least equal to (and ideally higher) the latest follow-up of the previous year or than the baseline figure if there is no previous follow-up.

End of CSP target:

The target value for the CSS is country specific and depends on the baseline figures, context, CSP duration and programme design (i.e., duration of assistance, complementary activities, etc.).

The score is expected to increase over time in multi-year interventions.

FREQUENCY OF DATA COLLECTION

Frequency of measurement is based on programme objectives and timeline. However, biannual monitoring is mandatory and should be conducted during the same period every year to enable comparability across surveys.

If the intervention is focused on the provision of climate information on seasonal weather events such as storms, floods, or droughts, it is recommended to collect follow-up data for this indicator immediately after the expected or actual occurrence of these shocks.

For years when a baseline is conducted, only one follow up is required.

INTERPRETATION

The CSS provides an indication of how useful the provision of climate services for the targeted households has been, indicating the percentage of households applying the information received to make changes in their livelihood practices. The score ranges from 0 to 100, with 0 indicating no access to climate services and 100 the use of quality information provided by climate service. An increase in the proportion of households with high CSS indicates that the information provided is more relevant, timely, understandable, and actionable.

The separate analysis of each question provides additional programmatic orientations on areas that need improvement. Therefore:

• Q1.1: % of households having access to climate services. A high value indicates that the channel used to disseminate the information is adequate.

• Q1.2: % of households considering the information provided is relevant to their needs. A high value indicates the adequacy of the information provided.

• Q1.3: % of households considering the information is timely. A high value indicates that information is received at the right moment to make decisions.

• Q1.4: % of households understanding the messages disseminated. A high value indicates that messages have been adequately tailored.

• Q1.5: % of households making use of the information provided to protect/adapt their practices. A high value indicates that messages are actionable.

REPORTING EXAMPLE(S)

Climate services have been provided in three regions over the past two years. The results of the monitoring surveys are as follows:

The graphs to complement the analysis are presented in the visualization section below.

From the analysis of the CSS, we find that in some targeted regions the perceived utility of the climate information has improved.

From the analysis of the single questions, we observe an improvement in access to access and timeliness of the information received: access has improved from 43% to 77% and timely receipt of information from 23% to 50%. Overall, improvements have led to a 20-percentage point increase in the share of households using the climate information to adapt their livelihoods to climate variability. In this example, results do not show significant differences when disaggregated by the sex of the household head. However, when this is the case, the above table could highlight potential additional barriers to access and use climate information services based on the head of the household. To do this, in addition to the overall value, each year should also include one column to reflect the perception of male and female headed households.

Therefore, we observe an improvement in the quality and use of climate information in all regions with a need for to improve tailoring and relevance of climate information.

Optional (information provided by Q1.6):

Based on the climate information received to adapt livelihoods to weather variability during the agricultural season in year 2, households modified the sowing date and changed the variety of crops cultivated in region A (100% and 50% respectively) and region B (75% and 60% respectively). Farming is the main livelihood of households in these regions. In Region C, however, households reported a change in grazing areas (45%) and an increase in water storage for livestock consumption (65%) as their livelihoods are based on pastoralism.

INDICATORS COLLECTED & ANALYSED AT THE SAME TIME

This indicator could be measured together with any other CRF indicators reported under SO2, but CSP activities measuring this indicator can also report as applicable on the following outcome indicators:

• 33. Climate / Resilience Capacity Score (C/RCS)

Also, other output and outcome indicators linked to the following activity tags should be reported as relevant:

• Climate Information Services (CIS)

COMPLEMENTARY QUALITATIVE RESEARCH

Depending on country specific context conditions and key programmatic assumptions noted in the corresponding TOC exercises, the information of this indicator can be analyzed together with process monitoring data/results that can be collected through qualitative approaches such as direct observation, Key Informant Interviews or Focus Group Discussions. No additional tools, questions or templates are required to do this apart from what is already used for the regular process monitoring.

DECISIONS DATA CAN INFORM

The CSS provides an orientation on the changes promoted by WFP intervention, the structure of the indicator in five questions reflecting the main characteristics defining a CIS (access, relevancy, timeliness, message tailoring and use of the information) enable program managers to identify the areas where the CIS can be further strengthened to increase its effectiveness.

VISUALIZATION

The following graphs are examples of recommended visualization alternatives:

Evolution of CSS components over time

Percentual variation of average ICI

Reliance on adapted practices based on climate information

LIMITATIONS

The indicator does not measure the frequency and reliability of information received.

FURTHER INFORMATION

Guidance: Planning and Reporting on Climate Action