We gathered consumer-reported issues from a range of sources; online forums, support agencies and submitted questions. We found that people with heat pumps often still have knowledge gaps and questions about their heat pump and often don’t know if their system is performing well or not.

The most common issues were:

• I have basic questions about how heat pumps work: 35%
• The operation of my heat pump seems overly expensive: 24%
• Is my house or system specification suited to a heat pump: 17%
• How do I better optimise my heat pump system: 13%
• I'm unsure if I am using or operating the system correctly: 13%

The issue tracker gave us a basic understanding of the types of questions people had, common issues that were reported and informed much of our interview content.

We conducted a small number of consumer and installer interviews to better understand the level of understanding in both groups. We also wanted to understand the relationship between consumer and installer, especially at the point of handover when the heat pump has been installed.

From the insights gained from analysis of the issue tracker, user interviews and existing content, we think there is strong rationale for a consumer-facing guide. However, there is also reluctance from some installers to inform consumers to a high level and for them to get ‘hands on’ with their system as it might require the installer to rectify any mistakes the heat pump owner may have made.

There was a recommendation to produce a resource for the installers to better hand over the system. But installers would require some control and ownership over over the resource to be able to customise it before providing it to their customers.

Selected insights from user interviews:

How much did you know about heat pumps before living in your current property?

• "I knew of their existence but did not understand the technology (and I still don’t)"
• "I may have some more knowledge now than other people. But I still struggle to explain it"

Would you have benefited from more information at handover of your heat pump?

• "Yes. What would be ideal was to have slightly longer time at the beginning and then a hour or two scheduled in two or three weeks later for more information"

What would you have liked to have been given?

• "There were some information cards and a big technical manual. There has to be something in between which would have been more useful"
• "The manual is intimidating. The fact I haven’t dug into it shows it isn’t user-friendly"

The interviews with installers surfaced a range of views on better informing consumers of their heat pump system, from wanting a high-level of awareness in users through to not wanting users to interact with the system and to leave any adjustments to the professionals.

The range of views largely reflects the installers prior experience with customers. It demonstrates that not every installer would support a guide that provides a lot of information to the consumer and that some installers may not willingly provide a guide to consumers during the handover process.

Selected insights from installer interviews:

• “If you put controls on [the heat pump] that are too easy to use, people can make a mess of the settings”
• “Installers have different priorities. They want to do a good job and achieve good efficiency. They also don’t want complaints or call backs when something goes wrong.”

On users understanding and adjusting their own system:

• “We ask [the user] to leave the heat pump alone as much as they can. We tend to say 'let it run and come back to us with your problem'”
• “We like to just have them interact with a thermostat on the wall. For the rest [of the problems] they should come back to us.”

On a guide that they could use with consumers during handover:

• “Open source the guide for installers. Give them a document that they can edit themselves”
• “A lot of installers will have their own processes but a handover document is maybe bottom of the list of materials for them.”
• “Could they personalise their own document? Could they choose the content, add their own details and it would then spit out a PDF?”

We mapped out content from online and printed resources, identified suitable content, information gaps and opportunities for new content. We proposed content for new heat pump owners to help them to understand their system. We also conducted a prioritisation exercise with an informed stakeholder group to understand which content was more or less relevant at the point of handover between an installer and a consumer.

Topics that were well covered throughout the content analysed:

• Basics of ‘what is a heat pump’ and ‘how does it work’
• Comparisons between different kinds of heat pump
• General cost estimates
• Myths and FAQs

Knowledge gaps:

Content that we identified as important via the issue tracker and consumer and installer interviews but that do not feature prominently in consumer-facing information:

• Hot water production and storage
• Buffer tanks
• The wider heating system
• The environmental impact and carbon saving of installing a heat pump
• Non-technical performance guides or tables
• Information or tools to calculate your own costs
• How to explain heat pumps to others
• Lists of common issues
• What to do if something goes wrong

Content prioritisation

Working with stakeholders from advice services, social housing and industry, we prioritised the content for a guide that would be given at handover of a new heat pump system and gave it a weighting score based on the up and down votes it received.

Top weighted content:

• Dos and don'ts of effective heat pump use: 8
• Energy efficiency improvements you could make to your home: 6

Lowest weighted content:

• How a heat pump helps decarbonisation: -6
• Explaining heat pumps to others: -7

We explored options for how to get heat pumps to consumers at the right time and through a variety of opportunities. We think it is important to work with others to produce the content and deliver it to consumers. We don’t see that as something Nesta is well positioned to do alone.

Three models of production and distribution came to the fore.

A consortium collaboration
A collaboration between a consortium of advice services could produce a single resource.
This resource would then be distributed through the consortium's own channels as well as providing it to "anchor organisations", local action or advocacy groups in the community such as Carbon Coop in Manchester or Loco Home in Glasgow.

Several members of the consortium can also fold the resource into their own project pipeline or add it as a best practice requirement for installers who are signed up to their standards.

The right information at the right time
This approach explores how the lifestyle resource fits as part of an information journey for users that starts pre-installation as they are finding out about heat pumps and continues on to support them to get quotes and commission the installation itself. Finally it helps them to use their new system once it has been installed. In this model, the user would get the first part of the guide through a variety of touchpoints, for example, where they contact an advice service. They would then get the final part of the guide on handover of the heat pump from their installer.

An open source configurable guide
For this approach we would create a very comprehensive content library crowdsourced from a group of agencies and covering a wide range of topics to be chosen from. In this instance, Nesta could act as a technology partner to produce a web-based compiler that enables the user to pick and choose which content they want and even have the option to add their own details (which could be useful for installation companies). Once the guide is configured with the desired content it would produce a PDF guide with only the chosen content, which could be emailed, embedded in a webpage, sent via messaging services (Signal, WhatsApp) or printed out.

We analysed a dataset of 305 reported Air Source Heat Pump (ASHP) complaints from the Renewable Energy Consumer Code (RECC), covering three years, from 2019 to 2021. Initial analysis has helped us to understand the nature of the complaints and to consider how a larger dataset could be of benefit to us and other organisations who provide advice to consumers .

Our analysis included:

• When people complain. Plotting the complaints onto a timeline, we can see that the number of complaints that mention heat pumps range between 1 and 18 complaints per month. There seems to be some seasonality with more complaints during winter time (which can be expected), although we do not have enough years of data to draw a lot of conclusions.
• Additional technologies. All complaints in the dataset are primarily regarding heat pumps. However, some of them mention other types of technology as well such as solar photovoltaic (7.2%), battery storage (2.6%), solar thermal (1.9%), biomass (0.3%) and EV charging points (0.3%).
• Category analysis. The top categories of the complaints were after-sales (72%), microgeneration certification Scheme (35%), completing the installation (32%) and estimates or quotes (29%).
• Expression analysis. We defined a list of keywords/expressions of interest to see how likely it is for them to appear in the complaints’ summary and grouped them into six groups: (1) performance, (2) industry (3) schemes, (4) advice, (5) finances and (6) installation. More than 50% of complaints contain the keyword ‘installation’ and 32% contain the keyword ‘issues’.

Important points and data caveats

• The complaint summaries are not self-reported, they are written down by call or email handlers
• There is a lag between problems occurring and the consumer complaining to RECC complaint dates
• As follow-up contact or subsequent complaints from the same complainant are added to the first complaint logged, it is hard to understand the progress or customer journey.

This project has given us a greater understanding of the issues people experience with heat pump installations and where they go to raise issues, questions or complaints about their installation.

We have evidenced a need for better information for consumers post-installation so that they better understand their heat pump and the system in which it operates. They can then act on that information in order to:

1. Be confident that it is working well and efficiently
2. Know when it is not working as it should
3. Know what to do about poor operation, either themselves (if appropriate) or through the contacting of a professional
4. To have a good experience of their system
5. Be able to advocate to others about heat pumps and their operation and performance as well as how they contribute to low carbon heating.

Nesta’s sustainable future team will be continuing to engage with partners about this project and will soon be commissioning a large-scale heat pump user survey which will provide more data on how heat pump owners use and experience their heat pumps.