In last week’s post, we discussed how the first phase represented a change in structure compared to the previous version of the fund, in a move that we hoped would help arts organisations to put forward proposals with a stronger strategic focus.

In this post, we will look more closely at the specific tool we used to try and achieve this stronger strategic focus – Theory of Change.

For those who are familiar with the work of Nesta, you will be aware that Theory of Change is used widely in our work. It is linked to the work we do around Standards of Evidence. If you’d like to read more about Theory of Change, we have added some links on the right hand side of this page.

In the case of this fund, we designed the application form to mirror the structure of a Theory of Change. It starts with defining a long term goal, followed by questions to explore the sequential steps that would be required to meet that goal, and the assumptions that would have to be tested along the way. We then delivered Theory of Change workshops open to any arts organisation interested in applying to the fund.

We captured two sets of feedback during this phase – firstly, a short feedback form completed by all participants at the Theory of Change workshops, and secondly, a more detailed online feedback exercise completed immediately after the application deadline, completed both by those who did apply to the fund and those who did not.

Transcripts of all feedback comments will be available as part of the interim report, published later this month.

A different approach

Whilst Theory of Change is a well-established planning tool in the public and voluntary sectors in the UK and internationally, we are not aware of many examples of the tool being used in an arts context. Needless to say, we feel that we still have a lot to learn about how it is best deployed.

“The social change innovation process [sic.] was a new concept and took a short time to adjust to a different approach.”
-Senior/mid-level manager, Pembrokeshire

We were pleased that we received feedback from many arts organisations saying that they felt they could use Theory of Change in other planning contexts, including those who did not apply to the fund, or who did not receive funding at this stage.

“Although it is disappointing news we did learn a lot along the process, it was very interesting to look at a problem from such a different perspective.”
-Senior/mid-level manager, Cardiff

Alien to a creative way of thinking and working?

It is fair to say, however, that Theory of Change was not universally popular, as the following quotation illustrates.

“The most valuable part of the process was speaking face-to-face with [the fund team] during one of the workshops as I was able to discuss the particular circumstances and context of our proposal... I found the least helpful part was 'the theory of change' workshop which I didn't feel was something that would be used in our organisation and it felt quite alien to a creative way of thinking and working.”
-Senior/mid-level arts management, Powys

This feedback is critical but not wholly negative. When given an opportunity at the workshop to discuss the logic of the application process in relation to their own circumstances, it did make some sense. Yet for this individual, they did not see this particular tool as something that would be useful for them.

For us, this emphasises the importance of putting principles before tools. As the first phase progressed, we responded to feedback by placing less explicit emphasis on the Theory of Change tool. Instead, we would introduce the tool, describe the principles behind it, and then make a judgement as to whether to go into more detail depending on the interests of the group.

The feedback quoted above suggests that this person found that the tool did not facilitate a creative process in their case. Perhaps they felt that the tool was trying to force them to come up with ‘the right answer’ rather than facilitating more exploratory discussions about the different options they might wish to pursue.

Some of the other feedback also suggested that some workshop participants did not feel ready to explore one specific problem and needed more time to identify which of many complex underlying issues their project might be able to address. In the interim report, and in our third and final post about it next week, we will explore in more detail what kind of preparation we think might be useful before introducing Theory of Change.

Creating different perspectives

The single most consistent piece of feedback was that discussion with colleagues at the workshop was useful – not just the training on using the Theory of Change tool but their opportunities to try it out and critique it in discussion with sector colleagues.

“Good to work through examples of Theory of Change with colleagues from other orgs, creating different perspectives and therefore understanding.”
-Theory of Change workshop feedback

What is most useful about Theory of Change is not necessarily having a highly polished diagram, set in stone and used as a guide through the life of a project. I suspect that I am not alone amongst my Nesta colleagues when, although I use Theory of Change on a weekly basis, I would struggle to produce a coherent Theory of Change diagram in a single sitting.

What is most useful about Theory of Change at a planning stage is the conversations that arise from trying to put one together.

It challenges you to agree on how you will measure longer term success, and to consider the extent to which you can be confident that your intervention will lead to the outcomes you want to see within the life of the project. By setting out the sequential steps towards meeting a long term goal it is necessary to explain the logic of how one step will lead onto the next, and any assumptions that you’re making along the way.

Whilst a finalised diagram might be a long way off even after a group discussion of two hours or more, you will be a lot more clear on the perspectives, ideas and values that your team members bring to the table.

A beginning rather than an end

All of the projects funding through the Digital Innovation Fund for the Arts in Wales will be publishing their Theory of Change diagrams as part of the reporting process. This will include not just the final version created as part of their evaluation, but also the earlier versions.

We hope that these will provide useful examples of the practical use of Theory of Change in an arts context, which can be used by others. The difference between the earliest versions of their Theory of Change and the more developed version produced in hindsight could even be illustrative in itself of the journey those funded organisations have undertaken.

“Grappling with the model was deeply challenging, I felt under prepared but the support and expertise was excellent, challenged my assumptions and supported a deeper thought process. A 'beginning' rather than end.”
-Theory of Change workshop feedback

So is Theory of Change a tool that can useful to arts organisations? We know that some of the arts organisations who attended the workshops have already started to use it in other work, and we intend to follow up with those organisations to see how they get on.

If Theory of Change is going to be used more widely in the arts, we believe that there is a need for more discussion across the sector about how tools like this can help to sharpen strategic focus whilst being careful not to stifle creativity, not to make people feel as if there is a ‘right answer’ measured purely in financial terms or only in the sheer size of the audience.

We are confident that the arts sector has a unique perspective to offer on this, particularly around how to pursue simple, measurable goals without losing sight of fundamental aesthetic and ethical choices.

At this stage, our view remains that Theory of Change can provide a suitable tool to do this. Still, we are open to the idea that the arts sector may wish to create its own tools as a more effective way to achieve the same ends.

If you would like to learn more about the fund and what we're currently working on, you can head to our new website, which is available in Welsh and in English.