Private finance initiatives (PFIs) and public-private partnerships (PPPs) are in the spotlight again thanks to the collapse of Carillion. There are many dimensions to this. But I hope that at least one lesson is that this whole unhappy story is a classic example of what goes wrong when policy is made in an evidence-free way.

The underlying ideas behind public-private partnerships were promoted vigorously in the 1980s. For some, the motive was purely ideological: to dismantle the state. There were also other more pragmatic arguments, in particular the claim that, although capital would cost more to the private sector than the state, these schemes would bring new methods and skills into the public sector which didn’t have a good record of large capital projects.

That was at least a plausible hypothesis. But the policies were introduced without any trials, tests, experiments or evidence assessments. They were pushed through by fairly tight-knit cliques that straddled the civil service, ministers and a few big financial organisations (their revolving doors involving figures like John Redwood have been well-documented). Together they successfully resisted any attempts to evaluate the ideas.

PFIs were clearly bad value to the taxpayer; they didn’t transfer risk; and they involved high transaction costs and high costs of capital

It was already clear 20 years ago that many of the deals were very bad value for money, even as the New Labour government proposed to greatly expand the programmes. I was favourable to a more mixed economy of provision but always sceptical of the PFIs and PPPs I had seen close-up. When I was running the Strategy Unit (in 2001/2) I therefore commissioned a very senior figure from a big city firm, with extensive experience of the public sector, to do an honest review.

His review was scathing. He showed that a majority of the then PFIs were clearly bad value to the taxpayer; they didn’t transfer risk; and they involved high transaction costs and high costs of capital.

The review was circulated internally. The response was explosive. I was shouted at by both Gordon Brown and John Prescott, and various advisers of theirs, for having had the temerity to question such a crucial policy. Various threats then emanated from government about penalising the company where the author worked. PFIs and PPPs had – bizarrely - become an article of faith, not a potentially useful idea that needed to be tested.

Some of the recommendations of the report were quietly adopted, despite the denunciations, and helped to stop the worst deals. But plenty of bad practice continued, fuelled by a big machine of beneficiary organisations in consultancies and investment who raked off huge fees for setting up the deals.

This is a particularly bad, and costly, example of how policy shouldn’t be done. It was driven by ideology not evidence; bulldozed through without experiment and testing; and then kept out of the gaze of proper scrutiny, partly because it had cross-party support at the time and partly because of powerful supporters.

Returning everything to state management would be exactly the wrong lesson to learn from Carillion and other PPPs: a purely ideological mirror to the neoliberal arguments that produced the problems in the first place. The right lesson is that government should be pragmatic in finding the right mix of roles of government, business and third sector, and should be open, transparent and experimental as it navigates how to get the balance right. It's not exactly rocket science.