Smaller Projects = Fewer Failures
One of the most heartening things the current government has been doing on ICT policy is the attempt to reduce overall contract size and lifetime value of projects. When the government came into power it established a temporary moratorium on projects over £1M (see this report from the Audit Office for details on this initiative). The new strategic implementation plan contains a presumptive denial of all ICT projects with a lifetime value above £100M, which, while still a big amount, is a step in the right direction. Typically, the advantages of such initiatives are couched in terms of giving smaller businesses access to the market and increasing competition to more than the usual four suppliers with whom central government has traditionally spent its ICT budget. Procurement efficiency leading to increased value for money, in a nutshell. The less publicized, but potentially even more significant benefit of this development is the simple fact that smaller projects lead directly to fewer failures. Look at the following chart for a moment from this paper on large scale software failures by Watts S. Humphrey of the Software Engineering Institute :
Two things should strike you:
1. ICT projects fail a lot, at any size 2. When they get too big they invariably fail
These facts aren't isolated to government, they relate to all industries and all suppliers. It's the nature of the game and no one, ignoring what their salespeople will tell you, has ever managed to crack it. There is lots of research into this and no one to the best of my knowledge has found anything approaching the definitive answer yet. Rather we seem to have a large number of partial answers, factors that contribute to the probability of failure, but explain only some types of failure for some of the time. More fundamentally, the enormously complex, non-linear, hard-to-parallelise, hard-to-estimate nature of such projects seems to run flatly against traditional notions of planning and control.
This being the rather bleak fact of the industry in which we find ourselves, we can however take heart in the fact that government moving towards smaller project sizes will in and of itself give us all better value for money as tax payers. Quite simply, spending £100M in £1M project sizes will give much better outcomes than spending £100M on a single project. This is true even if you spend the money with the exact same supplier and the exact same people managing it on the client side.
The government and its forward thinking ICT leadership should therefore be congratulated for taking steps in this direction. Let's hope they manage to shift the culture of Whitehall to reflect it.