Energy is critical to sustainable development. But somehow the world needs more of it while rapidly slowing the rate at which we’re pumping carbon dioxide into the atmosphere. That means switching to renewable energy sources as fast as we can.

Policymakers faced with this conundrum might be expected to use a variety of instruments, including market-based measures such as incentives and subsidies. Making carbon-based fuels more expensive would encourage users to invest in energy efficiency and switch to renewable energy.

Businesses would rush to invest in solar panels and wind turbines and would sign up for renewable energy contracts as fast as you could say “sustainable development”.

There is a move in that direction. The warehouse operator Gazeley became well-known several years ago for spotting the potential in solar panels on the roofs of its long, low buildings. Manufacturers such as GSK have battled through planning permission to put wind turbines on suitable sites and other companies have committed to renewable energy contracts and solar panels, with HSBC claiming to be the world’s first carbon neutral bank, for example.

But it’s more of a trickle than a torrent. And the reason is simple. Despite the logic of encouraging the switch, and despite quite a lot of noise about subsidies for wind, solar and biofuels, governments still perversely provide much larger subsidies for fossil fuels, especially oil. As for the heavily subsidised nuclear sector, readers will have different views on whether nuclear energy is environmentally harmful or beneficial.

Direct payments to producers, to expand production and keep their selling prices down, and to consumers, so they can afford the prices, are the most obvious form of subsidy. But there are others.

For example, the UK government applies a lower rate of VAT to energy. Environmental campaigners claim the hard-pressed US Treasury could save more than $60bn over the next five years by phasing out a host of allowances and exemptions such as tax credits, royalty relief, insurance and preferential financing. In Australia, the coal industry has received indirect support through funding for coal-fired electricity generation.

But it is the emerging economies that provide the biggest subsidies. Russia, China and India are leading examples, although they have begun to cut subsidies. More surprisingly, the International Energy Agency (IEA) has identified Iran as energy spendthrifts, using approximately a third of its annual budget to keep prices down.

The IEA estimated that global subsidies to fossil fuels were around $400bn in 2010, and that was one-third higher than in the previous year because of rising energy prices.

India provides an uncomfortable example of how this can rebound on businesses. Consumers – especially farmers – pay a fraction of the cost of electricity and this is partly compensated by higher costs for industrial and commercial users. Surveying one of the highest energy prices in the world for business, the World Economic Forum recently concluded: “Costly and inefficient subsidies are damaging the economy.”

It’s not just perversity that keeps subsidies going, especially in tough economic times when governments want to save money. Energy subsidies typically serve noble social and economic objectives, such as protecting or stimulating a particular domestic industry or regional economy, reducing dependence on energy imports, or supporting disadvantaged groups.

Abolishing them is also politically dangerous. When the Indonesian government reformed subsidies in 2005 demonstrators took to the streets to protest against the higher prices. There is a strong case for subsidising electricity for poor communities in developing countries, because it is such a powerful stimulant to development. But badly designed programmes lead to waste which can make it more difficult for utilities to extend services. The supposed beneficiaries actually end up worse off.

In any case, the IEA says only 8% of the $40bn spent supporting fossil fuels in 2010 went to the poorest 20% of the population. It says subsidies disproportionately benefit the better off.

Mindful of these powerful arguments, leading countries are beginning to make progress. The G20 group of leading economies agreed in 2009 to phase out fossil energy subsidies. The catch is that this is a “medium term” objective, and progress has been painfully slow even with the need in most countries to slash government spending.

Germany got there first. In 2007 the government agreed a gradual phase-out of its notorious coal subsidies, which had reached €90,000 per miner. The subsidy, designed to keep domestic coal competitive with imports and preserve local jobs, will end by 2018.

Much more must be done. The IEA reckons that fossil-fuel subsidies will reach $660bn in 2020, based on current policies. Eliminating them would cut the growth in energy demand by 4% and avoid 1.7 Gt of CO2 emissions. It would also help businesses make the case for energy efficient investments, a vital and often overlooked component of emission reductions.

Roger Cowe is a writer and consultant on sustainable business

This content is brought to you by Guardian Professional. Become a GSB member to get more stories like this direct to your inbox