In a report to the UN security council, Holmes said 2.2 million people had fled their homes in Darfur, and the number of displaced civilians has risen dramatically in Chad and the Central African Republic.
Holmes said it was time for politicians and concerned leaders to stop playing "protracted games with each other, with little or no thought to the lives and livelihoods of hundreds of thousands of their fellow citizens, whom the international community meanwhile keeps alive".
The UN emergency relief co-ordinator warned that despite 13,000 aid workers now operating in the region the poor security situation was putting efforts to help the population at risk.
"Despite its scale and success in sustaining millions and saving literally hundreds of thousands of lives, the Darfur humanitarian operation is increasingly fragile," Holmes said after returning from a tour of Sudan, Chad and the Central African Republic.
"If things do not get better, or if there were more serious incidents involving humanitarian workers, some organisations could start to withdraw and the operation could start to unravel.
"Then we could face a rapid humanitarian catastrophe ... We must do everything in our power to avoid it."
When Jan Egeland, Holmes' predecessor, first warned the Security Council of the humanitarian crisis in Darfur three years ago this week, about 230 relief workers were struggling to assist 350,000 people.
Holmes told the Security Council that aid workers had been "physically and verbally abused, offices and residences raided and personal belongings stolen."
He blamed both government forces and rebels for the violations of international law and widespread human rights abuses.
At least 200,000 people have died since the Darfur conflict erupted in February 2003 but some sources believe that the death toll is much higher.
The conflict broke out when rebels from minority tribes took up arms to demand an equal share of national resources. This prompted a heavy-handed crackdown by Khartoum and the Janjawid militia.
Holmes said that more than 250,000 people had fled to displaced persons' camps in the last six months and more than half of the population could be living in them within 18 months.
"Meanwhile, politicisation and militarisation of camps have become a fact of life, creating a future time bomb just waiting to go off," he warned.
The former British ambassador to France also emphasised the effect the conflict was having on Sudan's neighbours.
"The spillover effect from Darfur is clear, not least in eastern Chad."
He also called for better protection of the Central African Republic's border with Darfur, through the deployment of an international peacekeeping force.
* See also 'Darfur: the new Rwanda, The-Latest , Columnists