Just after finishing this important if depressing book, I saw TV news interviews with Peter Mandelson and John Prescott, the former denouncing criticism of Gordon Brown's alleged behaviour in Downing Street as part of a personalised Tory plot and the latter attacking the author of The End of the Party, Andrew Rawnsley, for seeking to make money. Prescott also defended the prime minister's management of civil servants and political retainers.

Beginning with the absurd and moving on to the surreal, there is clearly something of the Blackpool postcard about turning to Prescott for sermons on the wickedness of trying to make money out of political books, and on the treatment of staff. Prescott, as they say, has form. As for Mandelson, I never cease to wonder at his brass neck; it should be donated in due course to one of the South Kensington museums. There must be many Observer readers who will themselves have heard the Lord President of So-Very-Much talking about the prime minister in recent weeks. Has anyone (apart from Cherie Blair) ever been as rude about Gordon Brown as Peter Mandelson? While he is an accomplished master of the darker political arts, he should learn that there is a difference between a spin doctor and a whirling dervish.

It is inevitable, though a pity, that all the attention regarding this book has focused so far on the lack of anger management at the heart of government. It deserves far more serious attention. It was awaited in the Whitehall village with nervous anticipation because Rawnsley has established a justified reputation over the years for getting the members of the big, happy family of New Labour to sing. There are over 70 double-columned pages of references to sources, mostly identified, although the number remaining anonymous appears to increase as the story of Gordon Brown's premiership unfolds. Perhaps with flying staplers darkening the heavens, personal safety was a consideration here. The sheer weight of evidence about the years from the sanctimonious bling of Bush's henchman, Tony Blair, to the clunk of his successor gives a certain credibility to the tale. Do Rawnsley's detractors claim that he has fabricated all these quotations?

I do not normally much care for blow-by-blow instant political history. For those who like this sort of thing, this is the sort of thing they like. Yet the monumental scale of Rawnsley's evidential base, and his journalistic mastery of the story, make this a compelling read. The End of the Party will be a bestseller. But I doubt whether it will encourage many people to go into politics. John Buchan called politics "an honourable adventure". Not here, it isn't.

In his last book on the New Labour project, Servants of the People, Rawnsley followed Blair through his first term. Here he carries the story on to the present day. At the heart of the tale are the Iraq war, the personalities of New Labour's main characters and their style of government, and the triangular relationship  Blair, Brown, Mandelson  on which the whole enterprise was built. Turning over these pages of our recent history, the old joke about the inmates taking over the asylum often came to mind. I am mildly surprised that government in Britain appears to have survived the ordeal.

The story of the Iraq war is told in considerable detail. Blair's role as the Bush administration's pliant feudatory  "Yo, Blair"  is set out in all its gruesome detail. As Rich Armitage, Colin Powell's deputy at the state department put it, "We've taken your support and buried your conditions." Mind, Powell himself could not remember any conditions. "It was always given that Blair would back us militarily," he says. As the former senior law lord, Tom Bingham, makes clear in his recent book The Rule of Law, the invasion of Iraq was illegal. Blair postponed any discussion of the legal basis for military action, presumably fearful of the outcome, despite questions from Patricia Hewitt. When the attorney-general eventually produced his 337-word statement on the legality of the invasion, it appeared, as Rawnsley rightly says, to depend on the tautological proposition that the attorney could deem the war legal because this is what the prime minister had told him it was. Rawnsley's judgment is that Blair was not an out-and-out liar. He was "a sincere deceiver. He told the truth about what he believed; he lied about the strength of the evidence for that belief."

So the prime minister did not enjoy a Kosovo-style triumph in Iraq. There was no shower of petals. Iraqis and British soldiers died and were maimed. Dr Kelly took his own life, but of course no one in Downing Street was to blame. Chaos overwhelmed Iraq, but according to Sir Jeremy Greenstock, Blair "didn't want to confront the horror of it all". Not much of a surprise there.

We know from Lord Butler's Iraq report that the New Labour project eschewed the traditional processes of government  cabinet meetings, minutes, that sort of old-fashioned stuff  in favour of chats on the sofa. The idea was that you delivered a speech, got a headline and then someone, somewhere made whatever you wanted to do that day happen. But what complicated this seamless operational style for achieving very little was the relationship between the prime minister and his chancellor, the marriage from hell that parented New Labour.

You need to be pretty resilient to get through all this, with the plots, the counter-plots, the character assassination, the accusations of loose screws, the effing and blinding. On page 361 we read: "Blair's relationship with Brown was at a new nadir." The nadirs came thick and fast; one rock bottom after another. There are times when one almost feels sorry for Blair, except that he weakly abandons loyal colleagues like the brave Alan Milburn to the mercies of the manse, and should have taken the advice of his wife in the first place on how to handle the next-door neighbour. Brown's camp followers, such as Mr Balls, apparently proved their loyalty to the "capo di capi" by their brutality.

Then it happened. To adapt Frank Field's happy phrase, Mrs Rochester was let out of the attic. Paddy Ashdown thought that a Blair handover to Brown would turn Camelot into Gormenghast. According to Jonathan Powell, Blair's handling of the succession "will be a bigger criticism of Tony than the Iraq war".

But at first the owls did not hoot in Downing Street, even though Norman Tebbit offered the new prime minister a word of praise. We got a show of competence in the handling of floods and animal disease. The stage was set for an election. Then Brown bottled the decision, and an avalanche of bad news hit him and his government. The man who not long before had opened the spanking new headquarters buildings of HBOS and Lehman Brothers, who had lauded Alan Greenspan, watched these and other banks founder as "a flaw" was revealed in Greenspan's thinking.

Advisers came and went, including the clearly execrable Damian McBride. How on earth could someone with a famous "moral compass" have appointed and promoted him in the first place? At last, when all seemed lost, Peter Mandelson, forgiven but not forgotten, returned home from distant Brussels to save New Labour. The political symmetry was almost complete.

The attention given to Brown's behaviour as a boss is understandable. Yet I do not myself think that having a bad temper makes someone a bad man. The trouble about Brown is not necessarily his temper; it is that he is a bad prime minister. There is a certain tragedy about a life obsessively dominated by the ambition to get a job which your friends correctly predict that you will be very bad at doing. If Brown was any good, presumably his colleagues would not have spent so much time fruitlessly plotting to get rid of him.

So here we are. What has it all been about? A devolved administration in Edinburgh, half of one in Cardiff, a hard-won settlement in Belfast, no advance in Brussels, a splurge of public spending, a mountain of debt, Brown's very own "boom and bust", the stuttering beginnings of reform to our education system, the mother and father of all scandals in the mother of parliaments. But there has not been what Tony Judt recently called for, a redefinition of social democracy, an end to economism, the restoration of values to political debate. All that we got was the Third Way, described by Judt as "opportunism with a human face".

You would not expect a former Conservative chairman to like what has happened. But I recognise that politics is about give and take, in and out, turn and turn about. So the Labour party deserved its chance. Now we've seen what New Labour can do. Rawnsley explains it very well. Could old Labour have been worse? It is very sad that John Smith died so prematurely. At least under him there would have been a sense of public service and moral purpose about the government of Britain.

In The New Old World, his recent study of the European Union, Perry Anderson indulged in a piece of academic high camp when he defended the omission of Britain from his consideration on the grounds that "British history since the fall of Thatcher has been of little moment". Any friend of Anderson's thoughtful enough to dispatch a jiffy bag to California will surely force the New Left Review stalwart into a graceful volte-face. No dispassionate reader of Andrew Rawnsley's thumping 800 pages could doubt that we have lived through a strange and fascinating passage of British history which is still obscure. Our fate  through the lie-infested trauma of Iraq, the resultant general loss of faith in democratic politics and the clinching catastrophe in the money markets  has all the while been determined by a couple of the weirdest people ever to attain Downing Street.

Advance publicity surrounding The End of the Party had suggested that Gordon Brown's temperament would be its principal subject. Far from it. In his second book about New Labour, the Observer's political columnist is much more concerned with the transformation of Tony Blair. We had all understood that in a matter of only six years following the attack on the Twin Towers, Blair went from someone unhealthily obsessed with popularity to someone convinced that the only true mark of authenticity was its extreme opposite. "I've lost my love of popularity for its own sake." We had also grasped that this appetite for thrilling martyrdom played to something uncanny in Blair's religious character. But it is only through appreciating the sheer perversity of his decision needlessly to write what Rawnsley calls "an emotional blank cheque" to two very different men  Gordon Brown and George W Bush  that you begin to realise how dramatically skewed his period as prime minister became.

Nobody is ever going fully to understand why Blair felt recklessly beholden to people so much less gifted than himself; why, as Jeremy Greenstock put it, Blair was always happy to "handcuff himself to the wagon". All we can do is look at the evidence. Nor do we yet know why he preferred to fool himself about the influence he held over either. In both cases, he made bargains that turned out entirely in the other man's favour. In trying to explain his disastrous inability to confront Bush, a man possessed, as Rawnsley says, with considerable "peasant cunning", it has always been the conventional wisdom to say that Blair was a sort of head prefect with a fatal weakness for sucking up to headmasterly power. For that reason, it is said, he ignored Bill Clinton's stark warning "He's using you". But in these pages it is not so much power as mere activity which drives Blair. What on earth are we to make of a man who, on the day he left No 10, had already inked in 500 appointments for his first 12 months out of office? What are we to make of a government which came up with 3,600 new criminal offences in 10 years?

Any psychiatrist who began to question the behaviour of a leader permanently surrounded by half-eaten bananas would already have noted that images of insanity haunt the whole volume. Blair's closest confidant, Alastair Campbell, was a manic depressive who bears out Booth Tarkington's observation that arrogant people are the most over-sensitive. At one point, Campbell admits to liking nobody in the world but his partner and his children. Brown's corresponding best friends were significantly known as Mad Dog McBride and Shriti the Shriek. Before Brown ascended to the top job, Frank Field cracked a good joke to Blair about not letting Mrs Rochester out of the attic, but the prime minister had long ago been advised to put a sign above his desk reading "Remember the Chancellor is mad". Most interestingly, Blair kept quiet about his private beliefs because he worried that voters might think of him as a "nutter" who communed with "the man upstairs". His principal reason for leaving No 10, after his suicidal refusal to call for a ceasefire during the Israeli assault on Lebanon in 2006, appears to have been his fear of being taken out through the door as unhinged as Margaret Thatcher. "I don't want to leave like her."

The book's authority rests on an impressive breadth of research. But perhaps because he is driving a narrative at such cracking speed, Rawnsley rarely stops to risk profound analysis. Throughout, the author has gifted his critics free ammunition by putting so much of the story into inverted commas. This lends immediacy but not authenticity. As in bad plays, the characters all speak with the same voice. Peter Mandelson is allowed some tin-eared Victorian dialogue along the lines of "I love you, but I'll break you! If you do that, I can destroy you!" But almost everyone else is forced to forgo poor Trollope in order to mouth lousy Mamet. When made foreign secretary, Margaret Beckett's predictable reaction is to say "Fuck". You know she's going to use that word because Alastair Campbell has used the same word when told of David Kelly's death  which, characteristically, Campbell seems to regard as being more his tragedy than Kelly's. Brown tells Blair "You've stolen my fucking budget", and later asks "When are you going to fucking go?" Alan Milburn, the Blairite health secretary, does his best to confound Brown with what sounds like a Rada warm-up exercise: "I know what this is fucking all about. You know what it is fucking all about." But Brown zings back a reciprocal tongue-twister: "You shouldn't have fucking done what you did in the summer." David Cameron quickly masters the language of New Labour: "I should have stayed at fucking home." By the time the Queen makes a late appearance in the book, putting down an insufferable Silvio Berlusconi at the G8, you expect her to burst into the room with a heartfelt "Give us a fucking break!"

The book is by no means perfect. Rawnsley has taken such a long lease in the plushest quarters of the Westminster village that when he does finally allow himself to shift from telling the story to passing judgment he tends to equivocate, as if he fears losing access to people still active in politics. John Prescott's infidelity is cruelly rehearsed in excruciating physical detail, while Tessa Jowell's far more bizarre decision to split from her husband as soon as he represents a political danger to her is, for some reason, given a free pass. (This is not so much protecting your sources as wrapping them in hot towels and sprinkling them with eau de cologne.) A lonely sentence about the condoning of torture being "arguably the largest personal moral failure of Tony Blair's premiership" has clearly been inserted at a late stage and is not backed up with the investigative rigour it merits. The Observer's own abject eagerness to collude in the neoconservative adventure in Iraq, which lost a once-serious newspaper both the patience of most of its readers and some readers altogether, is passed over entirely, as is any more general mention of the press's wider disgrace. Throughout, Rawnsley shows a faith in the infallibility of Fleet Street as a political windsock that few outside the profession will share. Nor is Rawnsley's style free of taint. Is it only journalists of a certain age who insist that homosexual politicians are always "feline" and that they must invariably "sashay" down Downing Street?

Sadly, by their cack-handed strategy of pompous embargoes and ill-judged excerpts, thrown like raw meat into the 24-hour news grinder, Rawnsley's publishers have done their author few favours. By their clumsiness, Viking has forced a fair-minded writer into a position where, through no fault of his own, he has been made to appear as seedy and underhand as the party managers he despises. A more honest approach to releasing the book would have demanded a more honest response from the people it reports. This lively Shakespearian account is far too important to be remembered only for the stupid headlines it generated. When the smoke of mock-battle clears, we shall be left with the most thorough, the most enjoyable and the most original book yet written about New Labour.

David Hare is a playwright and screenwriter.