What is there still to know about the British empire? And what should you know? To some, the answers to both of these questions are clear-cut. Empires, generically, were evil creations of the past that have now been satisfactorily swept away by nation states. As for the empire that was hewn out over the course of three centuries by varieties of British and Irish actors, this was the biggest of its kind, affecting every continent. Consequently, these islands must bear a disproportionate share of imperial guilt, and all that is necessary now is mass British acknowledgment of this fact, and some attempts at reparation. This is an arguable point of view; and, if you hold it, John Darwin's books on empire may not be to your taste.

This is not because Darwin is out to offer exculpation or vindication  far from it. Rather, what distinguishes his work is a steady rejection of "thinking in monoliths". In this book, as in his earlier tomes, the award-winning After Tamerlane (2007) and The Empire Project (2009), Darwin draws on his long experience of teaching and studying imperial and world history at Oxford to set out, with great clarity and intelligence, and in a wry, un-illusioned style, the multiform meanings and manifestations of empire over time and geographical space.

As he argues, empire was emphatically neither a peculiarly British nor only a European habit. Instead, and for much of human history, empire has been "the default mode of state organisation". Even today, all of the great powers, China, the United States, Brazil, India, Russia  and the EU, perhaps  either still exhibit some of the characteristics of empire, or have passed through imperial phases in the past, or are one-time overland empires that have managed (not always convincingly) to repackage and rebrand themselves as nation states. If, despite the ubiquity of modes of empire, Britain's own version of the beast nonetheless seems remarkable, this is partly because of the sharp contrast between the smallness of these damp islands, and the extremely large, diverse and far-flung territories the British and Irish managed for a while to invade and appropriate.

Darwin outlines the story of how this came to happen in the initial chapters and at the end of his book, while devoting much of it to particular themes such as missions, commerce, settlement and war. Throughout, he stresses the huge role in this empire of "more or less organised violence", whether meted out to rival armies, or indigenous peoples, or ecosystems. Even when Britain's own troops were sparse  in India before 1770, or in the Caribbean because of disease levels  it often coped by hiring indigenous troops and slave soldiers to do the dirty work. The extreme smallness of British numbers in many overseas locations also tended to reinforce the use of racist distance as a tactic of rule. Keeping the "natives" (and women) out of certain clubs in imperial settlements was not just prejudice, but also an attempt to shore up the charisma of the local dominant white males.

Yet, as Darwin makes clear, there was never an "imperial project" in the sense of a strategic and ideological master plan for Britain's empire. It was simply too large and various, and experienced over time in too many ways for that ever to be possible. London's influence was, anyway, always very limited. Even in 1914, the Colonial Office contained only 30 senior officials who were ostensibly in charge of 100 different colonial spaces, not to mention 600 quasi-autonomous Indian princely states that technically owed allegiance to the British crown. Given this administrative sparseness, much inevitably depended on the men on the spot, and they frequently complained how little London and the British people understood or appreciated them and their work.

But it was not just limited reach, knowledge and interest at home that precluded any sort of imperial blueprint, but also successive "colossal explosions" that at intervals diverted the course of the empire, sometimes stopping it "dead in its tracks". The American revolution not only cost Britain the 13 colonies, but also forced it to rethink the slave trade and slavery, and influenced its power relations in Asia and the Pacific. The Canadian risings of the 1830s obliged the men in London to think much harder about settler self-government. The 1857 uprising in India did not free the subcontinent, but it changed the way the British viewed and sought to govern it. The so-called Boer war advertised British vulnerabilities; and these were confirmed by the Irish rising of 1916 and the subsequent creation of the Irish Free State, blows that attracted the notice and attention of colonial dissidents in Asia and Africa. As this catalogue of imperial rebellions suggests, and as Darwin points out, those in the colonies who caused the British empire the greatest trouble were very often men and women who were "white".

Yet, despite all the problems and anxiety, the blood, the guilt and  crucially - deep uncertainty always about just how profitable the business actually was in monetary terms, leading Britons especially found it hard entirely to let go of the empire. Even in 1965, Harold Wilson was speaking wistfully of Britain's frontiers being on the Himalayas, while Margaret Thatcher still felt it necessary to devote treasure and lives to winning back the Falklands.

And it has not just been the members of Britain's political class who have found adjusting to diminished power and reach challenging. It is sometimes argued that Britons "othered" their colonial subjects. Yet it would be more accurate to say that the British never succeeded in making up their minds, sometimes othering black and white colonial peoples, to be sure, but sometimes also viewing them as fellow British subjects, as an integral part of themselves. Empire's complex and long-drawn out impact on identities here  on the sense of who we are  helps to explain why adjustment to the EU has proved so hard. At some level, varieties of Britons still kick against being confined to Europe only, to a single continent. Consciously or not, many of them still yearn to be a global people, to be actors on a bigger stage. Darwin's new book is now the best single-volume guide as to why this is.

 Linda Colley's The Ordeal of Elizabeth Marsh: A Woman in World History is published by HarperCollins.