In 1971, Audrey Death has spent half a century catatonic in Friern mental hospital. Zack Busner  a radical psychiatrist just recovering from coming under the influence of RD Laing  has an idea. He recognises that she and several of the hospital's other long-term patients may not be mad, but are actually suffering from the 1920s sleeping sickness encephalitis lethargica. Oliver Sacks-style, and somewhat outside the formal channels, he tries treating them with L-dopa. They start waking up.

Meanwhile, if that's the word, we go back to Audrey's wartime past as a munitions worker in the Woolwich arsenal, and those of her older brother, Albert, a war office civil servant with an eidetic memory (like Zack Busner, another old Self theme), and her younger brother, Stanley, who goes off to fight in the war. A third strand of the story shows us Busner, in old age, shambling around north London and looking back on his life  a journey that is to culminate in a visit to the now-decommissioned mental hospital.

Umbrella is not, be warned, altogether easy going: 400 pages of unbroken stream-of-consciousness dotted across three time frames, leaping jaggedly between four points of view, and with barely a paragraph break, let alone a chapter heading. I started out wondering whether, as a prank, Self had decided to write the novel that Richard Tull, the unpopular writer in Martin Amis's The Information, is working on. Self's sentences themselves sometimes resemble Victorian lunatic asylums refitted for 1970s use: ugly, overstuffed, clattering with the moans of lunatics and bristling with redundant gothic spires. The antic gurgles of laughter you find in Self's earlier work are few and far between. In their place, though, is a sustained depth and seriousness, and an ambition of technique that I haven't seen in him before.

I don't mean to put you off. Umbrella is old-school modernism. It isn't supposed to be a breeze. But it is, to use the literary critical term of art, kind of amazing. To frame it in terms of the film Gremlins, if you fed Alan Hollinghurst's The Stranger's Child after midnight, it might come out looking a bit like Umbrella. Both share the historical range, the multiplication of perspectives, the interest in architecture and memory, and the central preoccupation with time. But where Hollinghurst's mode is fastidiously naturalistic, Self's is the exact opposite. Hollinghurst is interested in time's arrow; Self presents time as a dangerously faulty artillery shell.

It's hard to précis what happens in Umbrella, because it doesn't quite work like that. It's arranged like a vorticist painting: a fractal proliferation of rhymes and symmetries; a frozen explosion. Hospital corridors are overlaid on London streets, are overlaid on trenches at the front. The packing of an artillery shell rhymes with the encapsulation of an antipsychotic drug or the manufacture of an umbrella.

The encephalitis patients are frozen in time  but in their myriad tics and autonomic circulations of the hospital they encode their pasts. Audrey's tics see her turning artillery shells on an invisible lathe or typing dockets at an umbrella factory. Busner sees what they are doing only when he films them and slows it right down or speeds it up, capturing "a flirtatious gesture that it took her two hours and twenty minutes to make". Ticcing "links macro-and micro-quanta".

Song fragments recur, tic-like, in Busner's head as in Audrey's (a nod to Pynchon, perhaps), and  perhaps too showily; Self does like to tell you what he's up to  shards of King Lear, Hamlet and TS Eliot spike the text. Memory and hallucination interpenetrate, and the narrative cuts between time streams mid-sentence. Busner copulates with a medical technician in a mechanically detached way (Ballard's a presence here)  her sexual responses analysed in clinical terms that then give way to the vocabulary of trench warfare. This is a world of constant recursion, of repetition compulsion, of the paranoid-schizophrenic apprehension that everything is connected.

Crucially, the book also contains human beings  Audrey, Busner, and a supporting cast of characters on whose bones real flesh is to be found  and is anchored in something approaching reality by a stupendous density of research and observation. Self's London or, rather, Londons are horribly alive  period details crammed in like a cabinet of curiosities. The story of Audrey's younger brother, Stanley  a machine-gunner in the war  contains some extraordinarily well-imagined battlefield descriptions:

"Pull, rotate, pull, rotate  so the belt-filler makes use of its animal components: pull, rotate  pull, rotate  On the far side of the tumbled-down fence the jellyfish of camouflage netting rises and falls soundlessly  in this ocean of noise  the gunners, stripped to the waist, scamper about the heaving creature, their devil's tails of braces bouncing on their backsides."

Read it. Then go and have a bit of a lie-down. It may not be his easiest, but I think this may be Will Self's best book.

As if the first world war was not trial enough, those who survived were then exposed to two devastating global infections: an outbreak of Spanish flu and a mysterious "brain fever" or "sleeping sickness", subsequently medicalised as Encephalitis lethargica. The victims divided fairly neatly between three outcomes: death, recovery or catatonia, the latter patients surviving in either frozen silence or with frenzied, repetitive tics. However, having been hospitalised in this condition for five decades, some recovered after the experimental administration of L-Dopa, a drug developed to treat Parkinson's disease.

This real-life fairytale of suspension and reanimation has already inspired two remarkable literary works: Awakenings (1978), a non-fiction account by Dr Oliver Sacks, and one of Harold Pinter's finest plays, A Kind of Alaska (1982), in which a young woman stricken with the disease wakes to find herself an old woman in an unrecognisable country. Will Self, in his 15th work of fiction, creates a third brilliant and original work from the story of these "enkies".

As has become fairly standard in modern fiction, several narratives and time periods are simultaneously in play: London and the French trenches in 1918, Friern Barnet Mental Hospital in 1971, and North London again on the eve of the 2010 election. But whereas most fictional time-jumpers confine the separate strands within sections or chapters, Self switches focus between sentences or, sometimes, during them: in a printed equivalent of the dissolve device in films, one character thinking of another will somehow lift the reader into the latter's consciousness. With no textual divisions, speech represented without quotation marks and scarcely any line indentations, the book is, in effect, a single paragraph of 397 pages and around 120,000 words. In common with another book on the 2012 Man Booker longlist  Nicola Barker's The Yips, which also challenges the normal shape of a story on the page  Umbrella must have led to all leave being cancelled at the typesetters and printers.

The structure, though, is clearly encouraged by the subject-matter: Encephalitis lethargica represents a paradox of consciousness, in which the patient, though in medico-legal terms still here, is predominantly somewhere else. Audrey, the patient on whom the book mainly focuses, spends her half-century absence making repetitive hand motions which (the reader understands before her doctors) relate to her earlier life. While attempting to solve her mystery, psychiatrist Dr Zack Busner is distracted by a far more common problem of human consciousness, which is that memories and brain-lodged song lyrics, printed in italics, keep thwarting his thoughts.

These plot-lines make stream-of-consciousness the logical form for a novel which also plays a sharp literary joke on Audrey: having gone under in 1918, she missed out on modernism  four years into her trance came the influential splinterings of language and form in Joyce's Ulysses and TS Eliot's The Waste Land. Her story is being told in a fashion that is, like most of her startling 1970s surroundings, foreign to her.

The Man Booker longlist on which Umbrella stands (and from which it deserves to proceed) is widely seen as a deliberate correction to the controversial plea from last year's judges for "readability", and some may conclude that Self's book represents the opposite quality. But, though hard work is certainly demanded from the reader, it is always rewarded. Through the polyphonic, epoch-hopping torrent, we gradually construct a coherent and beguiling narrative. As the title-defining epigraph from Joyce alerts us  "A brother is as easily forgotten as an umbrella"  fraternity is an urgent concern. Busner carries the shadow of a brother and Audrey of two, although the ominous surname the siblings share  Death  has been variously scrambled by vanity and mistranscription. In a linear reassembly of events, Busner, in 2010, is remembering how, in 1971, he brought the sleeping beauty Audrey back to life, using the pharmaceutical prince-kiss of L-Dopa. Audrey had been a feminist and munitions worker whose brothers served the war effort respectively at the Somme and in Whitehall, and whose postwar fates were starkly different.

From this dense tapestry, dramatic set-piece scenes rise in high relief: Audrey's sexual awakening, her medical reawakening, Busner's tense visit to a retired colleague. Self links different eras and people through linguistic echoes, so that a sex scene adopts the metaphor of weapons construction, and medical advances take on the phrases of the battlefield.

The tightest binding, though, is the title word, which becomes, as they say, an umbrella term: referring to an anti-shell device in the trenches, a retracted foreskin revealing a prepuce, an intra-muscular syringe. The restored Audrey, shocked by newspaper photos of the moon landings, notes that even the lunar buggy seems to have an umbrella trailing behind.

In common with his early mentor, Martin Amis, Self has sometimes risked his violently distinctive guiding voice overpowering plot and characters. Umbrella does contain a few of the thesaurus-gorged sentences that inflame his detractors, but the author is generally invisible within the intricate Babel of voices (although Busner is given some brainy Self-ish riffs on subjects including Judaism, nazism, ageing and umbrellas).

More than once, the doctor refers to the experience of his "enks" as "astonishing", which it is, although Self's treatment of them is more pessimistic than that of Pinter or Sacks. Busner concludes that there is no such thing as a medical miracle, and that he cannot long delay the fate printed on Audrey's birth certificate under "name". The tale of the awakened patients is, though, miraculously captivating and, in Umbrella, receives another magnificent reimagining.

Mark Lawson's Enough Is Enough is published by Picador. Will Self will be appearing at the Edinburgh International Book Festival on 25 August.