Sean O'Brien's stunning last collection, The Drowned Book, swept all before it, winning both the Forward and TS Eliot prizes and confirming O'Brien's place at the forefront of his generation of poets. In his new collection, O'Brien continues to explore the post-industrial urban-pastoral landscape he has made his own; but November also contains his most doubting and vulnerable poems to date. After a brief, beautiful opening lyric ("Fireweed"), the collection gets under way with a darkly ambiguous tribute to the power and caprice of poetic inspiration, "Jeudi Prochain". In this brilliantly imagined portrait of the Muse, we see Miss Prochain surf various historical disasters and celebrations, as when she "rides a tank / Bedecked with roses through the ruined capitals". The poem begins: "The Muse, your ex, Miss Jeudi Prochain, / Keeps all your pleading letters but reads none. / One day in someone else's mail you find / A postcard from the nineteenth century  / A train, some smoky poplars, sheds  / But she's already gone to spend the winter / Nursing Rilke in a Schloss. The gods themselves / Don't have her private number. You once did."

The mysteries of love and inspiration are venerable themes, and these lines are densely but unobtrusively allusive. The second line carries an echo of Auden's "Who's Who", a sonnet about TE Lawrence, "the greatest figure of his day". To the bafflement of his biographers, Lawrence loved a homebody who, Auden tells us, "answered some / of his long marvellous letters but kept none". The nod to Donald Justice's "The Telephone Number of the Muse" is more likely to be missed, for Justice is little read in the UK, despite being one of a generation of American formalist poets (along with James Merrill, Anthony Hecht and Richard Wilbur) championed by Michael Donaghy, who is elegised in November. The reader's enjoyment does not depend on noticing such allusions, but it will be enriched if they hear O'Brien's quiet assertion of a tradition.

"White Enamel Jug" is another highlight. Set in the Ardennes in an unspecified past, the poem opens with a tender love scene in which a couple "take their happiness / For there and then and not for memory". But the man has already been described as the woman's "deserter": this affair will not last, a minor chord is sounded. The jug is maternal in both appearance ("the handle with its female curve") and function (we are told that, when not dispensing milk, it "Attended, patiently" on the scene). Throughout November, the maternal is linked to such private, stolen moments that stand outside history.

There are other, less happy forms of standing outside history, and O'Brien explores them in the elegies for his parents that form the heart of this collection. Both parents expressed a never-realised ambition to write a book, and both are depicted in acts of unrewarding reading. In "Elegy" the poet writes of his mother: "I see you reading, unimpressed, relentless, / Gollancz crime, green Penguins, too exhausted / For the literature you loved, but holding on". "The Lost Book" ends with a desolate image of the afterlife as an extension of a purgatorial existence, when O'Brien addresses his father's ghost:

Was there another world, where you belonged,
Or one more corridor where you still sit, rereading
With the patience of a lifetime
Last week's paper, hoping it might yield
To scrutiny and show the outcome changed?
O'Brien cannot quite forgive the man he calls "my mad father" who "made the little room no place at all". All the same, both parents have been betrayed: by "the state that failed to keep the faith" and by the art from which they felt excluded.

Exclusion is the issue at stake in the darkly comic "Sunk Island", November's most openly political poem. Set in a churchyard, it describes a standoff between the speaker and a woman on horseback who clearly thinks he is trespassing. She asks the speaker why he is waiting. The poem offers this response: "For the flood to accelerate over this ground, / For your helmet to circle and sink like a moral, / For a rag-and-bone man with his cargo of trash / To come rowing past slowly, his mind given over / To practical matters, the pearls of your eyes / Unforgiven and sold at Thieves' Market / For sixpence and never once thought of again." The poem operates on both a realist and a symbolic plane: the speaker has more than a touch of the grim reaper about him, and the woman embodies certain class prejudices. Both are, in a sense, waiting for the other to die in this "slow-motion replay of England".

With each new collection, O'Brien's imaginative reach has grown, making him impossible to pin down. Even in as elegiac a volume as this one, he finds room for jokes ("A line of Nietzsche turns to three of coke"), a Borgesian phantasmagoria based on a night out in Newcastle, and a dazzling translation of Rimbaud's "The Drunken Boat". But it is the gentler, more lyrical note that makes November an artistic triumph. It sounds with aching clarity here, in a depiction of long-lost love, "Josie":

She was saying goodbye.
And I cannot complain. What is under the stone
Must belong there, and no voice returns,
Not mine and not hers, though I'm speaking her name.

Paul Batchelor's The Sinking Road is published by Bloodaxe.

November is an unusual title to launch in spring. But there is every reason to cheer: it is Sean O'Brien's first collection since The Drowned Book, which won the Forward and TS Eliot prizes, and it is masterly. O'Brien is pitch-perfect, never swanks and is amazingly versatile. He is at home with the epic: "On the Toon", set in a dark, visionary Newcastle, is a show-stopping, Dantesque poem. There is also a revelatory version of Rimbaud's "Le bateau ivre", judiciously reconstituting the end of the poem, and full of beautiful rhymes. Yet O'Brien is as secure in his moments of frivolity  such as in a squat poem about Marmite, of which some readers will approve, but to which I took exception because of its alienating conclusion:

It's not for us to turn away

The sort who shun the dark brown jar,

But sure as sure come Judgement Day

The Lord will know who his folk are.

The collection is a mix of gravity and levity but it leans, as a collection with this title should, towards the elegiac. It includes elegies for parents, friends and a convivial farewell to the poet  and old friend  Peter Porter. But the envoi that stops one in one's tracks is to "Josie". It is mysterious, unforced and lyrical. What makes it moving is that O'Brien does not over-promote Josie. She has almost no adjectives to sustain her  just a handful of colours and a smile. Auburn, brown and white  colours of the living and dead. She is keenly remembered and, at the same time, it is almost as though she had never been. She is, as he poignantly allows, of little significance in the greater scheme of things. It is restraint that shapes this writing and stirs our hearts.

One is aware, too, of how naturally O'Brien mixes the classical and colloquial. The end of the poem is beautiful in its weighing of what a lament is and in its acknowledgement that an elegy must always compete with silence.

O'Brien's voice is authoritatively his own. Yet there are echoes here of Shakespeare, Larkin and, occasionally, TS Eliot. He has a devoted understanding of the iambic pentameter and an unerring sense of structure. His fresh, exact, unfancy approach to language is a tonic. In "Europeans"  an autumnal poem about transformation and a meditation on authenticity  he pictures translating himself into a French mushroom-seller by a roadside:

Broad-brimmed platefuls and uniform buttons

Plucked before dawn in the forest of birch

The dank delicious one-legged flesh

He gives a droll glimpse of the mushroom-sellers we might become:

On our fold-away chairs near the crossroads,

Hunched in black overcoats, pale as our produce

Seeking and selling the flesh of the earth

By the handful and kilo in brown paper bags.

We cannot be other than real.

The last line might sound conclusive but it is the opposite. It raises a doubt  present elsewhere  about where we stand in relation to our lives and about how "real" our reality is. In "Novembrists", a poem set in his 50s childhood, O'Brien places himself alongside his parents and ponders their position:

Her headscarf and his muted cigarette

The desultory familiar talk, whose virtue

Lies in its routine, because it so

Resembles happiness we do not ask

For more.

Perhaps his parents settled for too little. As readers of these richly sustaining poems, we are more fortunate: we do not need to ask for more.