The moment that made a success of Alexander Masters's first book, Stuart: A Life Backwards, is described in its opening pages. Masters's homeless street raconteur, Stuart Shorter, tells his biographer that the first draft of the book is "bollocks boring". Not wishing to offend Masters further, Stuart comes up with a brilliantly simple editorial intervention. "Do it the other way round," he suggests. "Make it more like a murder mystery. What murdered the boy I was? See? Write it backwards." Masters takes the advice and provides, at once, an inciting incident to establish the nature of the relationship and an alluring narrative construction.

The few British publishers who still care about editing could take notes from Stuart. At a stroke, he had understood what was naturally good in Masters's writing  its co-operative sympathy  but recognised that it needed reorganising around a different design. So the book became a journey back from mental disorder and suicide (Stuart is dead by the time Masters has finished taking his advice) into sanity and childhood  in search of causes. Everything that happens in the book between amiable biographer and compelling subject is given shape and meaning by this structure.

A similarly co-operative process unravels in Masters's new book, which is also about a compelling misfit  this time, an Eton-educated mathematical prodigy whose university career has descended into solitary chaos. Simon Norton was, until 1984, considered by many of his Cambridge peers to be a genius-in-the-making, operating at a pinnacle of hyper-dimensional mathematics. Norton worked with John Conway on a great work, Atlas of Finite Groups, investigating group theory in search of a mathematical entity they called the "monster".

The monster  apparently  has something to do with the nature of the universe, though all we mathematically illiterate readers can, or need, to know is that one of its pursuers, a man gifted with great powers, is now living in squalor in a Cambridge basement, obsessing over train timetables. Enter Masters, who lives upstairs as Simon's tenant, and who has suggested writing his biography  along similar lines to those with which he had approached Stuart: namely, that his subject can vet the manuscript, interject and change it.

Being the kind of mind he is, Simon's amendments are not so much grand structural interventions as complaints that relate to facts and details: especially those concerning the condition of his basement, Woking bus routes or his campaigns against motoring. The result is a comically tender portrait, emerging through this process of mutual exchange. Simon is a peculiar and remarkable man whose eccentricities cannot be reduced either to Asperger's-type cliches or to madness ("his only mental pathology is an excessive desire to obey local housing law"). Rather, they trace the outlines of a unique, irreducible, non-assimilable character.

The chaos of the basement is a product of low disorder and high mental organisation. When Simon speaks, it is often in grunts, which have a variety of different meanings that Masters illustrates typographically and in doodles. The two men go on trips together, on one occasion to Woking ("the coalface of biographical research") and finally to Norway, where Simon sees more reality in maps than the landscapes they delineate. He is a man from another dimension surviving the world with the tools of abstraction and tins of mackerel.

Masters builds up a collage-like composite of Simon's genius and examines the nature of genius itself (which is never achieved through mere labour, he concludes, but only through "delight"). But the actual mathematics in which, briefly, Simon was a world leader, is beyond Masters's grasp, just as it will be to the majority of his readers. Instead, he attempts to make mathematical concepts accessible through pages of doodled illustrations. Charming though these may be, they remain unequal to whatever is, or was, going through Simon's mind.

Simon's intellectual realm is a universe of its own. What we, as readers, are interested in is what made him turn out the way he did, as a "person". Unfortunately  unlike Stuart  Simon isn't very interested in himself, his ancestry (Iraqi Jewish), his family, or any defining moments in his life. His mathematical gift was clearly begotten, not made, formed by genetic accident rather than flowing from events.

Masters does come across some events that have narrative potential, at one point discovering childhood bullies who want to apologise for the way they treated their odd school-fellow (Simon remains oblivious) and scattering the text with incidents from his emergence as a child prodigy: Simon getting an impossibly high IQ score of 178, his precocious victory in mathematical olympiads, occasional anecdotes, such as being asked to spell "bikini" in a school quiz and instead coming up with "Picquigny" (after the English-French treaty of 1475).

No event, however, appears either to have made or unmade Simon. Innate character may inspire a fascinating portrait but, because it is resistant to influence, it is the enemy of story. And, although Simon is entertainingly full of criticisms of Masters's writing, he cannot provide a structural solution in the way that Stuart did. It wouldn't occur to Simon to view himself as the psychological outcome of a drama.

It is on page 286, in 1984, that we learn about the "biographical climax" of his life, the trigger of his collapse, the moment when he lost the capacity to be a genius. Surely this event should have been telegraphed from the beginning, to fuel our anticipation.

What happened? Simon made a mistake during a mathematical discussion that surprised his colleagues. That's it. He was never as good again. No crisis borne of narrative momentum, no release of psychological tension, just the falling away of something.

It was John Conway, not Simon Norton, who moved on from their work with a stellar career  and Conway's departure for America is referred to by Simon as a "bereavement". It is the only truly emotive acknowledgment he makes in the whole book (that and his grief at the Transport Act 1985) and stands markedly under-explored. Masters somewhat skims over the relationship of the two mathematicians, possibly because it took place in a dimension beyond verbal description. Yet one feels that an editor might have demanded more of this, perhaps even a restructuring around it.

Simon emerges as a great character, but one without a story. Then again, perhaps that's as it should be. He's very happy like that.

In 2008, at a Downing Street reception, Gordon Brown presented a young man, a member of Plane Stupid, with a Transport Campaigner of the Year award. During the ceremony, the young man superglued himself to the premier's sleeve. The prize is sponsored  £10,000 a year  by Simon Phillips Norton, a rich recluse and public-transport obsessive who lives, surrounded by timetables, ticket-stubs, packets of Batchelors Savoury Rice, in a run-down multi-occupancy house in Cambridge. A former child prodigy, he is still believed to be one of the world's great living mathematicians, although he hasn't held down an academic position since 1985, when he was 33. And he used to be Alexander Masters's live-in landlord, which is how he comes to find himself the subject of this book.

"I don't like your books, Alex," Simon says in the epigraph to one of Masters's chapters. "Your representation of me as interesting is inaccurate," he says in another. "You must be very careful not to jump to easy answers," says John Horton Conway, a fellow mathematician. "Oh dear, I have a feeling this book is going to be a disaster for me," Simon comments in the epigraph to the book.

Masters's first book was Stuart: A Life Backwards (2005) won the Guardian first book prize. Masters told the story of how, as a mathematics postgraduate at Cambridge, he got a part-time job at Wintercomfort (a drop-in centre for homeless people, where a director and a manager were jailed in 1999 for allegedly allowing drug-dealing on the premises). Clients, staff, friends campaigned to support them: and so, Masters found his lot thrown in with Stuart Shorter, a young man struggling with a past that included heroin, Largactil, solvents, progressive disability and repeated childhood rape. The book had in it press cuttings, Masters's own Ronald Searle-ish line drawings, and Stuart's running commentary on the work as it went: "It's bollocks boring  Nah, Alexander, you gotta start again." Polyphonic and reticulate, warm and harsh and very funny, the book was a triumph, a brilliant piece of authored documentary. Stuart died in 2002 before it was finished  the coroner recorded an open verdict after he stepped in front of a train.

Like Stuart, The Genius starts with Masters finding himself unexpectedly intimate with an unusual person. (In fact, Simon had a walk-on part in the earlier book  "Twice winner of a Mathematics Olympiad gold medal, co-author of Atlas of Finite Groups, my landlord is a generous, mild man, as brilliant as the sun"  "One fact to get right, and you got it wrong in four different ways," Simon protests in the present one.) And as with Stuart, Simon's story is presented in an engagingly open-textured and multivalent way. There are photos of his mother, brothers, the family business. There's an exam report from Eton: "!!" is all it says for maths. There are Masters's own drawings, animating the basic principles of group theory, the area of maths that was Simon's passion. There's the newsletter Simon now edits for the Cambridgeshire Campaign for Better Transport. There's a picture taken by Masters of Simon himself, smiling broadly, holding up a tin of John West mackerel fillets in sunflower oil.

As you can imagine, much of this is delightful  the bloggy, scrapbooky aspect, the kipple and backchat and disgusting food. In her still-unsurpassed The Last Samurai  another tale of childhood prodigiousness  the novelist Helen DeWitt imagines "the writers of the future" learning to do with words what Cézanne and Schoenberg were doing close on a century ago with painting and music, and in its best moments Masters's work has something of that excitement, something new and open and risky and humane. The drawings, for example, depicting the basics of group theory: rows and ranks of squares rotated through 90, 180, 270 degrees, apparently unchanged, if it were not for the way the moves are represented, with flailing little humanoid arms and legs  a lovely insight into the synaesthesia of giftedness. "Mathematics was simply there: the setting for existence; the touchstone for all activity. It was to Simon what green fields and dark woods were to other schoolboys: the enjoyable places that you rushed to, whooping, as soon as the afternoon bell rang."

Other bits, however, get whimsical and overegged. So yes, the squares are nice  but was it really necessary to invent "Saucy Miss Triangle" with her high-heeled shoes and bloomers? I could also have done without the chapter printed in white on black (because it's dark in Simon's basement) and the joke about calling him "Simon MINUS Norton" (because he features not as presence but as absence, ho ho). And the bit with Masters imagining himself turning into his subject  there's a picture of him going to a party dressed up as the number 7.

Also, there's something odd about the book's subtitle  The Biography of a Happy Man; call no man happy until he's dead, the Greeks used to say, and surely they were right. Of Masters's many virtues, the topmost is the way he refuses easy copout labels for a tricky person, his commitment to human variety and unfathomability. No matter what dreadful things Stuart did or what horror he suffered, he was not to be mollycoddled or monstered or let off any hooks. Simon's life  externally at least  does not appear so abysmal. But to call a person "happy"? Doesn't it soften the edges, dismiss the reality of a person's struggles, render a grown man cuddly and belittled, like a sort of gonk?

Besides which, if Simon's really that "happy", why does he keep talking about grief? "It's a cliché that mathematicians are over the hill by their mid-30s, but often it's not loss of mathematical intelligence that weakens their ability, but loss of focus  Simon says that in his case, it was grief." There's grief for his mother, who was his first muse as a toddler. There's grief for Professor Conway, who, some believe, precipitated Simon's "catastrophic intellectual collapse" by abandoning Cambridge for Princeton. And there's grief for "the 1985 Deregulation of the Buses Act", which, according to Simon, compelled him to give up academia for extra bus trips: "I'd say you ought to treat me as if I was currently watching the great love of my life being slowly murdered, torn between my desire to save her and expose her murderers and my wish to spend as much time with her as possible while she's still alive."

Throughout the book, Simon is admirably clear as to his reasons for agreeing to help Masters: "You said I could use the book as a soapbox for the issues on which I care deeply  The two things that I would recommend to anyone who is lonely: politics and public transport  Cars corrode mankind." And yet, he feels, Masters isn't entirely listening: "You confine all mention of my campaigning activities to the barest minimum, in spite of my repeated statements that they are essential," he complains in a long and furious email. Masters seems aware that, in order to get the book he wants out of his subject, he comes closer than he'd probably want to "satisfying stereotype". Now he's got it, though, I guess it's Simon's turn, so here he is, from the final page: "Fun is what this Cameron government wants to destroy with our public-transport services, cutting enjoyable and vital links for the people of Britain, leading to increased pollution, motor traffic and global warming." Not to mention, from the page before that: "Oh dear. Can I go now, please?"

Jenny Turner's The Brainstorm is published by Jonathan Cape.