Nelson Mandela disappeared, aged 44, into prison. For the next quarter of a century he became a mystery man, the missing leader. And when he finally emerged victorious in 1990, there was a pent-up demand to hear from him. Since then, books about and by Mandela have become an industry, practically a literary genre of their own: dozens of biographies, authorised and unauthorised, children's books, books distilling his leadership style, business books and art books have appeared. Is there really room for another book on the bulging Mandela shelf? What more is there to say? Quite a lot, it turns out.

Conversations with Myself isn't so much a book as a literary album, containing snippets of Mandela's life, shards from diaries, calendars, letters, and also transcripts from 50 hours of recordings by Richard Stengel, who ghosted Mandela's autobiography Long Walk to Freedom (and is now editor of Time magazine). It also contains passages from an autobiography Mandela had been working on himself, in moments snatched here and there, but has finally abandoned, and allowed to be folded into this volume.

If that all sounds somewhat scattershot and untidy, oddly it's not. The book is intensely moving, raw and unmediated, told in real time with all the changes in perspective that brings, over the years, mixing the prosaic with the momentous. Health concerns, dreams, political initiatives spill out together, to provide the fullest picture yet of Mandela.

Verne Harris, director of the Nelson Mandela Centre of Memory and Dialogue, who led the project to select and assemble these archives, reminds us in a foreword that Mandela has become part of the creation myth of the new South Africa. As such, his public utterances were never entirely his own: even Long Walk to Freedom, his autobiography, was overseen by ANC colleagues, who realised how important the text would be to the movement. In Conversations with Myself, there are some wonderful transcripts of the ANC's Ahmed Kathrada reading the draft of Long Walk to Freedom to Mandela. Kathrada quotes several insertions from the book's publisher, pleading for more personal reaction at various seminal moments. In the transcript, a stony Mandela rarely expands further on his emotional state. He is mostly reluctant to answer the question: "How did it feel?"

It is precisely this reticence that makes Conversations with Myself such a necessary book. By going to his most personal of jottings, we finally get a glimpse of the man behind the mask. Luckily, it turns out that Mandela has always been something of a hoarder, as well as a copious note-maker, though many of his notes were seized by the police over the years, so there are inevitable holes.

The prison years, as one might expect, are particularly moving. "Until I was jailed I never fully appreciated the capacity of memory," says Mandela. Some of the abstracts are taken from his letters to his family and friends, many of which never reached their intended recipients, because they were blocked by the censors. Mandela made copies of some of these letters in a hardback notebook (it was stolen by the authorities but returned in 2004 by a penitent ex-security policeman). The notes reproduced here show Mandela's impeccably neat writing, with prison numbering and stamps on the top right-hand corner.

The book is a useful corrective to our tendency to see history through retrospectacles, to think that what happened was somehow inevitable. To the prisoners on Robben Island at the time, the overthrow of the once mighty apartheid state was a distant dream, yet still one worth fighting for. In these vivid pages one is reminded, for example, that prisoner 466/64 could have been freed decades earlier, if only he had agreed to be released into one of the black "homelands" that his jailers had created, and to renounce the armed struggle against apartheid. But he would not do so. And in an application to the University of South Africa in 1987, for an exemption from a Latin paper in his law degree, Mandela points out dispassionately that he is unlikely ever to actually practice, "as I am serving a sentence of life imprisonment".

What emerges here is a man devoid of self-pity, who is immune to the temptations of self-aggrandisement. At one point, he insists scrupulously to ghostwriter Richard Stengel that he consistently missed his target while undergoing military training.

One is reminded, too, of how steeped in history and the classics Mandela is. He read catholically, quoted liberally from War and Peace, and when preparing to launch "the struggle" consulted texts as diverse as Machiavelli, Clausewitz, Mao Zedong, and Menachem Begin. He studied the Anglo-Boer war in detail, and was later to use the Afrikaner arguments against his own jailers. But the Mandela we see here can also be abrasively self-critical. In a letter to Winnie, his wife, he quotes from As You Like It, "Sweet are the uses of adversity ... ", then says he has been looking over some of his earlier speeches and is "appalled by their pedantry, artificiality and lack of originality. The urge to impress and advertise is clearly noticeable."

The book is a valuable lens onto how Mandela made historic decisions  what he felt about communism, his Christian beliefs, the armed struggle, and the inevitable backlash by the authorities against the innocent bystanders, as well as the perpetrators. It is telling that, as a role model, he preferred Nehru to Gandhi. He also makes it clear that he only believed in non-violence as a tactic and not as a principle, though he could not say that at his trial. He discusses how he fully expected to be sentenced to death, what it's like when you think a judge "is going to turn to you and tell you now, that 'This is the end of your life'."

Painful personal issues are dealt with here too  for example, the allegation that he had once assaulted his first wife, Evelyn, "grabbing her by the throat". (Mandela's own version is that, during an argument, Evelyn pulled a red hot poker from the stove and lunged with it at his face, and he twisted it from her hand.) In 1968, when his 76-year-old mother had made her way down from the rural Transkei on her own, to visit him on Robben Island, Mandela writes: "At the end of the visit I was able to watch her as she walked slowly towards the boat which would take her back to the mainland, and somehow the thought flashed across my mind that I had seen her for the last time."

He was right: she died several months later, and he was not allowed to attend her funeral, even under escort. Ten months later his eldest son, Thembi, was killed in a car accident. Mandela's letter of 13 July 1969 to the commanding officer of Robben Island prison, asking to be present at his son's graveside, makes heartbreaking reading. It was refused.

Conversations doesn't shrink from the highly personal (Mandela made no attempt to control the book) and the strains of family life show up early. When Winnie arrives on a visit, she brings him "some silk pyjamas and nightgown " Mandela returns them, saying, "this outfit is not for this place." The back and forth between Stengel and Mandela on the question of sex while he is in prison, and what Winnie would do without him for all those years, shows Mandela's enormous self-discipline. "She has a life outside, she meets other men" Stengel probes. But Mandela expresses no jealousy. And when Winnie is herself jailed, Mandela sends her advice on how to cope, suggesting that she meditate for 15 minutes before bed. But Winnie is an entirely different creature from her husband. When he writes to her after a visit from their young daughters, saying how beautifully the girls were growing up, he recalls that "It was as if I had committed treason She reminded me: 'I, not you, brought up these children whom you now prefer to me!' I was simply stunned."

There are substantive political insights here, in particular Mandela's account of the negotiations that ended apartheid. When the authorities moved him away from his comrades, isolating him in another prison, he decided to accept the move, as this would allow him to open secret talks with the apartheid authorities, without consulting his comrades. "So what I decided to do was to start negotiations without telling them, and then confront them with a fait accompli." He was taking a huge risk.

One element gleaned from the calendar section is how important the gestures made around the world were to Mandela while he was locked up. The mass petitions for his release and the attempts to make him honorary chancellor of universities, as well as the birthday cards  often dismissed as silly, ineffectual gestures  were all clearly vital in keeping up his morale.

There are unexpectedly lighthearted moments too. We get Mandela the movie critic  he finds the end of Amadeus "somewhat flat", and the very juxtaposition of Nelson Mandela and "The Nerds Take Revenge" (I think he must mean Revenge of the Nerds) is startling. But not even this prepared me for the revelation that his printed "from the desk of Nelson Mandela" message pad has pictures of a grinning Garfield in the right-hand corner.

Peter Godwin is the author of The Fear: The Last Days of Robert Mugabe (Picador).