"I've chosen this because it is a narrative, and narrative songs are particularly hard to write. It's a narrative that rhymes too, you always see people who do this really well and then they trip up on a couple of lines, which blows it. It's like that great quote from a writer who said that writing a novel has to be a continuous thread of imagination, without any break in the chain. And it's true, because if you lose your reader in a narrative, you lose them completely.

"I first heard this as a kid, and I know it backwards and could sing it to you. It's such a good story and I even love the sound of it and the finger clicking and the vocals - it's a strange piece of music. It obviously stuck in my psyche at a young age, along with Val Doonican’s “Paddy McGinty’s Goat”, but I figured your readers wouldn't really want to listen to that as much as “Big Bad John”!

“Bowie did the storytelling so well with “Space Oddity” and Pulp did it brilliantly with “Common People” but they're hard to pull off, and when people do it I’m always quite thrilled. This was probably one of the first [narrative-led songs] I ever heard as a kid. I think for a long time I thought it was James Dean the actor who sung this, so I had an even more romantic halo around the song than it probably deserves.

“There was one song I remember my parents would sing to me: "Over the wall we go / All coppers are nanas / Over the wall we go / Leave ‘em alone and sing Happy New Year." It was a song about breaking out of prison, which my parents didn't approve of as my father was a magistrate. I recently discovered that that was a Bowie song, which was hysterically funny, because it sounds more like Dick Van Dyke. It was in that period where Bowie was looking for an identity, when he was trying different personas as if he were trying on different suits."

"Talking Heads are, I think, one of the last of the great bands - I'd include The Pixies in that too. From the start I loved their first single, “Love → Building on Fire”, which I have. It's a strange title. And they're just idiosyncratic, the way they don't give a shit about the way they looked - they used to wear cardigans onstage, like we did.

“David Byrne was so nerdy and Tina Weymouth was astonishing. I thought she was the most charismatic bass player, and so generous - she always used to stare at David Byrne, with an intense look, that meant whenever you looked at her, you wanted to look at what she was staring at. So she boosted David Byrne's charisma by that look and I always thought that was very generous of her. She in herself I found so fascinating to watch, with that gorgeous early-years pageboy haircut. I definitely had a crush on her.

"At boarding school, I saw this clip of Talking Heads performing “Psycho Killer” on Whistle Test with a whole load of people and everybody laughed at them. I told them this was the future of music, and had quite a row with one particular person, who, 30 years later, actually wrote to me and said, ‘By the way, you were right about Talking Heads.’ I thought they were captivating. Everyone laughed at them and how they moved. You can't imagine that today, but back then that kind of music was really strange and eccentric. They went on an amazing journey with Brian Eno, and they were really the reason we wanted to work with Eno. Talking Heads are one of our favourite bands, but you won't find any Talking Heads in James, there's no influence there, but we have so much respect for them and love them so much.

"I'm not going to say the lyrics were my favourite part, but I really appreciated the fact that Byrne was trying to avoid the clichés of most lyric writing. Sometimes though, I felt he did that too much and they sometimes became too clever instead of heartfelt. But his phrasing was fascinating, as was the tight tension there, and being able to get a groove into the music.

“My favourite album is More Songs about Buildings and Food. The first four tracks take your breath away and leave you feeling so joyful. They were very uplifting, but never felt like they were escapist uplifting. Then Talking Heads became a massive funk band which I was less interested in in some ways, I was more into the taut four-piece, but I always appreciated the fact that they were always experimenting and they weren't standing still for a long time.

"They took risks. I saw them at a festival in Scotland where they played an album that hadn't been released and they played most of the record. The audience at first were furious because they weren't playing “Psycho Killer” and they wanted something that they knew. But then the final six songs were off the Remain in Light album, when everyone dropped in and were saying, ‘This is amazing! What is this?’ And I thought that was such a ballsy performance to do that.

"We toured with Tom Tom Club around America. It was a little disappointing, we were a little sad because they were quite bitter in that they felt they'd taken all the songwriting credit. But being one of the biggest bands in the world and having that experience, you can't be bitter about the money, you know? That's not in our hands. I understand that, but that was very sad to see.

"I nearly went to meet David Byrne. I was going to fly to America to work with Angelo Badalamenti, as a TV show put us together. The night before, he was going to play with Richard Thompson. I got sick and couldn't go, but I was meant to fly in early to see them. I would have been with the camera crew, who all had a car crash coming back from the gig and ended up in hospital. I just have to tell myself that I dodged that bullet, at least."

"This is Nick Cave's second band, the band that brought him to the UK. They were the most astonishing, visceral live band I've probably ever seen - up there with Iggy who was a very dangerous performer at the time too. You didn't know if he was going to jump on you or hit you with a mic stand.

"The first time I saw The Birthday Party they were opening for Bauhaus. Nick Cave came on with a vicar's dog collar and a Bible. The bass player had an Australian cork hat. Nick Cave was slapping the audience in the face with the Bible in the front row and tearing out bits of the Bible and shouting ‘Thou shall not, thou shall not!’ They were bonkers, and he was matched by the menace of the band.

"There's no good footage of The Birthday Party online, except this. This clip will give you an idea of how outrageous they were live. On this German TV show, they're smoking cigarettes and drinking whiskey in the song, and keep your eyes on the bass player, who does some things at the end which are physically impossible. They were a complete train wreck as a band, in terms of heroin mainly, I think.

"They took music to an edge which I'd never witnessed before. I think Nick Cave is the greatest artist of my generation, who maintained creativity and who was constantly moving and shifting. Anyone who is interested in Nick Cave should watch this clip, because it will blow your mind. Turn the song and the bass up loud. Keep your eye on the bass player, it'll make you burst out laughing."

"I first heard about Regina Spektor years ago, when I bumped into Gordon Raphael, the producer of Soviet Kitsch and who did The Strokes album, at a festival. He was raving about a record he’d just made with this crazy young woman who played a drum stool with a drumstick while playing piano and singing at the same time.

“So I’d always wanted to see her, but I think it was for the next album, when she came to Los Angeles and played at The Greek Theatre - which is probably the most beautiful venue in LA to see anyone. I went with my wife and two other people, and we must have wept through about 50% of the concert - partly through joy at seeing Regina. I think Regina has more heart in her out of anyone, apart from maybe Springsteen, that I’ve seen - probably more than Springsteen even. I think that’s his great quality, that he puts so much heart into what he does. But I think she has so much and it’s just a love-fest to some degree.

"Her work is so feminine and constantly intelligent, her musicianship is remarkable, her voice is beautiful, her lyrics are wonderful - they catch you unawares. In one song she’ll have more ideas than most bands will have in an album. I just fell in love with her and for about three months after that gig I couldn’t write a song. I went back to every one of her records, playing them over and over again, trying to understand what she was doing, and was none the wiser.

"Later, I was lucky enough to become her friend when we played some gigs together. But she just blows me away every time. She’s so vulnerable onstage. Again, there are so many songs - “Man of a Thousand Faces” is probably my favourite song of hers - but I could name you about five or six that would be in my top 20, which I want played at my funeral. I love this one partly because she sings some of it in Russian, and some other character comes out when she sings in another language. I’m really interested in that, that there’s the American Regina, and then there’s the Russian Regina. I love what she does with her voice, and her passion.

“The other clip I would have chosen, is “Poor Little Rich Boy”, where she plays the drum stool at the same time, and she dedicates it to Patti Smith. Again, it’s not my favourite of her songs, but just look at this person, playing a drum with one hand, the piano with another and then singing, all at completely different timings. Who the hell can do that?! She’s a remarkable musician, she’s a delight.

"One video will show you the playful side of Regina, and the other will show you this light touch. "Après Moi" is when she gets more serious. I don’t know why I love this song so much, but I just find it passionately brilliant. She never talks about her songs to anyone, so no one ever finds out her process, which is a fascinating mystery I think."

“Obviously I’m attracted to lyrics, and finding that balance between lyrics and poetry is such a fascinating thing. The latest anthropological studies are suggesting that calls and song were used to hunt and they came before language. Songs cross species and cross into the multiverse, so it’s a big deal to me.

"The song that probably made me into a singer was “Birdland” by Patti Smith, but that’s a hard listen for most people. It’s an 11-minute, improvised song. I chose “Hey Joe” because it blows my socks off and not many people have heard of it. I was interested too by Patty Hearst and her story. She was a billionaire's daughter and was kidnapped by the Symbionese Liberation Army, who were a protesting guerilla outfit. Then three or four months later, she ends up robbing a bank with them, with an automatic weapon. Finally, they were caught, some of them were shot, she was arrested, and a huge court case took place.

“In this song, she melds the story of Patty Hearst with the somewhat misogynist lyric of “Hey Joe.” It was a song I’d never really loved, I loved Hendrix’s guitar on it, but I never really took to his voice too much. Patti Smith turns this into an amazing art piece: “Patty Hearst standing there with the Symbionese Liberation Army flag with your legs spread / I wondered if you were just dead or if you were getting it every night.” Woah, what a fucking lyric. Who does that to a song? It’s so fucking ballsy and out there.

“At the end, you can tell that Patti’s trying to work out why did this girl do this? Because in the end it was down to Stockholm syndrome, where she ended up associating with her captors. She was of the Randolph Hearst family, so they had the money to get great lawyers and they argued her case and she got out. But it’s an amazing lyric where Patti Smith sings: “I am no pretty little rich girl / I’m nobody’s million-dollar baby / I’m nobody’s Patsy anymore.” I bet Patti Hearst felt very alive, robbing a bank having been brought up with a very pampered existence.

“It’s an amazing song with Tom Verlaine from Television playing guitar, making the guitar replicate birdsong at the end. It’s live recorded, it’s a fucking mess, and it’s a genius mess. There’s no great footage of it and she hardly ever plays it live. But I miss this song, and I wish she’d play it live more. I’ve seen her play it live once, with Flea on bass - it was fucking mind-blowing.

“Leonard and Patti are the two poets, in my eyes, who became musicians. Their lyrics are the ones that bridge the gap between poetry and music. The only other person I think who has done it as well is Kae Tempest. I’m not so much of a fan of her music as much, but she’s such a remarkable being that it doesn’t matter. She’s a genius. My tenth song would have been “People’s Faces". It's remarkable, it makes me cry, but obviously from this conversation quite a few things make me cry. Damn, I should have put Kae Tempest in - I’ve made a big mistake!”

All The Colours Of You is out now