He should be. He's sitting in a flash, eye-burningly trendy hotel in South London promoting the new film he wrote and directed, Sing Street, a feel-good mix of '80s pop music and teenage romance that has been warming the hearts of audiences and critics.

We should have a lovely chat about how fun '80s pop is, how talented his cast were, how satisfying it is to have this highly autobiographical teen romcom of a Dublin kid forming-a-band-and-getting-the-girl reviewed almost universally as "like The Commitments, only better".

It would have been insufferably wonderful. So it's a huge relief instead to find that Carney, 44, is a bundle of dark and hilarious neuroses and can't help but pick holes in his own film, wishing he'd done it better.

He wonders if his lead actor is too good looking, and the film would have been funnier if he hadn't been.

He mocks his lead character's "happy sad" mantra, which in the film is worthily employed to make sense of pop music (specifically The Cure), adolescence, and the grim realities of '80s Dublin, as "a childish idea that I thought 'that's the kind of shit that kids get so attached to'."

By the end of our chat he's reviewing his own life, and finds it wanting.

"The world is so depressing and life is so depressing and hard, and hitting 40 is so depressing and hard," he confides, in an incongruously warm and convivial Irish lilt.

"You hit 40 and things start to slow down a bit and you go 'actually have I survived that well? Were there issues in Ireland, were there things in Ireland that I regret? Were there things in my family that I'm worried about?'

"Was Duran Duran just escapist nonsense? Would I have been better listening to classical music and Bach and Mozart? These are the kind of questions as an adult I'm asking watching this movie."

Carney is not your typical film director. And you get a sense this is part of the reason he's doing so well, and why his previous work like the breakout Sundance hit Once and the more recent Begin Again won so many fans.

He's earnest but self-deprecating, a romantic pessimist, a clown, and he makes movies with personality.

He's also not very good at delivering studio-approved publicity messages – witness the mess recently when he called Sing Street "a small personal movie with no Keira Knightleys in it" and said the actress, who had starred in Begin Again, as "let's face it, a model" who couldn't act.

He later apologised, said he felt like a "complete idiot" and Knightley had been "nothing but professional".

Idiocy or not, it was the kind of worried emotional honesty that suffuses his work, stirring in charm and quirk to leaven the sweetness. Sing Street is a very personal and quite autobiographical movie. Like his lead character, Carney too moved from a private school to a Dublin inner-city comprehensive – the Synge Street School of the title – after his family hit money problems. He had to learn how a kid like that survives in a world like that – and how he achieves the holy grail of the teenage boy: girls.

"I didn't make mixtapes but I certainly tried to influence (them with music)," he says. "I took girls into piano shops and played them piano songs and tried to be whatever, Robert Smith, or Frank Sinatra when I was 19. I tried to seduce women through music. That was my main job when I was young.

"With my face you need to have an extra string to your bow. Humour was a good one but music was the killer because music is just ... immediate. It's so immediate."

He pauses to reflect on that and his writer's instinct kicks in. He wonders if he missed a trick by casting "himself" in the movie as a good-looking teenager.

"In a sense this film could have been better if Ferdia (Walsh-Peelo, who won the part in an open audition) hadn't been so good looking. If you'd picked a really dweeby unfortunate-looking kid, it could be a better film. It could be a really funny film. Because the guy's confidence is just insane then.

"(But) with Ferdia he's like Harry Styles or something. That kid's fine, he's not going to have any problem getting laid.

"I don't know what I had when I was young. In all honesty I don't know what I had that got me through life."

What he did have was confidence, he says – a confidence he put into his characters in Sing Street, but a confidence he wonders was misplaced in his own youth.

Once he says, at Synge Street School, the school bully tried to pick a fight and hit him in the face. "I just stood there and said 'you can keep hitting me all day, I'm not going to hit you back'. If I'd put that scene in the film people would just go 'that's unbelievable'."

But he worries about it now. "Was I a confident kid or was I a full of shit kid and bluffing and just getting through the day? And you pay that back. If your life is spent just getting through each day alive that builds up some tension that has to be released at some stage later in life.

"Whatever issues I have now in adult life I can trace back to those things. The survival of school and getting through school and the sensitivity I felt as a kid I'm paying back as an adult. I didn't pay for it as a kid, I survived as a kid – I did OK."

It's an odd thought. That the writer mistrusts the very success he grants to his characters.

After he decided to centre the film on music, specifically the distinctive sounds and outre looks of '80s pop, the "obvious first port of call" was to contrast that with the "greyness and the drabness of Dublin, not just architecturally and weather-wise but fundamentally, in terms of character and story".

He mixed in dark textures – divorce, poverty, class, the suffocating lack of hope of that era – "but at the same time I didn't want to make Dancer in the Dark".

And while the plot is ostensibly boy-meets-girl, at the end it is dedicated "For brothers everywhere", as the fraternal sub-plot plays out to the very last scene.

"It became central," he says, of the relationship between the main character and his older, wiser, drop-out brother.

It became central not just because it deepened the drama, but because it chimed deeply with him. He dislikes the modern blurring of the generations, of parents who "like Mumford and Sons".

Carney didn't learn how to be a teenager from his parents, he learnt it from his older siblings.

"This was never the original intention of this movie but it became clear in living life and making the film that those two issues were crossing over and affecting the drama of the film.

"And then it became very clear to me that the question of what degree is anything any good that I ever do, to what degree am I grateful or in debt to my elder siblings.

"I'm the youngest of a family of four so I feel that everything that I ever (did), every guitar I picked up, every handstand that I did, or when I said 'I'm a Mod now' ... It was to get the attention of my siblings and to get their validation.

"I think that my career is based on getting approval from my older siblings."

Sing Street opens on July 16.