But Banga now admits letting go was harder than he thought. Fast-forward to last month. Now we're sitting on a whole floor of Central Plaza, the 78-storey tower in Wan Chai district. This is Banga's new family office. He and his two sons - Guneet, 34, and Angad, 30 - along with 90 workers are running their new venture, Caravel. Banga, guessing the first question, looks sheepish. What happened to golf and reading fiction? He laughs heartily and says: "I know. Retirement. It just hasn't happened."

Stepping down from Noble, which Banga and chairman Richard Elman built together, proved a challenge for Banga, 62, who has received India's highest civilian honour, the Pravasi Bharatiya Samman Award for Indians and people of Indian origin. "It was very difficult to leave Noble," he admitted. "I'd been involved from the very beginning, in 1984."

The plan was to retire, settle back and relax. "Everybody was so happy, and then my two sons and my colleagues changed my life. They badgered me to work with them." Banga shrugged with mock helplessness: "What could I do? I don't want to work, but here we are, starting again, all together in Caravel."

The name Caravel comes from the type of light sailing ship that the Portuguese used to explore unknown places. The firm was started three years ago as a way to bring Banga's sons - one working at Citigroup, the other at the private equity shop KKR - and him into a family business.

There are three pillars of Caravel: asset management, logistics and commodities trading. With the younger Bangas coming from an asset management background, combined with Harry's decades of shipping and commodity experience, there's no shortage of expertise.

Caravel is not another Noble, though 30 of Caravel's 90 staff followed him from Noble, bringing with them 35-40 years of industry experience. "Caravel is a start-up with a 20-year history and a billion-dollar balance sheet," Banga laughed. In 2011, he bought shipping services company Fleet Management from Noble Group. The company provides technical management and crews to shipowners across the globe. Fleet Management, which generates stellar growth, is now under the umbrella of the Caravel Group.

Cutting ties with Noble was difficult. "Emotionally it was very tough. "Even now, with friends, they say we did this, we did that. I say: 'No, Noble did that.' But in my mind, too, it was us, so a big part of my mind and heart is always going to be there.

"When you give the best part of your life, blood, sweat and tears, and it creates a Fortune 100 company ... well, you can cut the umbilical cord, but you can't forget the child."

Noble's chairman, Richard Elman, 74, is listed by as Hong Kong's 36th richest man with US$1.7 billion. Commodities are part of the planned Caravel platform, potentially putting it into competition with Noble.

How does Banga perceive Elman's view of the business? Banga chooses his words carefully.

"I respect him and however he feels. He's a great friend and colleague and I can understand, at his age, he may not like what he's seeing." He pauses. "But you know, I respect his call and he should respect mine and that we both have to move on." There's enough space in the world for everyone, he adds.

"Ultimately the dream is the same: to create something different and wonderful again," Banga said.

Elman declined to comment for this article.

Caravel is a group of businesses, each with its own management and chief executive. Harry and sons stay out of the day-to-day operations. "We concentrate on strategy and risk balancing and funding," Banga said.

In each Caravel company, one of the sons sits on the board, focusing on asset management while learning the ropes.

Banga seems happy with the way things are going. The sons bring very different experience from their private equity background. He says his own buy-and-sell-tomorrow approach complements the sons' longer-term investment horizon.

"Their experience and my knowledge gel perfectly," he said. "But this time, with the age difference, it will be more what the sons want to do than just what I want to do." Best of all, he sees them every day instead of just on Sundays. "Now we can go out for lunch together. It's very different. Sometimes they complain that they get an overdose of Dad."

It seems so many years ago when the man born Harindarpal Singh Banga was growing up in Chandigarh in northern India. He moved to Mumbai aged 16 to study. He then went to sea, ending up in Hong Kong, he says, more by destiny than design.

After working in London, Hong Kong and Geneva for Gulf International, a huge trading and shipping conglomerate, he met Elman, and together they co-founded Noble.

To Banga, Hong Kong is the natural base for Caravel. As far back as the 1980s, he could see this was where the world was going to expand, where the demand was going to be and ultimately, where the wealth would be created, he said.

"Those assumptions were right," he said, adding that it still makes business sense, as Hong Kong is in the middle of the world's fastest growing economies. From India to Japan, China to Australia, 60 per cent of the world's population lives here. "Business is so easy here," Banga said. "What else do you want?"

He likes living in Hong Kong, too, as it's now home for him and his wife, Indra. "The children grew up here, they don't relate to any other city.," he said. "For me this is where friends are. Hong Kong is where the life is."

When it comes to measuring success the second time around, it's not about making money or a name; that's all been done and dusted.

"It's like a scientist when he creates a new toy: he is so happy with his new thing," Banga said. "And hey, I'm just helping the next generation carry the baton forward."

And yet. it's more stressful this time, as there's further to fall. "Before, we were on the first floor - if you fell, you were not going to hurt yourself. But starting something from the 50th floor ... that's scary."

Banga knows he's in a good place: "I'm proudest of two things: of having a great family with integrity and morals, followed by helping a lot of people grow with me and creating global wealth for those associated with me."

He looks around at his enormous office, its walls lined with striking contemporary art. "I feel happy here. It's something that is my own. I created it - not inherited it - created it."