There may be much to despair about India, from its broken state to endemic corruption, but after chasing its raucous election campaigns for more than two decades, I come away with deep optimism that democracy works in India.

What else explains the fact that India tosses out its governing class more often than any other country I know? At one level, political power in India is supreme. No business person will dare speak out against the ruling party, hundreds of sycophants constantly surround the country’s major leaders and for all the glamour and intrigue that Bollywood actors, godmen and astrologers bring to Indian elections, they are just accessories to the political stars. India’s bureaucrats may lord over the people but in the end even they serve at the mercy of politicians who can transfer them to the boondocks on a whim. A recurring image from our travels is that of voters stretching their arms skyward by the thousands, as if to reach up and touch their chosen leaders descending by helicopter to the rally ground. It is a constant reminder that real power in India resides with the political class.

And yet, for all their clout, the odds are against Indian politicians holding on to their offices. In theory the seated government has big advantages, starting with the fund-raising capacity to meet the ever-growing expenses involved in fighting an election. It can dole out favours and contracts, so business donors typically steer the bulk of their contributions to the ruling party to keep it happy. Yet incumbents don’t usually win, challengers do. Voters, though glad to pocket expensive campaign gifts, still vote their own minds. Ultimate power resides, then, not with the candidates or their moneybags, but with the Indian voter.

India became a democracy when it was still very poor, and perhaps more than the rich, the poor cherish the vote as a great leveller, their memo to the powerful reminding them who calls the shots.

We have often heard this undertone of Schadenfreude from Indian voters, relishing the moment the powerful incumbent will fall. Even if many toppled leaders stage a comeback, the fall is a chastening experience. Indian political power is hard won and fleeting.

For some political analysts, there is something inherently dysfunctional in the way Indian voters keep flipping governments.

In many states, dozens of parties compete in the elections and the winner often needs only about a third of the vote to take a majority of the seats. Falling short of that, they find themselves scrambling for allies to help form a government. Small shifts in the vote, or the allegiance of one small alliance partner, can make or break state or national governments. The whole thing looks like a recipe for instability.

But minority governments, built on compromises among rival parties, are not a special problem of Indian democracy. They are a standard feature of parliamentary democracy, particularly in countries that were formed by merging autonomous principalities into a unified state — as India was. A multiparty parliamentary democracy can produce serial political and economic crises, as it has in Italy, but also long-term success, as it has in Germany.

Democracy on the Road: A 25-Year Journey through India by Ruchir Sharma Allen Lane/Penguin Random House pp 389; Rs 699

Have weak minority governments hurt India’s development? History suggests not. The economy limped along at the so-called ‘Hindu rate of growth’ under mostly strong Congress governments until the 1980s, then started to reform and pick up speed under the weak coalition governments that followed. India’s complex entire country behind aggressive economic reform and double-digit growth, the way China has. But states from Bihar to Gujarat have achieved this feat, and more will. Together they are likely to keep the economy developing at a respectable if sub-miracle pace.

India has so many parties because it has so many different communities, separated by caste, religion, tribe or language and each one wants its own representative.

This is a fitting arrangement for a democracy encompassing the ‘Many Indias’. While in some opinion polls Indians express a growing desire for a strong leader, unshackled from an often gridlocked Parliament, the electoral reality is that the country rebels against domineering political bosses.

Ever since Indira Gandhi imposed the Emergency, and fell in the backlash, no Prime Minister has been able to gain political momentum without triggering fears that they were growing dangerously strong, and inspiring the fragmented opposition parties to unite. Indira, Rajiv, Vajpayee — all of them were undone by an alliance of normally squabbling opponents. Modi may face a similar obstacle.

Supporters praise Modi for raising India’s stature in the world. But more than once we have seen Indian leaders — from Manmohan Singh to Chandrababu Naidu—lionised by the global elite from Mumbai to New York, only to be thrown out by Indian voters who care more about the government’s impact on their daily lives than about such cosmic concerns as India’s image in the world. The more time you spend outside cosmopolitan cities like Mumbai and Delhi, the better your chances of understanding how India really functions.

We got the 2003 election wrong in Rajasthan because we didn’t get out of the big cities; if you miss the rural campaign you are likely to miss the story entirely.

Impressions gleaned on the road are inherently skewed by the route you choose, the voters you happen to meet, and the much larger pool of people and places you miss. We are now well aware of that, so we have tried to make sure we follow a carefully researched route through the most important swing constituencies, and the most important states. On our twenty-seven election trips to date we have typically covered between 1,000 and 1,500 km over about five days. In total we have driven a distance nearly equal to a lap around the Earth. We have been to more than half of India’s twenty-nine states and to the ten most populous and politically important ones more than once.

Voters in all these states express impatience with the pace of progress, and anger at the unresponsive bureaucracy, but not all take it out on politicians with the same intensity. Over the last three decades, among the ten most populous states, UP and Karnataka have never given a chief minister consecutive terms. Rajasthan, Maharashtra and Tamil Nadu have done it only once. Alongside these hotbeds of anti-incumbency are states like Bihar, Madhya Pradesh and Andhra Pradesh, which have been less hostile to seated leaders. At the far end of the spectrum lie Gujarat and especially West Bengal, which have given their chief ministers and ruling parties extended runs in the halls of power.

Narendra Modi came to power sounding like India’s version of Ronald Reagan, promising to minimise the role of the government, only to reveal himself as more than willing to use state power to meddle in and mould the economy. His response to suggestions that Delhi privatise the failing state banking sector has been that if he does that, how can he steer loans to India’s aggrieved farmers? The Modi model of statism is a perfect fit for his own self-confident character, but by promising India that he alone could fix its myriad problems, he ran the risk that many voters would hold him personally responsible for unfixed problems.

Delhi is not the place to look for big-bang reform. Once politicians seize the reins of national power they find it difficult to let go. If they push reform they do so incrementally and under pressure of hard times, then back off when prosperity returns.

Today I find myself less impatient for radical change, more confident that while India may never be a miracle economy, its entrepreneurial energy, spurred on by its most competent chief ministers, is enough to keep the economy growing at a steady pace of six to seven per cent.

We have seen this energy unleashed in states as varied as Gujarat, Bihar, Rajasthan, Madhya Pradesh and Delhi. They have grown rapidly for long stretches and economically outperformed not only the rest of India but much of the developing world, typically under a leader with strong regional roots, pushing a strong development agenda. India doesn’t grow as one economy, it grows as many, less like the United States than like the European Union. It is less a country than a continent, in fact more diverse in its communities and languages than even Europe or the Middle East.

The real strengths of our democracy — both economic and political — lie in this diversity. Consider how many empires have tried to dominate the Middle East, only to be swallowed by its warring tribes, which are in fact less numerous and varied than India’s. In no country are the community and family roots of political battles more complex or intense, or the behind-the-scenes battles to build winning alliances more fierce.

Those who fear that rising nationalism threatens India’s democracy also tend to underestimate the check provided by sub-national pride. Many Indians still see themselves first as Bengalis, Maharashtrians, Tamils, Gujaratis or Telugus, and they are much more likely to support a strongman (or woman) at the state level than in Delhi.

The 2019 election is being cast as a nationwide showdown between Modi and the rest, a referendum on India’s appetite for strongman rule and commitment to democracy. More likely, the election will shape up as a series of state contests. Modi is likely to win the most votes in many of the critical northern and western battleground states, but the outcome will depend on whether the Opposition parties work together to unseat him. Facing unified alliances, the BJP could win a third of the popular vote, as it did during the Modi wave of 2014, yet lose its majority of seats in the Lok Sabha.

The 2019 ballot will offer a choice of two different political visions, one celebrating the reality of the many Indias, the other aspiring to build one India. It will be a hard-fought battle, but either way I know where I will be when the carnival begins. Back on the road, confident that in an era when democracy is said to be in retreat worldwide, it is thriving in India.

Excerpted with permission from the publishers, Penguin Random House