OPEC failed in its third attempt to resolve a deadlock over oil production after divisions between allies Saudi Arabia and the United Arab Emirates spilled into public and into global financial markets.

The group called off a meeting scheduled for Monday with Russia-led oil producers after the U.A. E.—typically one of Riyadh’s most dependable supporters in the group—refused to agree to a Saudi-backed deal to boost output, according to people familiar with the matter. The deal, tentatively approved by the rest of the Organization of the Petroleum Exporting Countries and the Russia-led group, together known as OPEC+, calls for unleashing millions of barrels a day of bottled-up crude to help tame steadily rising oil prices.

Oil prices rose to fresh multiyear highs after the OPEC meeting was called off as investors weighed whether the spat could endanger a near-term deal for more barrels. Brent crude, the international benchmark, rose 1% to $76.96 a barrel, the highest level since late 2018.

The U.A.E. says it is asking for scope to produce more of its oil under any accord. Its unwillingness to compromise comes as Riyadh and Abu Dhabi—neighbors, military partners and traditional OPEC allies—diverge on several fronts, economic and geopolitical.

The U.A.E. has invested heavily in its own output—an effort aimed at pumping more oil, faster, to finance a transition to a more diverse economy. “The U.A.E.’s energy strategy has evolved to take into account energy transition, the speed at which it wants to develop its resources,” said Roger Diwan, an oil-focused vice president at energy consultancy IHS Markit.

Gulf officials and Arab diplomats say the standoff also represents a new assertiveness by the U.A.E.’s heir apparent, Abu Dhabi Crown Prince Mohammed bin Zayed, amid economic and policy differences with Saudi Crown Prince Mohammed bin Salman.

It’s unclear how far Sheikh Mohammed of Abu Dhabi is willing to take the standoff. OPEC didn’t say when it might meet again. Saudi Arabia, by far the biggest producer in the group, has in the past taken unilateral action amid OPEC deadlocks.

The U.A.E. expects it will get a better deal by holding fast for now, according to a person familiar with the matter. If OPEC doesn’t reach a deal, that will free up the U.A.E. to pump more oil anyway, and do damage to Saudi Arabia’s reputation for holding the cartel together, according to people familiar with the country’s thinking.

A spokesman for the Saudi energy ministry said the disagreements in the oil market weren’t connected to political differences.

The two crown princes, both of whom run their respective countries’ day-to-day affairs, bonded over a desert camping trip about a half decade ago, and initially fostered close ties between their two countries.

Their alignment on regional issues started to diverge more recently. In 2019, the U.A.E. said it would pull its forces from a Saudi-led coalition fighting in Yemen.

A deal by the U.A.E. for peace with Israel led to differences within the leadership of Saudi Arabia, the Arab Gulf’s traditional power broker.

More recently, according to officials and diplomats, the U.A.E. has been alarmed at the speed at which Riyadh has reconciled with Qatar, a longtime irritant to both. Riyadh and Abu Dhabi had together led a yearslong blockade against Qatar until the Saudis and Doha agreed to a U.S.-brokered truce early this year.

“The OPEC clash is not something that just erupted overnight,” said a senior Saudi official. “The U.A.E. and the kingdom just don’t see eye-to-eye in the Middle East.”

Shortly after the Doha deal was announced, the Saudi government said it would stop giving government contracts to businesses that have their regional headquarters outside the kingdom—a move that threatens the position of the U.A.E. city state of Dubai as the region’s hub for business, trade and tourism.

“It was only a matter of time before the relationship deteriorates and spills over at the OPEC meetings,” a senior Arab diplomat said.

The OPEC standoff threatens to hobble the cartel at one of its most pivotal periods in recent years. After the debut of U.S. shale oil, the cartel’s power to sway markets appeared greatly diminished. It teamed up with Russia, adding market share—and pricing power. Then, Covid-19 struck.

Early last year, the Saudi-Russia group agreed to cut its collective output by some 9.7 million barrels a day, taking out the equivalent of about 10% of 2019 global demand, when economies were shutting down. The move supported prices. As economies wind back up, the group has restored a big chunk of that. On Friday, most delegates agreed to a deal to gradually undo the remaining cuts, some 5.8 million barrels a day, by increasing production by 400,000 barrels a day each month through late 2022.

All that extra pumping capacity has given OPEC and its allies more influence over crude markets than they have enjoyed in years—if they can all agree on how to use it.

The U.A.E. balked at signing on to the deal, saying it wants a boost to its own production quota inside the broader OPEC+ framework. In November, it said it would pump $122 billion into plans to boost production by about one million barrels a day by 2030.

Write to Summer Said at summer.said@wsj.com, Benoit Faucon at benoit.faucon@wsj.com and Stephen Kalin at stephen.kalin@wsj.com