The global commodities market for banks has shrunk to about $4 billion from as high as $12 billion at the end of the last decade, partly as a result of reflecting stricter restrictions on banks trading with their own money.

JP Morgan (>> JPMorgan Chase & Co.), Deutsche Bank (>> Deutsche Bank AG), Barclays (>> Barclays PLC), Bank of America Merrill Lynch (>> Bank of America Corp) and Morgan Stanley (>> Morgan Stanley) have all either reduced or closed their European power and gas trading units.

"Liquidity is sinking and at the same time capital requirements are rising. The financial market is realizing that there are no good margins to be made here," said Stefan Dohler, senior vice president for optimization and trading at Swedish utility Vattenfall .

Financial sector derivatives trading on Germany's European Energy Exchange (EEX) fell to 40 percent of overall volume in 2013 from 50 percent a year earlier.

Analysts warn that with the resultant rise in the role of utilities, which own most of Europe's power and gas assets, Europe's energy markets are growing less attractive for investors.

Dieter Helm, an energy expert at Oxford University, said in a research paper published this month that a market dominated by a small number of producers would deter necessary investment in the power sector which is needed to upgrade ageing power plants and networks.

"Monopolies and collusive oligopolies can maintain their market power if they can deter entry - and then extract monopoly rents if they then keep the market tight by themselves limiting investment," Helm said.

"The result is that we have a system without adequate investment approaching a capacity crunch (in Britain)."

RISE OF NEW TRADERS

Along with utilities, commodity merchants such as Vitol , Glencore Xstrata (>> Glencore Xstrata PLC) and Mercuria, which are less affected by growing regulation, are stepping up their activity.

Geneva-based Mercuria is in advanced talks to buy JP Morgan's commodities business.

Small hedge funds, such as Cumulus Energy (owned by City Financial) and Danske Commodities, have also swept up some former bank traders.

"We've taken a lot of sales business that was dropped from the big banks and also stepped in as counterparty in outright power and gas trading," one London-based hedge fund trader said.

The banks that remain include continental lenders such as Scandinavia's Nordea (>> Nordea Bank AB), France's Societe Generale (>> SOCIETE GENERALE) and ABN Amro of the Netherlands.

"What you have left on the financial side now is specialist trading houses and smaller but focused bank energy desks," one energy banker said.

Two big banks not scaling back are Citigroup (>> Citigroup Inc), which is expanding its European power and gas trading and sales desk, and Australia's Macquarie Bank.

"Some big players got out of the market but some others see this as an opportunity to take over that business," said EEX's Chief Executive Peter Reitz.

Another major new player is Brazil's BTG Pactual , which has been sweeping up former energy traders from big U.S. and British banks.

"They've really stormed into the field big time, hiring a lot of former traders from banks here in London," said a power trader with a U.S. bank still active in Europe's power and gas markets.

(Editing by Jason Neely)

By Henning Gloystein and Vera Eckert