The discussions centre on whether RBS would be willing to raise a settlement offer of 82 pence per share agreed with the bulk of claimants by another 20 pence per share for the holdouts, the sources said.

A third source said RBS had not yet raised its offer to the investors. A spokeswoman for RBS declined to comment.

RBS last week agreed terms with organisers of the RBoS Shareholder Action Group on a settlement over the case, in which the investors allege the bank misled them over its financial health at the time of its bailout in 2008.

That left only a die-hard faction of a few thousand individual investors pursuing the case. Those investors need additional funding of around 7 million pounds if they are to keep the case alive, the sources said.

Lawyers representing the RBoS group have until Thursday afternoon to tell the judge on the case whether they will proceed with their claim.

The deals have so far cost RBS close to a billion pounds, but a final settlement with the holdouts would spare the lender the embarrassment of having the lowest point in its near 300 year history raked over in court.

The bank's former Chief Executive Fred Goodwin, named in the lawsuit as a defendant, would have had to justify his decision-making at the time of the bank's 12 billion pound cash call and subsequent bailout by the government during the 2008 crisis.

RBS investors, including thousands of current and former RBS employees, had alleged the bank's former executives deliberately hid its over-stretched finances and failed to disclose that the regulator had ordered it to raise cash.

RBS, which remains more than 70 percent state-owned, denies any wrongdoing and said its former bosses did not act illegally.

(Reporting By Andrew MacAskill and Lawrence White; Editing by Susan Fenton and Mark Potter)

By Andrew MacAskill and Lawrence White