NEW YORK (Reuters) - Bank of America Corp (>> Bank of America Corp) agreed to pay the insurer American International Group Inc (>> American International Group Inc) $650 million to settle long-running legal disputes over defective mortgage-backed securities that the bank sold in the run-up to the financial crisis.

The settlement which the parties announced on Wednesday ends securities fraud litigation brought by AIG against the bank.

It also removes the biggest obstacle to the bank's $8.5 billion settlement with investors in mortgage securities issued by Countrywide Financial Corp, the mortgage lender Bank of America acquired in July 2008.

AIG agreed to dismiss litigation in New York and California that accused the bank of fraudulently causing billions of dollars in losses by selling it shoddy mortgage securities. It originally sought $10 billion of damages in the New York case.

For the past three years, AIG has also led opponents of Bank of America's $8.5 billion settlement in June 2011 with institutional investors to resolve claims that Countrywide misrepresented the quality of home loans underlying $174 billion of mortgage-backed securities it issued before the housing crisis.

Twenty-two institutional investors, including BlackRock Inc (>> BlackRock, Inc.), Allianz SE's (>> Allianz SE) Pimco and Metlife Inc (>> Metlife Inc) had agreed to the settlement, with the Bank of New York Mellon Corp (>> The Bank of New York Mellon Corporation) serving as trustee.

AIG claimed the settlement undervalued its claims, and had been appealing January's approval by Justice Barbara Kapnick in New York state court of the bulk of the settlement.

In agreeing to drop its appeal, AIG said it will accept a pro rata share of whatever investors recover. A group of funds known as Triaxx and a pension fund for Chicago police remain as objectors.

A spokesman for Bank of New York Mellon declined to comment. A lawyer for Triaxx declined to comment, while a lawyer for the Chicago pension fund did not immediately respond to a request for comment.

Bank of America said it had set aside reserves by June 30 for the $650 million settlement, which also resolves three actions it brought to collect mortgage insurance proceeds from AIG's United Guaranty unit on loans it had originated and serviced.

The cases are American International Group Inc et al v. Maiden Lane II LLC, U.S. District Court, Southern District of New York, No. 13-00951; American International Group Inc et al v. Bank of America Corp et al, U.S. District Court, Central District of California, No. 11-10549, and In re: The Bank of New York Mellon, New York State Supreme Court, New York County, No. 651786/2011.

(Reporting by Karen Freifeld; Additional reporting by Jonathan Stempel; Editing by Lisa Von Ahn and Diane Craft)

By Karen Freifeld