The government-owned companies do not lend money directly, but underpin the U.S. housing market by guaranteeing most new mortgages in the country.

The Federal Housing Finance Agency Office of Inspector General said in a report that the purchases from smaller lenders raises the exposure of the two companies.

"Smaller and non-bank lenders may have relatively limited financial capacity," the watchdog said. "The enterprises face an increase in the risk that those counterparties could default on their financial obligations."

Fannie Mae and Freddie Mac, which were seized by the U.S. government in 2008 during a housing market implosion, purchase loans from lenders and package them into securities that are then sold to investors.

In the past, the two companies bought most of their loans from the country's largest banks. Small lenders generally dealt with larger banks, who in turn sold them to Fannie and Freddie.

However, tighter regulations have made big banks such as Bank of America Corp and Wells Fargo & Co reticent to buy loans from other lenders, so more small banks and non-bank enterprises now sell directly to Fannie and Freddie.

Freddie Mac's top five mortgage sellers provided 70 percent of its loans in early 2011, but only 44 percent in late 2013, according to the report.

(Reporting by Jason Lange. Editing by Andre Grenon)