Alibaba's could be the largest initial public offering in history and has seen "overwhelming" interest, meaning Yahoo's 23 percent stake could be worth more next week than it is now. Yahoo stock pulled back from a 14-year high and fell 0.8 percent to 42.55 with 71.7 million shares traded, more than double its 10-day average of 34.2 million.

"There is some nervousness out there, so some money is coming out of the high-flyers and some of it is people getting ready to raise some cash to put to work to Alibaba," said Ken Polcari, Director of the NYSE floor division at O’Neil Securities in New York.

Heavily-traded Facebook (>> Facebook Inc) and Netflix (>> Netflix, Inc.) lost nearly 4 percent each. The S&P technology sector <.SPLRCT> lost 0.6 percent as the worst performing of the 10 major S&P sectors.

Investors also exercised caution ahead of the policy statement from the Federal Reserve on Wednesday, which could provide clues on the timing of an interest rate hike.

"There is a lot of wait-and-see for what happens Wednesday, if anything new happens," said Tim Ghriskey, chief investment officer of Solaris Group in Bedford Hills, New York.

The Dow Jones industrial average <.DJI> gained 43.63 points, or 0.26 percent, to 17,031.14, the S&P 500 <.SPX> lost 1.41 points, or 0.07 percent, to 1,984.13, and the Nasdaq Composite <.IXIC> dropped 48.70 points, or 1.07 percent, to 4,518.90.

The decline for the Nasdaq marked its biggest drop since July 31.

The largest percentage gainer on the New York Stock Exchange was DRDGOLD Limited , which gained 20 percent, while the largest percentage decliner was BANKRATE INC (>> Bankrate Inc), down 13.75 percent.

Among the most active stocks on the NYSE were Bank of America (>> Bank of America Corp), down 0.30 percent to $16.74; Nokia Corp ADR , up 2.27 percent to $8.56; and Petrobras , up 0.98 percent to $16.54.

On the Nasdaq, Avanir Pharma (>> Avanir Pharmaceuticals Inc), up 85.3 percent to $12.49; Yahoo Inc (>> Yahoo! Inc.), down 0.8 percent to $42.55; and Apple Inc (>> Apple Inc.), down 0.03 percent to $101.63, were among the most actively traded.

Declining issues outnumbered advancing ones on the NYSE by 2,021 to 1,031, for a 1.96-to-1 ratio to the downside; on the Nasdaq, 2,092 issues fell and 638 advanced for a 3.28-to-1 ratio favoring decliners.

The broad S&P 500 index was posted 14 new 52-week highs and 6 new lows; the Nasdaq Composite recorded 32 new highs and 97 new lows.

Volume was modest, with about 5.51 billion shares traded on U.S. exchanges, slightly below the 5.6 billion average so far this month, according to data from BATS Global Markets.

(Reporting by Chuck Mikolajczak; Editing by Nick Zieminski)

By Chuck Mikolajczak