"The Fed buttered the market up. It was a done deal," he said. "It was a huge policy mistake."
The Fed also cut its economic growth forecasts for this year and 2014. Bernanke warned that the upcoming debt ceiling and budget fights between the White House and Congress "may involve additional risks to financial markets and to the broader economy."
On Thursday, the Standard & Poor's 500 index fell three points, or 0.2 percent, to 1,722.34. The Dow Jones industrial average slipped 40 points, or 0.3 percent, to 15,636.55.
The Nasdaq composite index rose six points, or 0.2 percent, to 3,789.38, helped by Apple's stock price.
The price of gold surged $61.70, or 4.7 percent, to $1,369.30 an ounce.
The yield on the 10-year Treasury note rose to 2.75 percent from 2.69 percent late Wednesday.
Despite Thursday's minor pull back, September has been great for the market. Stocks are on pace to have their best month in nearly two years.
The Dow set an all-time high of 15,767.93 on Wednesday following the Fed's decision. The S&P also closed at a record high — 1,725.52
However, Wilbanks and other investors believe the market cannot go much higher, particularly with an uncertain earnings season starting in a few weeks and the looming political fights in Washington.
"We're being very careful about U.S. equities," he said.