Federal tax reform has created a fiscal nightmare in a handful of major states that once reaped the benefits of hyperactive growth fueled by bustling stock and real estate markets.

Boom has turned to hard times in Massachusetts, California, New York, New Jersey and Connecticut after investors rushed to liquidate before the 1986 law change boosted capital gains taxes.Flattening in the real estate market in states such as Massachusetts has steepened the downward spin.

The phenomenom - reflected in a roller coaster-like ride of state tax receipts - also may have helped President George Bush deflate the image of the "Massachusetts Miracle" that Gov. Michael Dukakis had hoped to ride into the White House.

In Massachusetts alone, capital gains collections soared from $194 million to $602 million between 1984 and 1986 before plummeting to $276 million in 1987.

The plunge forced Dukakis close to a $636 million fiscal 1989 shortfall by cuts in spending, boosting fees, shuffling reserve accounts, laying off employees and dipping into a rainy day reserve account to balance an $11.6 billion budget.

When that proved insufficient, Dukakis, who pledged to raise federal taxes only as a "last resort," found himself asking the Legislature to approve a $604 million tax package that, ironically, would double the state's capital gains tax.

The scene is similar in New York, capital gains receipts went from $32 billion in 1986 to $14.1 billion one year later.

Gov. Mario Cuomo, facing a potential $2.6 billion deficit in a $44 billion budget, ordered layoffs and is asking for almost $1 billion in new taxes and fees, including a $7-a-day room-and-board charge on prison inmates in paid work-release programs.

Connecticut, which posted its first deficit in five years in the 1987-88 fiscal year, faces a projected $134 million gap in a $6 billion budget this year and a $882 million gap in the fiscal year that begins July 1.

Gov. William O'Neill will detail his tax and spending plans in a Feb. 8 budget message.

Nor is the problem limited to what Republicans have long-labeled "tax and spend" Democrats.

New Jersey's Thomas Kean, the keynote speaker at last year's Republican National Convention, faces a $290 million revenue shortfall, a figure that nearly doubled in just the last two months.

While Kean has taken pains to distinguish his problems from those of Dukakis, he nonetheless has vetoed new spending, dipped into reserves and imposed a hiring freeze.

And Bush's favorite "Duke," California Gov. George Deukmejian, is embroiled in partisan bickering over whether or not his state is running in the red.

In the meantime, Deukmejian is threatening to eliminate welfare cost-of-living increases, ax state agencies and hike fees at public universities to rebuild a $1.1 billion state budget reserve.

California voters have already approved a cigarette tax increase and Deukmejian said he's willing to let a gasoline tax proposal also go the ballot.

Analysts and budget watchdogs agree state leaders would have been hard pressed to see the violent swing in capital gains revenues.

"It was impossible to predict how taxpayers would react to the change in the capital gains tax," says Ron Snell, an analyst for the National Conference of State Legislatures in Denver. "We simply did not have the information."

"A lot of drama comes out of New York, California and Massachusetts just because of their size," said a congressional tax aide. "When they report a miss, it's bigger dollars."

The aide suggested the affected states may be, in part, a victim of their own success and the October 1987 stock market crash. "It was two shots in the tail that I think hurt us," said Richard Manley, director of the non-partisan, business-financed Massachusetts Taxpayers Foundation. "What was an equal problem to federal tax reform was Oct. 19. It was a double whammy."

He believes a third factor played into Massachusetts' woes, a sharp dropoff in what had once been a white-hot real estate market.

"There is one pile of capital gains received in this state from the same of real estate," he said.

But Manley believes one area Dukakis must absorb blame is for failing to rein in state spending after the trend began to appear.

"I personally would not have budgeted as high as they did in (fiscal) '89," he says, noting the Massachusetts budget was based on 10.9 percent revenue growth just one year after collections rose just 2.1 percent.

"When you're that far down, I think you budget as conservatively as possible," he said

Snell believes the eventual solution requires political courage.

"Even when a state suffers a serious economic shock, if it has the political will to do so, it can recover," said Snell. "But it takes time."

But congressional tax aides caution against allowing the pendulum to swing too far in the other direction.

"There is a potential for people to overreact and look at what happened in 1987 as the way things will be permanently. But its too hard to judge beacuse the peak and trough are too broad."