Striding through the still and shaded courtyard, past the rows of ancient amphorae, the broken marble statuary and walls embedded with carved Roman tablets, Joseph Connors was visibly awed.

"Isn't this incredible?" asked the art historian from Columbia University, waving a delicate hand toward all the fragments of Imperial Rome that graced the surrounding Palazzo. "All these artifacts were dug up right here when the Palazzo was built at the turn of the century.""This is where my work is, this is where my art is," said the scholar of architecture, pausing at the courtyard's gurgling fountain to gaze out over the sun-drenched domes and tile rooftops of Rome. "I feel like I'm coming home."

He was. Home for Connors, his wife Francoise and their two children will in fact over the next three years be this grand Palazzo, an accompanying 17th-century villa and the complex of manicured gardens, artists' studios and apartments. Together, they form a unique, if little known and often troubled, institution - the august American Academy of Rome.

Tall, thin, distinguished and prematurely gray at 43, Connors seemed at once delighted and nervous as he took over last month as director of the academy, created at the turn of the century by the dream of a visionary American architect, Charles Follen McKim, and the cash of such financial barons as J.P. Morgan, John D. Rockefeller and William Vanderbilt.

"This is an extraordinary place, a marvelous place," said Connors, author of studies on Francesco Borromini and Frank Lloyd Wright, who has often worked and studied here between his teaching chores at Columbia. "Taking over here is both an honor and a challenge."

The honor is obvious. As director of the American Academy, one of more than a dozen national academies of art and culture in Rome, he is an ambassador of American culture in one of the most cultured cities of the world, presiding over his own "embassy" of American classicists, medievalists, art historians, architects, painters, sculptors, poets and composers.

The challenge is more problematical. In an era of shrinking dollars and reduced national support for the arts, Connors faces the daunting problem of keeping his illustrious, often temperamental, charges comfortable, contented and productive while at the same time striving to make the academy live up to its ambitious, rarely realized potential.

The academy is supposed to be a place for select U.S. scholars and artists to gather in Rome to commune, exchange ideas, think, create and generally be exposed to the great cultural repository that is Rome and its history-steeped surrounding countryside.

"What makes this special is that unlike a U.S. university, where all the disciplines are isolated and separate, here we have them all together, living, eating and working side by side," Connors said. "It isn't always easy, and doesn't always work, but it is a great opportunity for horizons to be widened."

Each year, about 30 Americans are picked by special juries of their peers for the honor of attending the academy as Prix de Rome winners. Their ranks regularly are augmented by shorter visits from some of the pre-eminent masters of their fields, not all of them American. In recent years the academy has been graced by the likes of painter Frank Stella, poet Galway Kinnell, novelist Nadine Gordimer and composer Earle Brown, among others.

Unfortunately for many of the fellows, scholars and artists of modest incomes, the realities of life at the academy often fall short of the ideal, usually because of the low stipends they must live on in a city that in the era of the weak dollar has become as expensive as Paris or Manhattan.

Money, or the lack of it, has in fact marred the lofty experience of being honored by the Prix de Rome for many of the fellows, said one group of scholars and artists packing this summer before the arrival of a new crop in September.

"Being in Rome is great," said one outgoing architecture fellow as his yearlong stay at the academy came to an end, "but only if you can afford to experience it fully, and that is often rarely the case."

Life at the academy, for many fellows, reminds them of living in a college dorm: Rooms in the massive Palazzo are tiny, sparsely furnished, with only rudimentary amenities. More galling, fellows complain, in a city noted for its marvelous food and cooking, the academy cafeteria, where all fellows are fed at long wooden tables, is considered the purveyor of some of the least inspiring food in Italy.

The problem, members said, would not be so bad had they suitable options to spend more time away from the academy _ to eat out or travel more than the occasional arranged trips to various cultural monuments. But most don't. Stipends range from $6,200 to $11,000, with a room and a daily meal at the academy thrown in.

The ideal of putting so many different scholars and artists together for a year also often falls short of the ideal because of the clash of egos. Professional jealousies and rivalries often divide fellows, and the personality of a visiting star and the special treatment accorded him or her often causes disgruntlement and outright rebellion.

Connors said he is well aware of the problems, having had two stints at the academy himself in the past. He says the academy board is aware of the financial problems and hopes stipends will be raised in the coming year or so. He said the board is seeking to raise $20 million by 1994 to double the institution's endowment.

Beyond that, he said, he plans to do the best he can with the resources available, improving housing conditions and expanding the academy's programs of exhibitions, especially in his field, architectural history. He also hopes to find new ways for fellows to have fruitful exchanges with their Roman colleagues and environment.

"This is Rome, after all," he said. "Nobody can come here and go away unchanged, and that, in the end, is one of the goals of the academy."