Patriotism is said to be the last refuge of a scoundrel. "Freedom of speech" may be the first.

A chorus of whining can be heard in Hollywood now about decisions by networks and stations to cut back on sensationalist programming. Advertisers are being scared away from such shows by the protests of pressure groups. "Tabloid TV Meets with Ad-Versity," said a headline in Daily Variety, the industry trade paper.And so producers are crying censorship, worrying loudly that America may return to the allegedly dark ages of the 1950s, when the Almighty Sponsor sent forth decrees on what was and wasn't permissible on television. Their wailing will not elicit much sympathy. The fact is, the proverbial pendulum has swung too far in the realm of smut and sleaze.

An outbreak of skittishness may be just what television needs right now.

ABC seemed to be leading the retreat when, earlier this month, it abruptly pulled from its schedule two tabloid specials, "Crimes of Passion II" and "Scandals II." Then NBC announced it was delaying from May until next fall a TV movie called "The Prize Pulitzer," about wild doings among the rich in Palm Beach, Fla.

If this be censorship, it would appear the republic will survive it.

TV critics have been complaining about trash TV for months and months. But it took a few letters from Terry Rakolta, an affluent Detroit suburbanite, to throw a scare into the TV business. The reason for the scare is that Mrs. Rakolta directed her letters to advertisers - mainly those whose commercials she'd seen on "Married...with Children," a dirty-minded sitcom on the dirty-minded Fox network.

Reporters beat eager paths to Rakolta's door and she became a media darling. All the publicity served to re-awaken Donald Wildmon, an evangelist in Tupelo, Miss., who has campaigned perennially on behalf of what he considers "decency" in TV. Wildmon's new group is called Christian Leaders for Responsible Television.

One problem with Wildmon's approach has always been that his definition of decency is keyed to the standards and mores of the year 1954, and even that's giving him the benefit of a doubt.

It's ironic that much of the protesting now about TV's permissiveness comes from political conservatives, since we may not have had this mess at all if it were not for Ronald Reagan and his fanatical policy of deregulation. Reagan took all the teeth out of the Federal Communications Commission and then took away the gums besides. Under Reagan appointee Mark Fowler as chairman, the FCC threw out many of the rules that had helped keep broadcasters in line.

It was Fowler's tireless litany that "the marketplace" should be the only regulator. So when critics complained about the rising smut quotient in prime-time programming and the proliferation of racy, pandering tabloid specials like Geraldo Rivera's, producers and broadcasters said that the marketplace was out-shouting them. They were only giving people what they wanted.

By this morality, anything that can attract an audience is automatically appropriate. The deregulators forget that traffic accidents on the highway attract an audience. Nero was able to attract an audience to see Christians fed to lions.

The fury over tabloid TV will subside as the producers of such shows slink off into the night. You can become hypnotized by watching the pendulum swing back and forth, forth and back. There are much more essential and abiding issues at the heart of the controversy: Who owns the airwaves, and what is the social responsibility of those who use them to make money?

By congressional mandate, the airwaves belong to the public, and broadcasters who use them have a duty to do more than generate profits for themselves. Television is an enormously influential component of our social environment; we have an electronic ecology as well as a natural one. There ought to be penalties for those who pollute it and, better still, guidelines to help prevent the pollution from occurring in the first place.

Maybe it's naive to imagine after all these years that television can be an instrument to enlighten and ennoble us as a people, but even a casual observer can see it's gone way too far in the other direction. The instrument is out of whack, and needs a good swift kick.