House and Senate conferees agreed Friday to ban ocean dumping of sewage sludge after 1991, concluding a 10-year legislative effort designed to stop sewage authorities in New York and New Jersey from disposing of their waste in the Atlantic.

The new agreement to ban the dumping beginning Jan. 1, 1992, still maintains stiff fines aimed at making sludge dumping exorbitantly expensive. The fines would be pooled to pay for alternatives to dumping.The nine agencies in New York and New Jersey are the nation's only remaining ocean dumpers.

"Clearly the end of ocean dumping is in sight," said Rep. William Hughes, D-N.J., the prime House sponsor. He called the agreement "tough, but fair."

The compromise reconciles differing sludge-dumping bills passed by the Senate in August and by the House this week.

Senate and House votes on the compromise are expected Wednesday. President Reagan is expected to sign the bill.

"This means that the day when we stop using the ocean as a sewer is getting closer," said Sen. Frank Lautenberg, D-N.J., who authored the Senate bill.

The proposed ban was initially opposed by New York lawmakers and New York Mayor Edward I. Koch, who complained that the deadline would be impossible to meet.

But the opposition weakened as a bipartisan coalition of East Coast lawmakers rallied in favor of a ban.

Though the House bill called for a Dec. 31, 1992, deadline, conferees chose the earlier ban as called for by the Senate.

But, in a bow to New York lawmakers, the conferees agreed to lower fees dumpers must pay until the deadline, and penalties on those who fail to stop by 1992.

Some 8 million tons of sludge a year are dumped by the nine agencies at a site about 106 miles east of New Jersey. Congress closed a site just 12 miles offshore in 1987.

Experts differ on the environmental threat posed by the dumping of sludge.

The new bill requires dumpers to pay fees of $100 dollars per ton of sludge beginning on Jan.1 of next year.

Dumpers, meanwhile, would be required to negotiate with federal authorities individual timetables for ending their dumping.

Those agencies who cannot strike a deal will pay fees that rise from $100 to $200 a ton by 1992. The fee scale is half that suggested by the House.

Any authority still dumping sludge in the ocean by 1993 would have to pay penalties in addition to the fees.

Those fines begin at $600 per ton - $200 less than in the House plan - and rise by 11 percent a year for continuing violators. New York City alone would be liable for tens of millions of dollars in fines and fees if still dumping by the year 2000.

Congress orginally banned ocean sludge dumping as of 1981, but that deadline was suspended in a successful lawsuit by New York. The original deadline was set in a 1978 law.