Federal regulators announced completion of an agreement Wednesday to rescue the country's largest insolvent savings and loan institution in a bailout that will cost the government $1.7 billion.

The Federal Home Loan Bank Board said it had completed an agreement for the sale of American Savings and Loan Association of Stockton, Calif., to an investment group headed by Texas billionaire Robert M. Bass.According to the details of the sale, the bank board, which regulates the country's 3,100 savings and loans, will inject as much as $1.7 billion in federal aid.

As costly as the American Savings bailout will turn out to be, it was dwarfed by a second announcement concerning the rescue of five large Texas institutions.

The bank board said it will provide $5.1 billion in government assistance over the next 10 years to assist in the takeover of five of the most financially troubled Texas institutions.

The Texas S&LS are being taken over by the investment group of Mac-Andrews & Forbes Holding Inc., which is headed by Ronald O. Perelman, the takeover specialist who is the head of Revlon Inc., the cosmetics company.

As costly as the rescue packages will be, bank board chairman M. Danny Wall said that the cost to the government of simply closing the insolvent institutions and paying off depositors would have been far higher.

In the case of the California institution, Wall estimated the cost of closing the institution and paying off depositors would have cost the insurance fund, the Federal Savings and Loan Insurance Corp., $3.3 billion, compared to a cost of $1.7 billion for the sale to the Bass group.

With Wednesday's actions, the government has resolved the case of 181 failed S&Ls this year, up from 48 last year.

The board is rushing to complete transactions because tax breaks for institutions getting government assistance will be reduced after Jan. 1.

The purchase agreement for American Savings, the country's second largest S&L and the biggest one that was classified insolvent, will require the Bass Group to invest $350 million in cash and add another $150 million over the next three years.