Dissidents dogged by scores of KGB agents and police officers met for the third day Monday to draft the manifesto of the first political party to challenge the Communist Party, sources said. Five were detained.

Instead of meeting in Moscow apartments as on Saturday and Sunday, dissidents trying to set up the new Democratic Union in a crucial test of Mikhail Gorbachev's "glastnost" reforms decided to hold their third and final day of talks at a country cottage or dacha outside Moscow."As they gathered at the railroad station in Moscow this morning, scores of KGB and uniformed police were at hand," said a witness. "They appeared to allow them to take the train out of town."

But sources said five people were detained as the dissidents met at the dacha of Sergei Grigoryants, a human rights activist who spent three years in a labor camp and now edits the journal of the unofficial Glasnost Press Club.

Further details of the meeting were not immediately available.

Sunday, spokesman Alexander Khartov said 25 of 47 Democratic Union members taken to police stations that day were released within hours after showing proof of residence in Moscow. There was no immediate news of the remaining 22 who were expected to be held overnight and sent home Monday.

Khartov, who had a list of those taken in, said they were determined to continue with their final meeting Monday to draft a manifesto despite the harassment. Scores of police and plainclothes KGB agents remained mostly on the sidelines during the opening Saturday of the three-day meeting of the Democratic Union, but increased their pressure Sunday with the detentions. Police stopped short of shutting down the conference.

There has been no official comment on the formation of the Democratic Union, one of the most open and direct challenges to the Communist Party to surface under Gorbachev's policy of openness. It is also a key test of the Soviet leader's call for pluralism and more democracy.

"Gorbachev is afraid of real discussion. He is afraid of people and people's ideas," said Alexander Bogdonov, who traveled from Leningrad to attend the unofficial party congress.

More than 100 Democratic Union members from 14 cities split up into three groups Sunday to discuss a party manifesto they said would be made public Monday, a national holiday marking the 43th anniversary of the defeat of Nazi Germany in World War II.

Union representatives, who spoke openly to Western reporters about their aims, said the groups met to formulate position papers on the economy, law and order and democracy.

The union's self-proclaimed aim is to challenge the stranglehold of the Communist Party on Soviet society, and its founding document strongly criticizes Communist rule.

"We define the contents of our activity as political opposition to the present order," the document says. "The whole system of (Communist) Party autocracy has become the main source of the people's troubles during the last 70 years.

"No one has the right to decide for the people what direction to follow. Only the people themselves, according to their sovereign rights, can choose the way on the basis of agreement and free voting," it said.

With the summit between Gorbachev and President Reagan in Moscow only three weeks away, it appears the Kremlin is reluctant to cause a major incident that may reflect badly on Soviet human rights policy. Only last week, Reagan conceded there had been some improvement in the Soviet human rights record.

At the unofficial headquarters of the union, an eighth-floor apartment in Moscow's Mikhailkovo district, uniformed officers and leather-jacketed KGB men Sunday stood watch outside, questioning people as they left the building and in some cases hauling them away in police cars.