Israeli Prime Minister Benjamin Netanyahu officially cancelled an agreement made with the United Nations that would have relocated thousands of African asylum seekers to Western countries.

The cancellation on Tuesday afternoon came a day after the agreement was first announced and hours after the prime minister said he would freeze the agreement in order to consult with members of his government coalition and with the residents of southern Tel Aviv, where many of the migrants live.

“In the past 24 hours, I have held many consultations with Interior Minister Aryeh Deri, with professionals and representatives of residents of southern Tel Aviv,” Netanyahu said Tuesday in his announcement. “I listened attentively to criticism of the agreement. As a result, and after evaluating a new balance of advantages and disadvantages, I decided to cancel the agreement.”

He also said: “Despite the growing legal and international difficulties, we will continue to act with determination to exhaust all the possibilities available to us to remove the infiltrators. At the same time, we will continue to seek additional solutions.”

Under the agreement announced earlier Monday with the U.N. High Commission for Refugees, Israel would have allowed thousands of African migrants to stay in the country on a temporary residency permit for up to five years. The rest, some 16,000 or so, would have been settled in countries such as Canada, Germany and Italy.

The deal put Netanyahu under fire from several conservative politicians, including some in his own Likud party. Naftali Bennett, head of the right-wing Jewish Home party, said it would “turn Israel into a paradise for infiltrators.”

It is not known what will happen now to the up to 40,000 African asylum seekers in Israel; an Israeli government plan to deport them this month was put on hold following the announcement of the U.N. deal.

Israel had been scheduled to begin deporting the migrants next week under a plan and budget approved by the country’s Cabinet in January. The Supreme Court had frozen the deportations in mid-March after a petition filed by opponents, and had been waiting for the government to respond to the petition.

According to the government plan, migrants who had chosen to leave by March 31 would receive a payment of $3,500 as well as free airfare and other incentives.

The Prime Minister’s Office said in a statement Monday that the original mass deportation plan was canceled “because of legal considerations and political difficulties on behalf of third-party countries.”