The New York Immigration Coalition and the New York Civil Liberties Union in a joint statement on Thursday raised concerns over the Department of Justice memorandum that outlined new policies for immigration enforcement in the U.S.

While the memo has largely dealt with the enforcement of undocumented immigration, the organizations are worried lawful citizens and immigrants could be impacted as well.

For example, the DOJ could begin a crack down on those who harbor or provide a ride to undocumented individuals or protect someone who is being arrested by ICE officers. At the same time, the groups worry a person could under legal scrutiny for marrying someone who then receives a green card.

“This is an unprecedented attack by the federal government on its own citizens, many of whom oppose the anti-American policies of the Trump regime, and want to support those victimized by it,” said Steve Choi, Executive Director of the New York Immigration Coalition. “Americans are a fiercely independent people with a patriotic respect for decency and fairness: Jeff Sessions does not get to decide who we let spend the night in our own homes or who we give a ride to. We should not wind up fighting criminal charges because the federal government has lost sight of the constitutionally appropriate limits of its power.”

For immigrants who live legally in the U.S., the groups fear they could be deported should they run afoul of the DOJ directive.

“The Attorney General’s remarks in Nogales on Tuesday betrayed a hateful disdain for immigrants and immigrant communities,” said Donna Lieberman, Executive Director of the New York Civil Liberties Union. “His memo turns that hate into policy, by ordering federal prosecutors to pour their limited resources into further criminalizing the act of migration. Immigrant New Yorkers should not be imprisoned simply for seeking safety and opportunity in America. New York’s prosecutors should focus their energies on keeping our cities and towns safe, not jailing our friends and loved ones.”