WASHINGTON (CNN) -

President Barack Obama called House Speaker John Boehner's plans to file a lawsuit over the President's use of executive action "a stunt," in an interview with ABC News Chief Anchor George Stephanopoulos.

"You notice that he didn't specifically say what exactly he was objecting to. I'm not going to apologize for trying to do something while they're doing nothing," the president said.

"You know, the suit is a stunt. But what I've told Speaker Boehner directly is, 'If you're really concerned about me taking too many executive actions, why don't you try getting something done through Congress?'"

The interview aired on ABC's "Good Morning America" Friday morning. It was conducted Thursday during the president's trip to Minneapolis, where Obama held a town hall meeting.

The president touted the accomplishments of his first five years in office but lamented the tone in Washington.

"What I do worry about is that right now we've got a Republican Party that seems to only care about saying no to me," Obama said.

He repeated his frustrations about the immigration reform bill passed a year ago by the Senate that's been stalled in the House, and having to use executive action.

"The majority of American people want to see immigration reform done. We had a bipartisan bill through the Senate, and you're going to squawk if I try to fix some parts of it administratively that are within my authority while you are not doing anything?'" the President added.

Responding to Obama's comments, Boehner spokesman Michael Steel said Friday in a statement that "the American people, their elected representatives, and the Supreme Court have all expressed serious concerns about the president's failure to follow the Constitution. Dismissing them with words like, 'smidgen' or 'stunt' only reinforces their frustration."

Republicans argue that the president is breaching his constitutional power by side-stepping the legislative process. Obama has used executive actions as a way to bypass a deeply divided Congress, avoiding inaction on issues the White House has made hallmarks of the President's second term agenda.

So far, the Republican-controlled House has passed two bills aimed at curbing executive orders by the President, neither of which have gone anywhere in the Democratic-controlled Senate.

At this point in his presidency, Obama has issued less executive orders than his most recent two-term predecessors in the White House, George W. Bush, Bill Clinton, and Ronald Reagan.

In speaking of immigration during the interview, the President was asked about the thousands of unaccompanied children from Central America coming across the southern border. He implored their parents to not send their kids.

"Our message absolutely is, 'Don't send your children unaccompanied on trains on through a bunch of smugglers.' That is our direct message to the families in Central America. 'Do not send your children to the borders. If they do make it, they'll get sent back. More importantly, they may not make it'," he said