CKE Restaurants CEO Andy Puzder withdrew his name from consideration for labor secretary in President Donald Trump’s cabinet on Wednesday.

“After careful consideration and discussions with my family, I am withdrawing my nomination for Secretary of Labor. I am honored to have been considered by President Donald Trump to lead the Department of Labor and put America’s workers and businesses back on a path to sustainable prosperity,” Puzder said in a statement.

CBS News revealed earlier that a source said Puzder is “very tired of the abuse,” and wasn’t going to appear for Thursday’s confirmation hearing, which was set to take place before the Senate’s Health, Education, Labor, and Pensions committee.

The New York Times said the move came “amid growing doubts among Republican senators that he can be confirmed.”

CKE, the parent company of Carl’s Jr. and Hardee’s, named Puzder executive vice president and general counsel in 1997 after Puzder met Carl’s Jr. founder Carl Karcher and became his personal attorney to help him overcome financial difficulties. CKE now owns or franchises more than 3,250 restaurants in the U.S. and 26 other countries, generates $1.3 billion in annual revenue, and, with franchisees, employs more than 70,000 people stateside.

In recent days, Puzder has come under scrutiny, including from Republican Senators who have questioned his ability to perform the job adequately. Democrats have argued against the nomination based on his history with workers.

“The simple truth is that given his relationship to employees at the companies he runs, he was not fit to lead a department responsible for defending workers’ rights,”  Senator Bernie Sanders told the New York Times.

Puzder, both an adviser and contributor to Trump’s campaign, has been outspoken about cutting back on government regulations like mandating large minimum wage increases and the Affordable Care Act, which he says created a “government-mandated restaurant recession” through increased labor costs for businesses as well as premiums reducing consumers’ discretionary income.

According to CBS News, at least five Senate Republicans did not make public their voting plans for Puzder: Sens. Susan Collins of Maine, Johnny Isakson of Georgia, Lisa Murkowski of Alaska, Mike Enzi of Wyoming and Tim Scott of South Carolina—all members of the Senate HELP committee.

In early February, The Associated Press reported that Puzder was taking steps to eliminate any conflicts of interest that might threaten his nomination. The article said that Puzder was working to rid himself of assets from the fast food company in hopes of being confirmed.

According to the AP, Puzder’s spokesman George Thompson said the process, made complicated by the fact CKE is privately held, was underway, and that Puzder would step down as CEO once confirmed.