WASHINGTON (AP) — President Barack Obama is proposing a program to allow homeowners with privately held mortgages to refinance at lower interest rates.

The program would cover both loans issued by government-controlled mortgage giants Fannie Mae and Freddie Mac and private mortgage lenders. Congress would have to approve, a difficult hurdle.

Obama announced the plan, his latest of several housing initiatives, in his State of the Union speech.

“There’s never been a better time to build, especially since the construction industry was one of the hardest-hit when the housing bubble burst,” Obama said. “Of course, construction workers weren’t the only ones hurt. So were millions of innocent Americans who’ve seen their home values decline. And while government can’t fix the problem on its own, responsible homeowners shouldn’t have to sit and wait for the housing market to hit bottom to get some relief.”

A punctured housing bubble was at the center of the recession, prompting widespread foreclosures and leaving millions of homeowners with houses valued at less than their mortgages.

Under the plan, any homeowner current on his or her mortgage could take advantage of historically low lending rates. Mortgage rates have been below 4 percent for months.

The program would be paid for by a fee on large banks.

Administration officials offered few details but estimated savings at $3,000 a year for average borrowers. It’s likely that millions of homeowners would be eligible, but they would have to seek out refinancing options under the program with their lender. Other government programs allow lenders to seek out potential applicants.

The new program would expand the Obama administration’s Home Affordable Refinance Program, which allows borrowers with Fannie and Freddie-backed loans to refinance at lower rates. Few people have signed up for that program. Many “underwater” borrowers — those who owe more than their homes are worth — couldn’t qualify under the program.

About 1 in 4 Americans with a mortgage — about 11 million — are underwater, according to CoreLogic, a real estate data firm. Roughly 1 million homeowners have refinanced through the refinancing program. Government officials had estimated it would help 4 million to 5 million homeowners.

About half of all U.S. mortgages — about 30 million home loans — are owned by non-government lenders.