On Tuesday afternoon at the Grand Central Terminal in Manhattan, homeless New Yorkers and activists announced a lawsuit against Federal Housing and Finance Administration Acting Director Ed DeMarco and the Obama administration for illegally withholding funds from the National Housing Trust Fund, a fund created explicitly to support low-income, affordable rental housing.

Maria Walles, the housing campaign leader for Picture the Homeless, said, “The money that should have been invested in housing wasn’t. We could use that money for rent and mortgage assistance so families and single adults could stay out of the shelter system, which is inefficient and disempowering. I was in the Advantage program living in my own apartment, but because that was recently cut, I was thrown back into the shelter system. If the Trust Fund had money, I could still be in my apartment.”

This lawsuit is a nationwide one, as it involves the groups Right to the City and the National Low Income Housing Coalition. Danielle Stelluto, one of the three plaintiffs in the nationwide suit, also demanded that whoever succeeds DeMarco continue to contribute to the National Housing Trust Fund.

“You think it’s hard raising a toddler?” said Danielle, a home care worker who currently lives in a shelter. “Try doing it without a home.”

Kendall Jackman of Picture the Homeless agrees.

“There are over 50,000 people—20,000of them children—in the New York City shelter system,” said Jackman. “The government needs to honor its promise and put that money into creating housing for the people.”

According to public records, DeMarco has withheld at least $382 million of funding for affordable housing, as required by the 2008 Housing and Economic Recovery Act. The Housing and Economic Recovery Act requires Fannie Mae and Freddie Mac to contribute a portion of their revenue to the National Housing Trust Fund. Although the banks reported $28 billion in record-setting profits last year, the Federal Housing Finance Agency, which oversees Fannie Mae and Freddie Mac, failed to contribute to the National Housing Trust Fund. In the plaintiffs’ eyes, the government’s shirked their responsibility to those in need of affordable housing.

In 2008, the Housing and Economic Recovery Act, signed by President George W. Bush, established the National Housing Trust Fund to build, preserve, rehabilitate and operate rental housing that’s affordable to the lowest income Americans. Under the law, Fannie Mae and Freddie Mac are required to transfer a portion of the value of their new business to the National Housing Trust Fund. That policy was temporarily suspended for the director of the Federal Housing Finance Agency when Fannie Mae and Freddie Mac were taken into conservatorship.

According to filings by the Securities and Exchange Commission, in 2012, new business activity for Fannie Mae and Freddie Mac in 2012 was approximately $1.4 trillion. The $382 million, if paid into the fund, would represent the largest new investment in low-income affordable housing in more than 30 years.

According to public records, over two-thirds (68 percent) of our country’s poorest households spend more than half of their income on housing costs. For every 10 low-income households, there are three affordable and available homes.

Shelia Crowley, president and CEO of the National Low Income Housing Coalition, said that the housing shortage in places like New York City means that it’s high time for the government to lend a hand.

“The time has long past for Fannie Mae and Freddie Mac to be supporting the National Housing Trust Fund as Congress meant for them to do,” said Crowley in a statement. “The delay caused by the Federal Housing Finance Agency is unconscionable given the growing shortage of housing that is affordable for the lowest income Americans. The National Low Income Housing Coalition is taking this action today in hopes that these critical resources can be put to use as soon as possible.”