The stock market is up, unemployment is down but things aren't rosy for all Americans.

A new analysis of government data by the Consumer Federation of America found that the number of Americans in default on their student loans jumped by nearly a fifth last year.

As of the end of 2016, there were 4.2 million Federal Direct Loan borrowers in default, meaning they've not made a payment in more than 270 days. That's up 17 percent from 3.6 million at the end of 2015.

"Despite all improvements in the economy, student loan borrowers are still struggling," said Rohit Chopra, senior fellow at the Consumer Federation of America and formerly the Consumer Financial Protection Bureau's Student Loan Ombudsman.

As of the end of 2016, 42.4 million Americans owed $1.3 trillion in federal student loans, according to the U.S. Department of Education data. This doesn't include borrowing through private student loans, credit cards, and home equity loans to finance the growing costs of college.

The Federal Reserve System puts the measure slightly higher at $1.4 trillion, as it includes private loans as well.

Defaulting on a federal student loan can be a financial disaster for the borrower. Unlike other types of debts, most federal student loans cannot be discharged in bankruptcy. Those who go into default face serious repercussions including wage garnishment, damaged credit scores and potentially added costs in fees, interest and legal fees.

Student debt has risen along with the cost of education, which makes repayment difficult. The average amount owed per borrower rose to $30,650 in 2016, after rising steadily for years. In 2013, borrowers on average owed $26,300.

The good news is that the number of people who are defaulting for the first time is down. But the number of people defaulting for the second time or more is up. And that worries the CFA, an association of more than 250 nonprofit consumer groups

"We thought in an improving labor market, default rates would improve but we simply are not seeing that," Chopra said.

Other reports have also painted a bleak picture for student loan borrowers in the U.S.

The New York Federal Reserve, which looks at slightly different data, reported last month that total household debt in America in 2016 began nearing its previous peak from 2008, driven largely by student debt and auto debt.

It finds that student loan defaults jumped sharply in 2012 and the default rate has held fairly steady since. But the NY Fed also warns that the true number of people unable to pay is much higher because about half of loans are in forebearance, deferment or a grace period so are not at risk of being in default.

The CFA also said Tuesday that the report validates some of the recent claims made by federal regulators that the nation's largest loan servicer put borrowers at greater risk for default by failing to help them find the best repayment plans for their needs.

The USDE contracts with four major servicers to collect payments owed the federal government. Navient, formerly known as Sallie Mae, is the largest of them. The report found Navient had the lowest percentage of borrowers in two income-based repayment plans, which are designed to help struggling borrowers.

In January, the Consumer Financial Protection Bureau sued Navient, accusing it of making it harder for borrowers to repay loans by giving them bad information, processing payments incorrectly and failing to act on complaints. Several states followed with lawsuits of their own. Navient disputed the charges.

The USDE did not immediately respond to a request for comment.