This article was published in partnership with the Center for Public Integrity.

As a third-generation Black farmer in Arkansas, Dewayne Goldmon understands the frustrations of Black farmers trying to get more aid for past injustices from the U.S. Department of Agriculture.

But as the senior advisor for racial equity to the USDA secretary, he also understands why the agency has struggled to address discrimination — and still does today.

So much comes down to data.

Legislators and farm advocates argue that if they had better information on who participates in agricultural programs from the USDA, including those who have applied for or received loans, they could better monitor how effective agency programs have been at reaching Black farmers and other farmers of color. They could write better policies that address historical problems. And maybe they could better substantiate claims of discrimination. The information held in USDA’s own records could result in more funding to programs that benefit farmers of color.

In a renewed push for more data transparency, legislators on Capitol Hill are working to get language into future measures that would compel the department to report more detailed annual personal data across programs in compliance with a 2008 law, as opposed to a database they say is dated, inconsistent and incomplete.

Legislators and advocates argued the law requires the USDA to compile and publish data every year broken down by race, ethnicity and gender on each of its agricultural programs. That data, they say, should include raw numbers about the application and participation rates for the United States and every state and county.

But the USDA doesn’t do so. Some programs reflected in the public database have both applicant and participant data; others have only one or the other. The number of years covered by the database is inconsistent between programs.

The agency says it’s following federal privacy law by only sharing information that was voluntarily provided. Privacy law can also challenge federal agencies when asked to report granular data in sparsely populated, rural areas where it may be possible to identify an individual in a particular county. Still, there are other federal agencies that provide more data on race, ethnicity and gender, including the Small Business Administration through its major loan programs.

It’s a tiny political issue in this year’s $1.5 trillion farm bill, but advocates for more data say that what they’re seeking is critical to better holding the department accountable and would not disclose any individual’s personal information, only aggregated details. They say the “transparency and accountability” law is intended to shed light on how much progress farmers of color and women are making in accessing USDA programs.

“Maybe now, there’s a chance to try to put something with teeth into this farm bill that, moving forward, would at least get some of this data so many of us feel is important to get from USDA,” said Adam Zipkin, an agriculture policy staffer in the office of Sen. Cory Booker (D-NJ).

Beyond data reporting, privacy law can cause problems for how federal agencies collect race and ethnicity information. It’s an ongoing area of concern for the Biden Administration, which is updating the standard for how federal agencies collect this data. The Administration also formed an Equity Data Working Group to recommend ways federal agencies can address data collection challenges and improve the data they have available for measuring equity.

“We strongly believe that program participation data is a vital tool for informing policy decisions and ensuring accountability, so we’re putting in a significant effort to modernize USDA’s approach to data and align our work with broader federal initiatives,” a USDA spokesperson said.

Trying to measure progress

The push to require USDA to provide the annual reports dates back more than two decades. The 2008 law, in fact, was predated by one in 2002 with similar mandates. At the time, Congress authorized the position of assistant secretary for civil rights at USDA. Its responsibilities would include reporting on minority participation in farm programs.

Back then, the department was defending itself against several lawsuits that alleged discrimination in loan and benefit programs. The first suit was brought in 1997 by African American farmers, followed over the next several years by Native American, Latino and female farmers who made similar claims. Only the African American and Native American farmer groups were granted class certification, but all the lawsuits ended with settlements.

Pigford v. Glickman, the Black farmers’ case, alleged discrimination in loans and other USDA assistance between 1983 and 1997. In 1999, the agency agreed to the country’s largest civil rights settlement at the time. But the agreement did not require the USDA to make any reforms, only payouts.

Read more from Black Joy by Reckon: White House protest demands justice for Black farmers, but can Congress deliver?

Understanding whether the agency was rooting out patterns of discrimination would require data. The 2002 law directed the agency to report it.

“The purpose of this section is to ensure compilation and public disclosure of data to assess and hold the Department of Agriculture accountable for the nondiscriminatory participation of socially disadvantaged farmers and ranchers in programs of the Department,” the law reads.

The USDA issued annual reports on socially disadvantaged farmers — people belonging to groups “subjected to racial, ethnic or gender prejudice” — in 2003, 2004 and 2005. But the department warned that only the Farm Service Agency, which administers its farm loan programs, had mostly reliable data. Even so, for some FSA programs, applicants who did not identify their race were automatically designated as “white male” until 2004, according to several reports from the Government Accountability Office (GAO) and congressional testimony. The GAO is the investigative arm of Congress.

In other arms of the USDA, staff collected demographic data by visual observation of applicants. That too could be unreliable, especially for ethnicity, according to the GAO. In addition, the GAO concluded in 2008 that USDA’s reports are not very useful because they lacked basic reference data needed for understanding or examining trends.

In 2008, Congress passed a provision titled “Transparency and Accountability for Socially Disadvantaged Farmers or Ranchers” to expand what USDA was mandated to report. Now the agency had to include an application rate in addition to a participation rate. And Congress required “raw numbers of applicants and participants by race, ethnicity and gender, subject to appropriate privacy protections.”

In the same 2008 bill, Congress allowed late filers to the 1999 Pigford settlement to bring new lawsuits, which subsequently led to a second settlement of $1.25 billion.

Three years later, the USDA made two data-related improvements. It won approval from the Office of Management and Budget for standardized data collection of demographic information directly from program participants, according to the GAO. And a department regulation prohibited employees from assigning race, ethnicity and gender to farmers through visual observation, according to an academic paper by researchers at Stanford University, Stanford Law School and the Massachusetts Institute of Technology.

But the practice continued as late as 2019 because the agency’s data management system required employees to enter a demographic field, according to the researchers. The system has since been updated, according to USDA’s Office of the Inspector General.

A renewed push

This year, members of Congress yet again pressed the USDA to fully comply with the 2008 law.

“As of present, the policy has not been implemented,” the Congressional Black Caucus said in a letter to Agriculture Secretary Tom Vilsack dated May 30. “It is of utmost importance that this crucial information is provided across all USDA programs.”

Though the agency released at least a few annual reports after the 2002 law, it’s never done so in response to the 2008 law, both legislators and advocates say. In Vilsack’s June 28 response to the caucus, he said that’s because the 2008 law does not supersede the 1974 Privacy Act.

“USDA customers are not required to share private demographic information when applying for USDA programs,” Vilsack said. “Therefore, USDA agencies can only transmit information that was voluntarily provided by producers, farmers and ranchers.”

The Privacy Act of 1974 sets requirements for how personally identifiable information may be collected or disclosed by federal agencies. Agencies may only retain records of individuals that are deemed “relevant and necessary” to accomplish their purpose. Agencies cannot disclose the information “to any person, or to another agency,” unless they have some form of written consent from the individual the information involves.

Emily Spiegel, a senior research fellow at the Center for Agriculture and Food Systems at Vermont Law and Graduate School, said the 2008 law requires USDA to collect race, ethnicity and gender data where it’s offered and compile that data into a report, “which on its face is not inconsistent with the Privacy Act.”

“USDA is obligated to be following both of those laws, unless they are irreconcilable, but I don’t think that’s been shown to be the case,” Spiegel said.

Vilsack told the Congressional Black Caucus that USDA complies with the 2008 law by maintaining a database with race, ethnicity and gender statistics from certain programs that is updated as program participation data is provided. The database covers programs administered by four agencies, the Farm Service Agency, Natural Resources Conservation Service, Risk Management Agency and Rural Development, which covers most agricultural programs. But it’s missing program data from two other agencies, and what is reported isn’t comprehensive or complete.

The database isn’t “user-friendly,” Vilsack told the Congressional Black Caucus, but his staff is reviewing how to improve it. Agencies under the USDA umbrella also maintain and report participation data on their websites.

“Whenever possible, USDA will continue to report data on underserved producer participation in its programs to ensure transparency and accountability,” Vilsack told the caucus.

U.S. Sen. Tina Smith and Rep. Dean Phillips, both Minnesota Democrats, introduced a bill in July that would strengthen the 2008 law, require USDA to publish five and 10-year trends in the data and include metrics and timelines for making improvements.

Sen. Smith’s bill is based on recommendations by the USDA Equity Commission, which Congress created in the American Rescue Plan Act of 2021 to advise the secretary of agriculture on racial equity issues and closing barriers to accessing the department’s programs and resources.

Colorblind remedies

The Congressional Black Caucus said it was especially interested in receiving annual reporting on debt cancellation and discrimination-remedy programs outlined in the Inflation Reduction Act of 2022. Those are both colorblind, but their roots are in race-based programming in the American Rescue Plan Act of 2021.

“The American Rescue Plan provides historic debt relief to Black, Indigenous, Hispanic, and other farmers of color who for generations have struggled to fully succeed due to systemic discrimination and a cycle of debt,” Vilsack said in a press release following the bill’s passage in March 2021.

But legal challenges brought by white farmers and former Trump administration officials alleging reverse discrimination against whites halted such debt cancellation.

Read more from Black Joy by Reckon: This land is your land – until it isn’t: why Black farmers are demanding debt relief

U.S. District Judge Marcia Morales Howard temporarily blocked loan forgiveness payments under the American Rescue Plan, which had earmarked $4 billion in debt relief for socially disadvantaged farmers, not including white women.

She said the provision “applies strictly on racial grounds irrespective of any other factor. Every person who identifies him or herself as falling within a socially disadvantaged group who has a qualifying farm loan with an outstanding balance as of January 1, 2021, receives up to 120% debt relief — and no one else receives any debt relief.”

“It is undeniable — and notably uncontested by the parties — that USDA had a dark history of past discrimination against minority farmers,” Howard wrote. But, she said, “It appears that in enacting [the payments plan] Congress relies, albeit without any ill intention, on present discrimination to remedy past discrimination.”

The program was later rescinded. A subsequent $3.1 billion, colorblind program in the Inflation Reduction Act relieves borrowers who are behind on loan payments or face other financial risks.

A new $2.2 billion program will provide financial assistance to farmers who are found to have experienced discrimination in the department’s farm lending programs prior to Jan. 1, 2021. That program was created under a title that targeted socially disadvantaged farmers in the American Rescue Plan Act but changed to “underserved farmers” in the Inflation Reduction Act, a broader category that can include not only farmers of color and women, but also military-veteran and beginning farmers.

Despite this history, the USDA has declined to report the race of farmers who participate in the program for distressed borrowers. According to officials, the department has found that when it asks people to provide information like their race, they are more likely not to respond, which results in data that is unreliable.

For example, when USDA began to implement the program in the American Rescue Plan Act to relieve the debts of farmers of color, the department discovered that many farm loan borrowers had not disclosed their race, said Goldmon, the senior advisor for racial equity to Vilsack.

“When you have a history of discrimination, there is some reluctance in checking that box that identifies you as Black or Native American or Latino on a farm loan application,” Goldmon said.

But when the department explained that there was a tangible benefit to identifying as a member of the class, Goldmon said it picked up an additional third of farm loan borrowers. Although he understands why people want to see racial breakdowns in the Inflation Reduction Act program, he said that is not what Congress intended when it wrote the law.

“What I’m really more concerned with is making sure that we continue to advance programs that make people feel comfortable indicating where they are,” Goldmon said. “If we get to a point where we need to pull the data, then we’ll have data that’s certainly more reliable based on better service into all of the producers that we’re trying to reach.”

This story was produced in partnership with the McGraw Center for Business Journalism at the Craig Newmark Graduate School of Journalism at the City University of New York.