Last week the U.S. Securities and Exchange Commission proposed its first expansion of open corporate data in nearly nine years.

Here’s where the new proposal came from, what it means, and why it matters.

For Financial Information: a Phantom Menace

As longtime Data Coalition supporters know, the SEC was an early adopter of open data. In February 2009, the agency finalized a rule that requires all U.S. public companies to report their financial statements using the eXtensible Business Reporting Language (XBRL) format.

In theory, as the CFA Institute explained last month, XBRL should have allowed companies to automate some of their manual compliance processes. And XBRL should have made corporate financial information easier for investors, markets, and the SEC itself to absorb.

But it hasn’t gone well.

As a new report from our sister organization, the Data Foundation, explains, the SEC chose an overly-complex structure for public companies’ financial statements. Worse, the complex structure doesn’t match the accounting rules that public companies must follow (see page 22). As a result, it is unnecessarily hard for companies to create these open-data financial statements, and for investors and tech companies serving them to absorb such data.

These difficulties have led some in Congress to conclude that the SEC shouldn’t require most companies to file financial statements as open data at all. This is the wrong reaction. The problem isn’t open data; the problem is the way the SEC has implemented it.

Fortunately, the SEC is taking steps to fix its financial statement data. Last year, the agency signaled it may finally upgrade from the current duplicative system, in which companies report financial information twice (once as a document, then again as XBRL), and replace it with a single submission that is both human- and machine-readable. And the recommendations of the Data Quality Committee (see page 37) show how the complex structure could be demystified.

And then there’s last week’s announcement.

For Non-Financial Information: A New Hope

Financial statements are an important part, but only a part, of the information public companies must file with the SEC. Companies also report their basic corporate characteristics, stock structures, executive compensation, subsidiaries, and much more. Partly because its XBRL mandate for financial statements has been a failure, the agency never got around to transforming any of this non-financial information into open data.

For the last decade, companies have continued to report their non-financial information – even information that could easily be structured, like tables and check marks – as plain, unstructured text, in electronic documents that "mimic yesterday's paper ones." 

Here is the cover page of Google’s annual SEC report. It should be expressed as open data. It isn’t. But change is coming.

Last year, the Data Coalition filed a comment letter recommending that the SEC should replace all these documents with data. We recommended (on pages 2 and 12) that the SEC should begin with the “cover pages” of the main corporate disclosure forms. Cover pages include every company’s name, address, size, public float, and regulatory categories.

The information reported on cover pages is very basic and fundamental for U.S. capital markets – and yet it is not available as structured data, creating surprising inefficiencies.

To create a list of all well-known seasoned issuers, for instance, the Commission and investors must manually read every filing, employ imperfect text-scraping software, or outsource those tasks to a vendor by purchasing a commercially-available database. (Page 2.)

Last November, the SEC signaled a change. In a report on modernizing non-financial disclosure, the agency recognized that adopting standardized data tags for cover page information “could enhance the ability of investors to identify, count, sort, and analyze registrants and disclosures” (page 22). And the report cited the Data Coalition’s comment letter as its inspiration for this (footnote 90).

And last week, the SEC made it official, in a formal rule proposal on Wednesday, October 11. The agency proposed to begin requiring public companies to submit all cover page information as standardized, open data (page 105).

Over the next two months, the SEC will collect public comments on this change from the financial industry, the tech industry, transparency supporters, and other interested parties. Then, it will decide whether to finalize the proposed rule.

For the Future of Corporate Disclosure Information: the Force Awakens

By transforming the basic information on the cover pages into open data, the SEC can unleash powerful analytics to make life easier for investors, markets, and itself.

Software will be able to identify companies of particular sizes, or regulatory characteristics, automatically, from the SEC’s public, freely-available data catalogs. There will no longer be any need to "manually read every filing, employ imperfect text-scraping software, or purchas[e] a commercially-available database."

Of course, the cover pages are only the beginning.

Beyond the financial statements, whose slow transformation began in 2009, and the cover page information, whose transformation is starting right now, the SEC’s corporate disclosure system is full of important information that should be expressed as open data, but isn’t. We recommended a strategy for tackling the entire disclosure system in 2015.

Where to begin? The recommendations of the SEC’s Investor Advisory Committee are a good place to start.

The more information is turned from documents into data, the more analytics companies, like our Coalition members idaciti and Morningstar and Intrinio, can turn that data into actionable insights for investors (and for the SEC itself).

The SEC’s open data transformation has been grindingly slow. But last week’s announcement shows it isn’t dead – and in fact is full of promise. We’ll keep pushing.