How open is company data in Open Government Partnership countries?

Posted on April 16, 2012 by

Today, the day before the Open Government Partnership meeting starts in Brasilia, OpenCorporates is publishing a major new report into access to company data in OGP countries, and the picture is not good.

Out of a total of a possible 100 points, the average score was just 21, with several major countries (including Spain, Greece and Brazil) scoring zero. A score of 100 means that the company register is an open data register, making detailed information free for reuse under an open licence, and also makes the information available as open data. A score of zero means the central register can not even be search without payment or registration.

Highest score is the Czech Republic, with a score of 50, though the UK will achieve a score of 70 when it starts publishing a limited set of data under an open licence in July.

Virtually all OGP countries score very badly for openness of company data, with several – including countries such as Spain, Greece and Brazil – effectively closed for the public, civil society and the wider world, undermining corporate governance, and providing a fertile ground for corruption, money laundering, organised crime, and tax evasion

The full report is available here, and the data is available (under an open licence at http://bit.ly/ogp_open_company_data), but a summary of the data should be available below:

Update: broken links fixed

Posted in open data, opencorporates | Tagged , , , , , , , , , , | Leave a comment

Announcing the OpenCorporates Advisory Board… governance with teeth

Posted on April 12, 2012 by

Right from the word go, we wanted OpenCorporates to be a different kind of organisation, and we had three core goals we wanted to achieve.

First, we wanted it to have as its central mission the opening of company data for the public good.

Second, we wanted it to be sustainable, not dependent on the vagaries of grants or government funding, but generating income from adding value to public data by making it more useful, more accessible and, crucially, more reusable.

Third, we wanted it to help develop the open data ecosystem with an innovative and successful business model that could help make the open data world a first-class player in the business community, the way open source is in the software world.

Because of this, we set up OpenCorporates as a for-profit (technically speaking we set up a for-profit company, Chrinon Ltd, that is the publisher of OpenCorporates), giving us a fair degree of freedom, and allowing us, if we wanted to, to accept investment. Were we founded in the US, we may have gone down the route of a B-Corporation, a for-profit that includes social good in its governing principles, and that’s something we may well look at in the future.

But as we grow, we need to make sure the third of those goals doesn’t subsume the first. We also needed an honest friend to give us feedback on tricky issues – helping us tread the path between the various competing rights – and to tell us when they thought we were going down the wrong path.

To do that we’ve set up an Advisory Board, but one with teeth. Their core goal is to be that honest friend that we wanted – one who shares our goals and ideals but is willing and able to criticise us when they disagree. But they will also have the power to make a difference to what we do. Cutting to the chase, this is what we’ve agreed:

  • The Advisory Board would be consulted on key strategic directions, to help ensure that OpenCorporates/Chrinon Ltd (the company behind OpenCorporates) acts in the wider public interest.
  • The Advisory Board would be a resource for OpenCorporates to ask for advice, e.g. on policy where there’s a balance of competing interests (privacy etc).
  • The Advisory Board would normally keep its discussions with OpenCorporates private, but could go public if it strongly felt OpenCorporates was behaving badly, or going in the wrong direction. Prior to doing that it would engage with OpenCorporates to try to find a mutually acceptable solution.
  • The Advisory Board would be regularly given a data dump of the entire OpenCorporates database (which is under the ODbL licence), which it could release to the community in the case that either OpenCorporates went into liquidation, or dissolved in some other way, or should OpenCorporates have veered sufficiently from acting in the wider public interests the Board felt it necessary to help create a fork of the project.

We think this goes further than any other similar project, and helps ensures that we always keep central that goal of opening company data for the public good.

There are currently three members of the board, all of whom have exceptional credentials in the open data and transparency world. We may well appoint further members to the board (with the agreement of the existing members) in the future, for example to widen the area of expertise. The members are:

  • David Eaves David needs no introduction to the open data community, and his 3 Laws Of Open Data were a key influence on us, but he describes himself as “a public policy entrepreneur, open government activist and negotiation expert”. He is retained by several governments to advise on open government and open data, works with two spin-offs of the Harvard Negotiation Project and advises businesses on open source strategies and community management.
  • Kaitlin Lee Kaitlin is the senior developer for the Subsidyscope, Clearspending and Six Degrees of Corporations projects at the Sunlight Foundation. She researches transparency related issues in several areas of federal spending, including tax expenditures, risk transfers and contracting. She holds a B.S. in Applied Math from Johns Hopkins.
  • Andrew Stott Andrew was the UK’s first Director for Transparency and Digital Engagement.  He led the work to open government data and create data.gov.uk; and after the 2010 Election he led the policy development and implementation of the new Government’s commitments on Transparency of central and local government.
    Following his formal retirement in December 2010 he was appointed to the UK Transparency Board to continue to advise UK Ministers on open data and e-government policy.  He also advises other governments both bilaterally and through the World Bank and the World Wide Web Foundation.

In fact the Advisory Board has already proved invaluable to us in numerous discussions, and helped us draw up the OpenCorporates principles. They also provide an important reassurance to the rest of the community, and perhaps even a template for other open data organisations too.

Posted in milestone, open data, opencorporates | Tagged , , , , , , | Leave a comment

Corporate confidentiality, company relationships, and why this ‘competitive advantage’ is anti-competitive

Posted on April 5, 2012 by

Restricted access

Last week OpenCorporates was in Basel at the Financial Stability Board‘s workshop on Global Legal Entity Identifiers (LEI) for corporate entities, as a member of the advisory panel.

It’s worth giving a sentence or two’s background here for those who aren’t familiar with this – and that’s probably everyone who’s not part of the global financial regulatory community or concerned with identifying corporate entities in the financial community.

The core idea of the global LEI is to allow all participants in financial activity to get a unique id that could be used to identify them in financial activity anywhere in the world, in any activity. This stems from the desire to reduce systematic risk such as those that nearly brought down the entire financial system post-Lehman’s. Part of this clearly requires the surfacing and publishing of the relationships between different companies within the same corporate group, sometimes called a corporate hierarchy, though we’re actually taking about a web of corporate entities, rather than a hierarchy as such.

The workshop was conducted under Chatham House Rules, and so we can’t share any papers or attribute quotes, but we can touch on some of the general discussions, including the one on why collecting this hierarchical information was potentially problematic.

This is because some companies – and in fact some company registers, especially secrecy jurisdictions – consider this information to be confidential. Specifically, to quote a paper by Linda Powell of the Federal Reserve Board et al, “A common reason provided for why a relationship between entities should not be public is that releasing the ownership data to the public could harm the competitive advantage of an entity.”

We were speaking on one of the panels regarding the scope of the work, and when it was our turn to speak, this was one of the key issues we addressed. We said, in short, that this was nonsense. We couldn’t think of a single legitimate reason why a company would refuse to publish its corporate structure.

First, this is not about legitimate commercial confidentiality but about protecting incumbents’ power, position and, often, legacy business models.

Second, this is often as not about evading scrutiny – from society, from shareholders, and from regulators. Make it difficult enough to understand the hundreds or thousands of complex legal entities making up a global corporation such as BP, and you can do what you want. This is not how a responsible company should behave, and not a reason for keeping this secret.

Third, if it is does reduce a company’s competitive advantage, that’s a Good Thing, because this is one competitive advantage that is positively harmful. Competitive advantage should be about coming up with a better way of doing something, taking a risk on a new product and reaping the benefits, having a closer relationships with your customers, and so on. Not obscuring the public record so that it’s impossible for the wider world to understand who they’re doing business with, competing with, investing in or regulating. (Encouragingly this got a excellent response from the workshop attendees.) I think we know where this ends up (Enron, CDOs?), and it’s not in a good place.

Fourth, this lack of transparency, enabled in part by company registers restricting information about companies to those who can afford the high prices, forces all companies to play the dodge the regulator/tax/public scrutiny game. This reduces the attention spent by good companies on real-world innovation and being a good corporate citizen, and forces all into a game of regulatory arbitrage, playing jurisdictions and regulators against each other to force them to a race to the bottom that leaves companies able to act outside the law.

[It's worth remembering that company are artificial legal entities formed and sanctioned by the state, and that limited liability companies are ones where the cost of failure is borne not by the owners but its customers, suppliers, workers and society at large.]

Finally, as the World Bank’s compelling Puppet Masters report makes clear, this opacity of corporate structures is precisely the fertile ground which allows corruption and organised crime to flourish, as this cloak of ‘corporate confidentiality’ is used to hide all manner of illegal, criminal and anti-social behaviour.

So the next time a large corporation says they can’t or won’t publish the information about their corporate structure ask them just what they’re trying to hide.

p.s. In the spirit of full disclosure, Chrinon Ltd, the company behind OpenCorporates is 90% owned by Chris Taggart, 10% owned by Rob McKinnon and has no subsidiary companies or parent companies.

Posted in open data, opencorporates, standards | Tagged , , , , , , , , , , | Leave a comment

Major milestones: 40m companies, 50+ jurisdictions, moving out of alpha, getting respectable

Posted on March 27, 2012 by

We did it – last week we went over the 40 million companies mark, with the addition of our 52st jurisdiction: Mauritius. As well as being quite a significant milestone in itself, we’re also using this as an excuse to replace that alpha tag that’s graced the top right hand corner of OpenCorporates since we launched with a beta one (we’re still rapidly developing, after all).

In truth, we should have replaced it some time ago, as not only have we become the largest open database of companies in the world by far, we’re also being increasingly recognised by (for want of a better word) the establishment. We’ve previously written about our work with the UK government, and with the EU/W3c’s work on a business vocabulary, but recently we were also appointed to the advisory panel (pdf) for the Financial Stability Board‘s work on a global legal entity identifier. Like the EU/W3c work this is important for a couple of reasons.

First, it’s a recognition that the problems of finding, understanding and influencing companies is a global problem that can’t be solved by the regulators and officials alone. If the community – and by community I mean open data community, journalists, civil society, innovators, and even companies themselves – isn’t empowered with the tools it needs to do this, we’ll end up with not just innovation atrophying as large companies entrench their position, but democracy being marginalised.

Second, it’s a recognition of the important and pioneering work that OpenCorporates has done in this area, on resources that are, frankly, what the big business-data companies spend on tea and biscuits.

Third, it’s a sign that these big institutions – check out the Financial Stability Board’s website for an idea of what we’re talking about – are beginning (and I’d stress the beginning) to embrace the concept of open data, and the dangerous effects of the existing asymmetries of access to data. In fact, our inclusion is doubly impressive, given that we’ve written publicly and forcefully about company identifiers, and can be relied upon to say what we think is right, rather than going along with the cosy consensus.

And this recognition is important too, because it says that open is right, and thus begins to isolate those company registers who are not interested in transparency, but only in selling the data to those with deep pockets – in Europe countries such as Italy, Spain and Greece come to mind, none of which even allows free and open searches to check the existence of a company, and are even less transparent than most tax havens.

Posted in milestone, open data, opencorporates | Tagged , , , , , , , , | Leave a comment

OpenCorporates: The directors’ cut

Posted on March 12, 2012 by

One of the most often requested features for OpenCorporates, especially by journalists, is for us to extract information about the directors and officers for the over 35 million companies we’ve opened up.

And as Alex Howard reported on O’Reilly Radar last week, that feature is now live (we also gave a sneak peak during our presentation at the Investigative Reporters and Editor’s NICAR conference last month).

So far we’ve imported over 10 million positions, and the ability to search by name across all the jurisdictions for which we have officer information. At the moment that’s:

and coming soon: New Zealand. (We’ve also got a handful from Massachusetts but they’ve blocked our scraper – now why would they want to prevent people getting access to this information?)

You’ll notice that the UK’s not on there (nor in fact are any European countries), which is somewhat embarrassing for it, given the UK government’s stated aim of being the most open government in the world. Unfortunately while the information is held, it’s only sold to those with deep pockets, thus undermining access, and reducing both corporate transparency and governance. In short, we could buy it, but then we’d have to stop being openly licensed, and that’s a route we’re never going to go down.

It’s also worth noting that there are several key jurisdictions that don’t actually collect this information, including Delaware and Nevada – and here you’d think there’s nothing we could do about this. However, you’d be wrong, and I’ll explain why with a concrete example.

Search for Mitt Romney, and you see that he’s listed as being an officer for 7 companies, including chairman and director of Healthcare Acquisitions, which is a foreign branch company in Mississippi. In fact this is a branch, set up in 1989, of a Delaware company, as you can see if you click through the company page, and then to the official registry page. So even though we don’t yet have that Delaware company (and even we did, we wouldn’t be able to get the directors), you can still find out the directors information.

Click instead on the name Mitt Romney on one of the entries on the search page and you get this page:

As you can see, not only do you get the company this particular position belongs to and the other officers,names of the fellow directors, you also get the other companies with similarly named people. And that’s an important distinction that’s worth stressing. We’re not saying that John Smith, Director of Megacorp Inc, is the same person as John Smith, Secretary of AnotherCo LLC. We don’t have that information, and, actually, neither does anyone else in general.

But what it does allow you to do is in a matter of seconds do the sort of research that would have previously taken hours. And if the web isn’t about reducing by several orders of magnitude the time or cost of things, then it’s not being truly disruptive.

p.s. We’re adding the officer search and info functionality to the next version of the OpenCorporates API, which we’re currently working on. Watch this space.

Posted in milestone, open data, opencorporates | Tagged , , , , , , , , , | Leave a comment

Over 10 more US states, millions more companies

Posted on February 23, 2012 by

One of the things we’re most proud of with OpenCorporates is the way we quietly get on with making the world’s biggest open database of companies bigger and better, adding territories with the help of the open data community. Every day we make thousands of HTTP requests, scrapes, and downloads to OpenCorporates, constantly adding companies, filings, and other data.

There’s probably a blog post just itemising the stats behind OpenCorporates, but for today to we just want to focus on one in particular (and it ties in nicely with the presentation we’re giving this afternoon at the Investigative Reporters & Editors’ NICAR conference). This weekend, we finished adding Louisiana to OpenCorporates, the 11th US state we’ve added this year, and bringing the today number to 22, and the total number of companies we have to 36 million. For the record the other states are:

Florida doesn’t quite make that list, as it’s still being imported (we’ve just over a million Florida companies so far). While we haven’t yet got every state in the US, we’re well on our way, and we’ve got several more in the pipeline, meaning the day is in sight not only when you can do one search across all the US and get back every Google entity, for example, but get the whole thing back as openly licensed data, too.

Of course, there are some obstacles in the way, not least Delaware and Illinois, but we’re hoping that before we get to those they’ll embrace transparency and open data ;-)

Posted in Uncategorized | Leave a comment

3 reasons why the EU’s new Business Vocabulary is so important

Posted on February 22, 2012 by

This weekend, the EU published its public draft of its Business Vocabulary (along with Person and Location), to help make it easier for organisations within Europe, including governments themselves, to exchange information relating to companies.

Now if I haven’t already lost your attention, I think this is pretty important, and not just those who handle corporate data, but to also to all those interested in openness in general, for three reasons:

  1. The vocabulary democratises the ability to share this important information, removing the need for restrictive central registers, which are inevitably tied up with process, governance and access issues;
  2. The results are free of IP restrictions;
  3. The fast, lightweight process was an example of a huge organisation (the European Commission) for once being focused on solving immediate problems rather than grandiose undeliverable: in short the EC went for a Minimum Viable Product.

I think it’s worth tackling these in a little more detail, but feel free to skip the first or second and skip to the process bit, if that’s what floats your boat.

[I should also state that  OpenCorporates was a member of the Working Group that put the vocabulary together and was fairly heavily involved in the Business Vocabulary discussions, having arguably done more to open company information than any other organisation in the recent history. It's also worth saying that this process wouldn't have achieved anything had it not be for the excellent work of the W3c's Phil Archer and the EC's Piotr Madziar and Vassilios Peristeras]

1 The Business Vocabulary

Like including mathematical equations in a book, the phrase ‘business vocabulary’ is an excellent way of putting off any ‘normal’ people who might otherwise be interested. Yet like the protocols that underlie the internet, getting this sort of thing right matters, and really all we really mean by business vocabulary is not some heavyweight XML schema, but a lightweight set of agreed terms and principles that remove the barriers for communication.

In this case it’s very lightweight, as there as there’s only really one critical part: what is a company, and how do we identify it? Despite the simplicity of the question, this is an area in which it’s easy to tie yourself into knots, should you want to. Countries have very different ways of thinking of companies (are partnerships or sole traders companies, for example?), and different ways of creating them too, sometimes handling them centrally, or giving the job to regional registers or courts.

The Business Vocabulary neatly sidesteps this, instead focusing on two core elements: a Legal Entity and a Formal Identifier. If you’re going to exchange information about a company it needs to be a legal entity, and it needs to have a (single) formal identifier. And that’s pretty much it. The only thing to add is that the formal identifier is made up of two parts: the identifier (e.g. a ‘company number’) and an issuing authority (e.g. a company register), which would ideally be identified by a URI.

See that wasn’t so bad was it ;-)

There are a handful of other properties that are listed, including the date of issue of the identifier, the registered address, company type, some of which are more clearly defined than others, but really that’s it. But already, it allows company registers around Europe to start publishing their data, and consuming other company registers’ data (for example, to understand the status of home companies for foreign branches they have registered) without the need for a highly centralised clearing house with its own closed system of data exchange.

It’s also worth stressing that this solution is not tied to any particular representation. It could be turned into a string identifier, linked data, or XML of some sort. Whichever is used, transformation from one to the other should be easy.

2. Open data and open standards

One of the best outcomes of the process, is that the resultant Business Vocabulary is genuinely open, unencumbered by IP restrictions. More than this, however, the whole process was focused on this outcome, with all agreeing this from the start. (In fact all participants were required to explicitly agree that their contributions would be free of IP restrictions, meaning the contributed use cases and discussions on them can also be openly published.)

This means, for example, that the list of identifying authorities also needs to be free of IP restrictions, and it’s this sort of detail which really matters when we’re talking about open standards – one solution, particularly if the vocabulary is to be used outside the EU (which it certainly could be) would be for the W3c to maintain and publish this list, given its interest in the semantic web and IP-free solutions.

3. Minimum Viable Product process

Although it was never called this, the concept of a Minimum Viable Product seemed embedded in the process from the start, and it took a very different route to other governmental ones I’ve been involved with. Contributions were encouraged to made on a wiki, conference calls were held weekly with a strict one hour limit, and we were given a target deadline of the end of January (we started in November). No long meetings. No backroom deals. Admittedly by technology startup standards this may be slow, but for government bureaucracy standards this was definitely agile, and the tight timings really focused people’s minds on results.

So congratulations Phil & co and let’s hope we not just get some useful feedback on the vocabulary, but that other governmental  organisations can learn from the process too.

Posted in open data, opencorporates, standards | Tagged , , , , , , , , , | 1 Comment