I’d like to introduce you to the newest VisibleGovernment.ca project: Disclosed.ca. Disclosed.ca is a tool for searching federal contract disclosures across government departments. These contract disclosures have been available online since 2004, when the current Proactive Disclosure laws came into effect. However, until now the records have been spread across ~100 department websites. Further, the disclosure websites are set up such that the records are effectively unsearchable.

Using well-honed coding skills and determination, Disclosed.ca’s founder Ilia Lobsanov has scraped these records from this multitude of department websites. For the first time, Disclosed.ca makes the bulk of federal contract disclosures available on the web as a single searchable database. Lobsanov estimates he has completed scraping about 70% of the federal disclosure websites.

Collecting this information is no easy task. Under the Canadian Westminster system of government, each department is responsible for its own IT. As a result, every department has implemented their own unique disclosure system, publishing the same information in widely varying formats.

Even partially complete, Disclosed.ca has attracted a following of users. The most traffic so far has come from investment websites, who are using Disclosed.ca to do due diligence on companies receiving government contracts and to research trends. This was an unexpected use of the site that took Lobsanov by surprise.

The fact that one of the first uses of Disclosed.ca is investment research is a perfect example of the economic value that is so often locked up in in-accessible government records. Making a few simple changes in how information is published -- making it machine-readable and re-usable -- can free it up to be used in new ecosystems of websites that create value, be it financial or social.

An interview with Disclosed.ca’s founder, Ilia Lobsanov, follows.

What motivated you to start disclosed.ca?

I stumbled upon the Proactive Disclosure web pages by accident, by way of the Environment Canada weather pages. Then I saw every agency was publishing contracts and grants and travel expenses. It was insightful to see the spending amounts. But it was impossible to search any ofthis data. So I got to work writing scrapers for each agency. The first iteration of the site went live in April 2008 hosted on Google App Engine.

How do you think disclosed.ca will help Canadians?

It will raise awareness of the fact that their Government is in fact transparent to an extent. Having applications like disclosed.ca will steer the Government to be even more transparent. This is good for Canadians.

What was the hardest part of creating the site?

Dealing with the Google App Engine API and its limited Search feature was sometimes like fumbling for a light switch in the dark.

On the other hand, acquiring all the contract data was a difficulty I had expected and welcomed. It forced me to write the scraper as generically as possible with the format variations between agencies defined in a configuration file. At the moment however the scraper is not as generic as I would like: there's some code that is specific to certain agencies which should be factored out.

How do you see disclosed.ca, and sites like it, evolving in the future?

Definitely more visualizations. More community involvement at the source code development end. More citizenship participation. For example: letting the public comment on and discuss specific instances of Government data.