• London, England, launched its catalogue of open data: ‘London Data Store’ with 200 data sets. Read BBC coverage here.
• Shortly following, the United Kingdom launched their official federal data catalogue, data.gov.uk, which by some reports puts the US's data.gov to shame. More coverage is here, and an interview with Tim Berner's Lee, who helped put it all together, is here.   The Open Knowledge Foundation’s (a UK non-profit’s) CKAN powers the back end technology.  In an interesting forward-looking move, the UK's Office of Public Sector Information is planning to create provisions to buy or liscense the best of the apps created using the data.
• Six months after the launch of San Francisco’s catalogue, dataSF.org, GovFresh posts a round-up of 17 apps using the information to help citizens get around, be safe, find parking, and generally learn about their neighbourhoods.
• FrontSeat launched CityGoRound, a website listing web sites and IPhone apps based on open transit data, encouraging cities to make their data available.
• Worth noting: US President Barack Obama got together with a number of business and high tech leaders to talk about strategies for modernizing government, providing better customer service, and streamlining operations. (Sounds great.)

In Canada

• Edmonton launched its open data catalogue, becoming the fourth city in Canada to do so. The catalogue is the first one by a North American city to use a new Microsoft product, the Open Government Data Initiative platform. Read news coverage by the Edmonton Journal, and/or an account from the point of view of the volunteers and city workers who put it together. A report on open data prepared by the city, which includes a timeline for the site, is here (pdf).
• Vancouver released an upgraded version of its catalogue with more data sets, and a particular emphasis on geospatial information. Kevin Bowers, Manager of IT Technology Planning at the City of Vancouver posted here on how updating information on the website is now part of the city’s routine procedures.
• K. Bower’s note reminded me of a comment at WiredCamp from Phillip Scott, an IT manager involved with the City of Toronto's catalogue, that “the city looks at open data as part of routine disclosure”. If only all Canadian governments saw things that way.
• Note:  There are rumours of an open data initiative underway at the city of Ottawa, though no formal announcement as of yet.    See comment #2 here. Good job in advance, you folks.

In ‘The Dark Country’, an article in the January/February Walrus, Gil Sochat describes the frustration of journalists whose work is blocked by systemic delays in Canada’s Access to Information process. In the article, Mary Agnes Welch, president of the Canadian Journalist’s Association -- a group that recently co-authored a study ranking this country’s Access to Information laws behind that of India, Mexico, and Pakistan -- blasts our current government:

“[Prime Minister Stephen Harper has] gone beyond merely gagging cabinet ministers and professional civil servants, stalling access to information requests and blackballing reporters who ask tough questions. He has built a pervasive government apparatus whose sole purpose is to strangle the flow of public information.” 

The article details the current problems with the Access to Information process, such as an ‘Amber Light’ flag, where half of incoming Access to Information requests are labeled as 'sensitive', requiring delays of four to nine months. Further, if the government refuses to release a document, the arbitration process for appealing the decision can take up to two years. This systemic reluctance to release information has a very real impact on our democracy.

These days, journalists rarely have the luxury of chasing a story that takes a year or more to write. Increasingly, journalists are giving up on filing requests; stories that break in other countries via Access to Information requests, such as the UK’s recent parliamentarian expense scandal**, now must be discovered in Canada by other means, if they are to reach the light of day at all.

Sochat asks: “Is it possible for our political culture to shift away from its traditional emphasis on discretion and toward transparency?” But, after posing the question, Sochat stops short of suggesting any solutions. A look at how other countries are handling the problem may provide some answers.

In the UK, the non-profit MySociety has built a website, WhatDoTheyKnow.com as a citizen gateway for filing Freedom of Information requests (that country’s equivalent to Access to Information requests). The site helps citizens target what department to send their request to, and provides an online form for filing the request. The site then sends the request on their behalf. Departmental answers are sent back to WhatDoTheyKnow.com, which posts them for all to see. The site also tracks how long requests have been outstanding, highlighting those that have remained unanswered. In October, MySociety estimated that over 10% percent of all UK FOI requests in the 2nd quarter of 2009 were made through the site.

Does a website making it easier to file FOI requests mean that more requests get filed? It stands to reason that it would. But according to BBC Journalist Martin Rosenbaum, while the number of UK requests has increased since WhatDoTheyKnow began operating, the increase has been primarily in requests to departments that are not particularly popular on WhatDoTheyKnow, so the site is not directly to blame. In fact, by providing searchable results of past inquiries, WhatDoTheyKnow may well decrease the total number of FOI requests, as citizens don’t have to ask the same question twice.

FOI administrators in the city of Dallas, Texas recently discovered a similar principle: that publishing more information can lead to less requests. They found that once they implemented a virtual reading room of their city archives, the number of FOI requests dropped by 83%, as ‘problem’ citizens – those that had filed the most requests -- were reassured their government had nothing to hide.

Interestingly, Canada’s Access to Information law may have already paved the way for providing this type of virtual reading room. A little-known stipulation of the Access to Information Act compels federal departments to maintain offline reading rooms – such as the one described here (search on Reading Room) – which include archives of the answers to filled ATIP requests, going back two to three years. These dusty, neglected archives are maintained by the Information Commissioners for each department, and any citizen or journalist can make an appointment to browse their contents in person. It takes only a baby-step of imagination to go from an offline reading room to an online one, with all the benefits that would bring. The amount this would save in freeing up the office space used for maintaining the archives alone seems like it should be enough to justify the cost of the online archiving system.

I've heard rumors that explorations regarding virtual reading rooms may be underway at the Office of the Information Commissioner as we speak. In the meantime, journalists can do more to not just publicize the issue and its impact on our democracy in articles like ‘The Dark Country’, but to also characterize the problem using concrete statistics. Tellingly, none of the journalists I’ve interviewed for VisibleGovernment’s stab last year at cloning WhatDoTheyKnow for Canada -- which, as discussed here, is made more complicated by our government’s insistence on communicating via paper -- were able to tell me the average delay for an Access to Information request in Canada, or provide general statistics on the process. They just knew that their own requests were taking a long time. The journalists I spoke to, with only one exception, were using a paper-based process to file and track requests, meaning old, unanswered requests went in a drawer, and were usually forgotten.

It's an old management standard that what’s not measured doesn’t get fixed. The Information Commissioner, whose job it is to address systemic issues with the Access to Information process, could help matters by starting to publish the Access to Information response time statistics collected under section 70 c-1 of the act in a central location where they can be easily browsed and compared. (While the office currently publishes compliance report cards, these reports seem to be only done for selected departments, and are not a comprehensive review.) Statistics published for each department would highlight where the issues are, and provide a baseline for improvement.

The answers are out there, we need only the political will to implement them.

** For an entertaining first-hand account from Ben Leapmen, the UK journalist who filed the FOI requests for parliamentarian expenses that ended several MP’s political careers, including the speaker of the house, see this video, starting at about minute 21. My favourite quote, from an MP who was criticised for using public funds to build a house for his pet ducks: “The ducks never liked it anyway.”

Last week saw three big open data announcements, from governments on three continents:

In the US, the Obama administration released an 11-page Open Government Directive, the culmination of 2009's Open Government Initiative public consultation process. The directive's language is concrete and actionable, even including specific dates for implementation milestones. Here's a sample:

• To the extent practicable and subject to valid restrictions, agencies should publish information online in an open format that can be retrieved, downloaded, indexed, and searched by commonly used web search applications. An open format is one that is platform independent, machine readable, and made available to the public without restrictions that would impede the re-use of that information.
• To the extent practical and subject to valid restrictions, agencies should proactively use modern technology to disseminate useful information, rather than waiting for specific requests under FOIA.
• Within 45 days, each agency shall identify and publish online in an open format at least three high-value data sets (see attachment section 3.a.i) and register those data sets via Data.gov. These must be data sets not previously available online or in a downloadable format.
• Within 60 days, each agency shall create an Open Government Webpage located at http://www.[agency].gov/open to serve as the gateway for agency activities related to the Open Government Directive and shall maintain and update that webpage in a timely fashion.

In the UK, Her Majesty's Government released its Smarter Government Action Plan, which sets out "how Government will improve public service outcomes while achieving the fiscal consolidation that is vital to helping the economy grow". The first part of the plan, "Strengthen the role of citizens and civic society", includes a set of actions on "Radically opening up data and promoting transparency". Items here include:

• Releasing public datasets and making them free for reuse, with several data sets listed as a first priority for the next year.
• Making government data accessible through a single access point at http://www.data.gov.uk/ by January 2010
• Creating new ways for citizens to interact with public services and public policy -- specifically mentioning a program for public comment on all aspects of the UK's National Heath Service by December 2010.

Australia's Government 2.0 Task Force published a set of draft recommendations for engaging citizens in the business of government.

Recommendation #6 is "Make Public Sector Information open, accessible and reusable". Here's a sample of what they're saying down under:

By default, Public Sector Information (PSI) (..) should be:

• free (provided at no cost in the absence of substantial marginal costs);
• based on open standards;
• easily discoverable;
• understandable (supported by metadata that will aid in the understanding the quality and interpretability of the information);
• machine-readable; and
• freely reusable (not having limitation on derivative uses).

In a move relevant to Canada, and other former colonies, the Australia task force has chosen to follow New Zealand’s lead in recommending that Crown Copyright be abandoned, and replaced with CC-BY as the default licence for all government data.

Note: It's terrific to see the thinking of Canadian advocate David Eaves included in the Australian Task Force recommendations.

Implications for Canada:

With co-ordinated releases from three major western governments, it’s safe to say that the demand for accessible, searchable, and useable government information is no longer a fringe issue. Yet the Canadian federal government stays silent. As Michael Geist said this morning in the Toronto Star:

“These new initiatives herald a dramatic shift in the way governments use the Internet to make themselves more transparent and useful to their citizens. They may also leave Canadians asking if their government is not prepared to lead, then why not at least follow?”

Lack of awareness is not the issue: representatives from Canada’s Treasury Board meet regularly with CIOs from the US, UK, Australia, and New Zealand in what’s called the 'Five Nations CIO Council’ to exchange plans and information. What’s missing is political support from either those who set national policy in Canada or their critics in parliament.

With the government in the middle of a stimulus program which is at present mostly electronically un-traceable, and whose results will be largely un-knowable, it’s not surprising that the powers that be would shy away from adopting policies promoting greater transparency. What’s puzzling is that the opposition parties have not yet found a way to leverage this international movement to their benefit.

TORONTO: Earlier this month, Toronto city officials met with local web designers, developers, and community activists at the city’s first ever “Open Data Lab” at Toronto Innovation Showcase.  The event followed the launch of TO’s open data site, toronto.ca/open, that morning.  Here’s a picture of the action, taken from Mark Kuznicki’s summary of the event:

The event had a particularly novel structure, featuring 10 minute speed-dating sessions between city IT staff and local developers, followed by parallel half-hour brainstorming sessions.  The Mayor of Toronto, David Miller, later met with open data volunteers for beer.

 

ITWorld quotes Mayor Miller describing the city’s open data initiative:

“Anyone can download, analyze and mash up our data or write applications to make it more accessible and useful. It is an invitation to Torontonians to do what they do best: create, innovate and build a better city,”

The Open Data Lab spawned a vigorous discussion group, here, where ideas are debated and formats are suggested to the city.  City officials have been doing an admirable job of keeping on top of the discussion.  To help with the task, a Toronto citizen created a web app, datato.org to organize and prioritize requests for the city.

Note: A panel discussion on Open Government from earlier in the day at Toronto Innovation Showcase is available as a webcast.

EDMONTON: City of Edmonton IT staff hosted an Open Data workshop at City Hall this weekend.  The focus of the event, attended by 45 local developers, was brainstorming around the city’s upcoming open data catalogue.  Read coverage of the event by the Edmonton Journal.

VANCOUVER: The City of Vancouver will be hosting another Open Data Hackathon event at City Archives December 9th.

As a side note, there are indications that Vancouver will start publishing road repair advisories next month in GeoRSS, a format which includes encoded geographic co-ordinates.  The change in format will enable applications that, for instance, allow users to subscribe to upcoming repairs to roads on their commute, or in their neighbourhood.

In Canada:

• Vancover’s Open Data site launched last month to wide acclaim. The city recently completed a survey to help prioritize which data sets to open up next.
• The City of Toronto will be hosting an Open Data Lab to engage the developer community around the city's soon-to-be-launched data portal at http://toronto.ca/open. The event, featuring a talk by Peter Corbett of Apps for Democracy, will be part of ‘Toronto Innovation Showcase’ on November 2. See details here.
• I was happy to participate as a panelist at the National Town Hall on 'Citizens' Engagement and State Accountability’ hosted by the Office of the Information Commissioner in Ottawa on Sept. 28th. A video webcast of the panel is here.

Elsewhere:

• The National Association of State Chief Information Officers has released “A Call to Action for State Government – Guidance for Opening the Doors to State Data”. The report gives recommendations to help state governments get started with data transparency portals.
• The cities of Portland and Seattle have announced open data initiatives.
• The w3C has released a draft guide to "Publishing Open Government Data", which features a list of "Staightforward Steps to Publish Government Data".
• The UK’s Cabinet Office has asked for help from citizens in designing its open data portal. From the Cabinet’s Digital Engagement Blog: “With over 1000 existing data sets, from 7 departments (brought together in re-useable form for the first time) and community resources, we want developers to work with us to use the data to create great applications; give us feedback on the early operational community; and tell us how to develop what we have into a single point of access for government-held public data”. Read the full announcement here.
• In a move diametrically opposed to open government initiatives elsewhere in the UK government, the UK’s national postal service has threatened legal action against a citizen website providing free postal code lookups. The website, called Ernestmarples.com after the British postmaster general who introduced the postal code system, allowed other websites to circumvent the 4000 pound fee the postal service charges for this information. In the short time Ernestmarples.com was available an ecosystem of non-profit and other websites providing location-based lookups flourished around it – all of whom are scrambling to find other options. As many of the websites are volunteer-based public services, the 4000 pound fee is not affordable. The BBC quotes Jim Killock, a digital rights activist: "It is easy to see that large numbers of small business ideas and not for profit services are being blocked by these licence fees," he said. "It is in effect a tax on innovation." Canada’s postal code boundary definitions are locked down by similarly prohibitive access fees.

This week, VisibleGovernment.ca launched a last-minute campaign to solicit submissions to Canada’s copyright consultation process asking for Crown Copyright reform.

This may seem like an esoteric issue: but crown copyright is a constant legal threat to citizen projects that use and share government information. It’s an issue that we run up against regularly here at VisibleGovernment.ca, and one that, ultimately, hurts all Canadians. Here are some examples of how:

1. First: Send the Lawyers.

Crown Copyright includes ‘Fair Use’ provisions that are meant to protect Canadian’s ability use information in a way that is consistent with public good. However, these provisions are routinely interpreted by government lawyers in only the most narrow sense.

As described by Michael Geist, in 2007, the group Friends of Canadian Broadcasters were sent a legal notice to cease and desist publishing videos of parliamentary committee proceedings on their website. According to government lawyers, re-publishing unaltered videos put the group in contempt of Parliament.

FoCB challenged the finding, and managed to extract a special, ground-breaking exemption from a Parliamentary committee which allows all forms of recordings of parliamentary proceedings to be re-used and re-distributed – as long as they are unaltered, and not used for commercial purposes.

FoCB is to be commended. Many groups, when faced with a potential lawsuit, would have backed down. Instead, it stood up for what should have been obvious to even government lawyers – that this information belongs to the public. Unfortunately, the new exemption allowing re-distribution of Crown Copyright material only applies for this one type of government information.

2. Crown Copyright Promotes Information Monopolies.

In the US, the watchdog website fedspending.org (whose influence on the US government is described in more detail here) posts statistics on the government contract tendering process. For instance, users can look at a particular company, such as Boeing, to see how many of its contracts were awarded without a bid process. The website helps citizens, and other companies, spot unfair practices in contract tendering.

In Canada, this same information is distributed via Merx, who holds exclusive agreements with many departments and agencies. The information is only viewable before a contract is awarded – after which it disappears. After a delay of several months, the awarded contract may show up on a federal disclosure sites, but not in a way where it can be easily traced to its tendering process.

As described by a contributor to our discussion group, the company behind Merx is prepared to vigorously defend its contract agreements as the exclusive provider of this information. Any group that uses Merx’s information – information which originates in our public institutions -- is opening the door to an expensive lawsuit. This application of Crown Copyright, which provides Merx with an apparent information monopoly, is a barrier to replicating fedspending.org in Canada.

3. Open Source? Not if our Government Touches It.

This spring, at an Ottawa conference where governments share their web 2.0 innovations, I watched a demo of custom software for sharing internal information that a department of the Government of Ontario had cobbled together from a combination of open source components.

The conference was attended by representatives from different municipal, provincial, and federal governments, and it seemed likely that some of them could also make use of the software improvements done by this team. Indeed, the gist of the presentation was ‘We did this – you can do it too.’ I asked: since the software was constructed on an open source base, would they be releasing their modifications back to the community so that they could be adopted by other governments directly? The answer was that they wanted to, but it was ‘with the lawyers’. Because the work was produced by provincial employees, it was protected by Ontario Crown Copyright and could not be released.

This was actually the second project I heard of at the Government of Ontario that wanted to be open source, but was held up at the legal stage. That project had been ‘with the lawyers’ for over a year. Meanwhile, the software lingers, unshared, and unused by other governments. The lawyers apparently have other priorities.

Earlier that day, a visitor from a US government department had shown some custom software his agency had built to visualize the dynamics and movement of fires. Their software was open source, and hosted on Google Code. He estimated that about 40% of their contributions came from members of the public. When asked what the software license was, he said ‘Public Domain’ – as all works by US government employees are not copyrightable. He seemed surprised by the question, as if it were inconceivable that things could be any other way.

4. Models for Change

Other countries are reforming Crown Copyright to promote access to and sharing of government information. In February, 2009, the UK government’s Power of Information Task Force final report found that Crown Copyright was a major barrier to the re-use of Public Sector Information, and recommended that Crown Copyright be changed to a ‘Crown Commons’ license to encourage re-use.

The UK government is following up on this recommendation: Tim Berners-Lee, the inventor of the word-wide web, is one of the people tasked with creating the new Crown Commons license.

New Zealand has gone one step further. In a recent published draft, the state has proposed getting rid of Crown Copyright entirely. Wherever Crown Copyright would be used, Creative Commons Attribution (CC-BY) would be used instead. The proposal argues:

“Now more than ever is there a very present need to bring information the Government holds on behalf of its people into the public domain so that it may be used in ways that stimulate innovation, generate cultural creativity, social interaction and dialogue, while also kick starting economic growth.”

Reading this recently released New Zealand proposal has inspired us to action. Already, some Canadian departments are bravely inventing their own licenses for sharing information. In other situations, like parliamentary proceedings described above, exceptions are being made on an expensive, case-by-case basis. The system should be changed to favour allowing the re-use of government information by default, rather than by exception.

It so happens there is a copyright consultation going on right now in Canada. While the public focus so far has been almost entirely on digital rights and its impact on the music industry, this is a unique opportunity to submit your views on Crown Copyright.

We’ve prepared a draft submission, modeled directly on Vancouver’s recent Open Data, Open Standards, Open Source motion (Open3), which you can use to either send as is, or modify to express your own views. The important thing is that you speak up -- we may not get another chance for some time.

News and ideas on Open Government Data from around the web.

• Nanaimo, BC and San Francisco join the list of cities with Open Data initiatives.  Nanaimo beat out Toronto and Vancouver to launch the first municipal open data site in Canada.   David Eaves traces the timeline of open municipal data initiatives in his blog post 'Rise of the Open City'.  Also of note from Nanaimo is an innovative council minutes site, created on the cheap by city employees.
• With the excitement around the 'open city' movement, stories on open government have been breaking into Canadian mainstream media -- see recent stories in the Toronto Star and Edmonton Journal.
• Elsewhere, the Sunlight Foundation announced the finalists in its 'Apps for America 2' contest.   Finalists include an app described as an Everyblock for state data.
• The government of New Zealand has released a draft Open Access and Liscensing Framework to promote changes to state copyright licensing that allow the freedom to re-mix and re-use.
  From the paper:

  “Now more than ever is there a very present need to bring information the Government holds on behalf of its people into the public domain so that it may be used in ways that stimulate innovation, generate cultural creativity, social interaction and dialogue, while also kick starting economic growth.”

  (hat tip to Tracey at CivicAccess)
• Australia's Government 2.0 Task Force has been pushing out some valuable content in the last month, see especially their of Government 2.0 Issues Paper.   The government task force has allocated $2.45 Million (AU) for web projects, creating a public consultation site to:

  1. approve or criticise the projects we’ve set out
  2. propose improvements to those ideas
  3. propose alternative ideas
  4. suggest people or firms/agencies that might do a good job of these projects

  Will we see a Canadian Government 2.0 Task Force, equipped with a similar budget?

At a panel discussion on redesigning government websites at PDF09, a question was raised about regulating US agencies to publish standards compliant information. A suggestion from the floor came up: “Don’t worry about regulation. Create software that makes it cheaper and easier for governments to publish standards compliant data, and agencies will use it.”

It’s a great point. Showing governments that something can be done by doing it first can inspire/shame the powers that be into action. It also creates a path of least resistance for agencies to follow. As Clay Johnson, director of Sunlight Labs argues: “We can get government to change by writing code and writing websites faster than we can get members of congress to agree how to do it. So let's do it for them." (from video linked here)

The US government website USASpending.gov shows how this process can work. Three years ago, a group of senators including Barack Obama and John McCain introduced a bill to US congress called the ‘Federal Funding Accountability and Transparency Act’. The act stipulated that the office of Management and Budget create a searchable website of nearly all government spending by 2008. According to a report by the US non-profit OMBWatch:

“At the time, the Bush administration and others believed that such a website could not be created, and if it could, it would be very costly and take many years to put into place.” (p. 19)

Regardless, the act passed. Faced with having to implement the website, the US Office of Management and Budget found that the easiest and most cost effective thing to do was to copy a non-profit site already in use that was doing the same thing. fedspending.org, built by the non-profit OMBWatch, had for some time been collecting and sharing information on federal spending obtained from Freedom of Information requests and other sources. The Office of Management and Budget licensed the software behind fedspending.org, and launched it as USASpending.gov, supplying the same information directly from the government.

As described in the Washington Post, the OMB was initially reluctant to participate with the non-profit, quoting OMB associate director Robert Shea as saying:

"OMB Watch spends a great deal of its efforts criticizing what I do the rest of my day, trying to improve program performance, [so] my level of interest in cooperating with them was very low,"

But finally Shea decided:

"OMB Watch had already proven it could be done, so why do it from scratch?"

Exactly. The OMB delivered the site ahead of schedule, and for exponentially less than the originally anticipated cost. Further, USASpending.gov kept many of the innovative features implemented by OMBWatch, including an API for programmers to access the data – a groundbreaking move at the time.

Creating websites that push the envelope on visualizing, sharing, and publishing government information drives innovation within government by showing what’s possible. As in the example of USASpending.gov, these websites can even become the cheapest and best option for governments to adopt internally. Governments are, in general, terrible places for creativity and experimentation – the Outside-In model may well be the most viable way of getting innovation into government websites.

Stay tuned for an in-depth comparison of the federal contract data available on USASpending.gov, and the data that’s available in Canada under proactive disclosure laws.

News and Ideas on Open Government Data from Around the Web

• US Open Government Initiative Collaboration Policy Drive Complete

  Approximately 20,000 people participated in the US Federal Government's Open Government Initiative, providing ideas and input on "How to Make Government More Open".   The initiative, run in three stages on three separate tools, was itself an experiment in using open, collaborative tools to collect and distill  feedback.  While reviews of the effort were mixed -- with many columnists focusing on early efforts by a fringe groups to derail the process -- governments everywhere have something to learn from the experience.

• Sunlight Labs to One-Up data.gov

  Another open government effort that has received mixed reviews is data.gov.  While off to a promising (and precedent-setting) start, the website's collection of data sets has not grown as quickly as hoped, and  lacks some key features, such as the ability to provide feedback on the data.  Sunlight Labs has announced its intention to help the government, and users of data.gov, by doing one better.

• Groups launch citizen-created data.gov's

  Inspired by data.gov, citizen groups in Russia and New Zealand are first off the mark in setting up their own government data listings.  Watch this space for Canadian updates.

• Open311 Initiative Underway

  The Open Planning Project is facilitating an Open311 Initiative, "A collaborative effort to create an open standard for 311 services".

• UK Guardian Crowd-sources Analysis of MP Expenses.

  In a great example of crowd-sourced journalism, Guardian readers are mining thousands of pages of MP expenses. Sample the discussion here.

• UK ‘Power in the People's Hands’ Report Released.

  The UK Cabinet Office has published a concise but wide-ranging survey called 'Power in the People's Hands: Learning from the World's Best Public Services' that includes several open data case studies.  While not strictly open-data related, I found the 'Entitlements' case study, where Sweden was able to cut wait times in half, profound.  A good read.

In Canada:

• Senate committee recommends Community Indicator data, used in programs such as CommunityAccounts.ca, be freed across the country.
• David Eaves posts an update on Open Vancouver.
• ChangeCamp organizers gather to reflect and pass best practices on to other cities.

I’d like to introduce you to the newest VisibleGovernment.ca project: Disclosed.ca. Disclosed.ca is a tool for searching federal contract disclosures across government departments. These contract disclosures have been available online since 2004, when the current Proactive Disclosure laws came into effect. However, until now the records have been spread across ~100 department websites. Further, the disclosure websites are set up such that the records are effectively unsearchable.

Using well-honed coding skills and determination, Disclosed.ca’s founder Ilia Lobsanov has scraped these records from this multitude of department websites. For the first time, Disclosed.ca makes the bulk of federal contract disclosures available on the web as a single searchable database. Lobsanov estimates he has completed scraping about 70% of the federal disclosure websites.

Collecting this information is no easy task. Under the Canadian Westminster system of government, each department is responsible for its own IT. As a result, every department has implemented their own unique disclosure system, publishing the same information in widely varying formats.

Even partially complete, Disclosed.ca has attracted a following of users. The most traffic so far has come from investment websites, who are using Disclosed.ca to do due diligence on companies receiving government contracts and to research trends. This was an unexpected use of the site that took Lobsanov by surprise.

The fact that one of the first uses of Disclosed.ca is investment research is a perfect example of the economic value that is so often locked up in in-accessible government records. Making a few simple changes in how information is published -- making it machine-readable and re-usable -- can free it up to be used in new ecosystems of websites that create value, be it financial or social.

An interview with Disclosed.ca’s founder, Ilia Lobsanov, follows.

What motivated you to start disclosed.ca?

I stumbled upon the Proactive Disclosure web pages by accident, by way of the Environment Canada weather pages. Then I saw every agency was publishing contracts and grants and travel expenses. It was insightful to see the spending amounts. But it was impossible to search any ofthis data. So I got to work writing scrapers for each agency. The first iteration of the site went live in April 2008 hosted on Google App Engine.

How do you think disclosed.ca will help Canadians?

It will raise awareness of the fact that their Government is in fact transparent to an extent. Having applications like disclosed.ca will steer the Government to be even more transparent. This is good for Canadians.

What was the hardest part of creating the site?

Dealing with the Google App Engine API and its limited Search feature was sometimes like fumbling for a light switch in the dark.

On the other hand, acquiring all the contract data was a difficulty I had expected and welcomed. It forced me to write the scraper as generically as possible with the format variations between agencies defined in a configuration file. At the moment however the scraper is not as generic as I would like: there's some code that is specific to certain agencies which should be factored out.

How do you see disclosed.ca, and sites like it, evolving in the future?

Definitely more visualizations. More community involvement at the source code development end. More citizenship participation. For example: letting the public comment on and discuss specific instances of Government data.

Thanks, Canada! You pitched in to help us raise $1005 during our Beers for Canada campaign between June 30th and July 2nd.  Now that the dust has settled, the final donation tallies are:

• $1005 raised.
• 15 beers at $7
• 10 pitchers at $20
• 7 rounds at $100

Special thanks go out to these blogs for the great coverage that helped us get the word out:

• Michael Geist
• Data Libre
• Apathy is Boring
• Freshbooks

As well as Tim O'Reilly, Austin Hill, Tara Hunt, and other twitter power-houses who kept the campaign's energy up.

Stay tuned for an announcement on how this money will be spent.

Top Stories

• Michael Bloomberg, Mayor of New York City, announced a BigApps contest for applications based on open NYC data.
• Vivek Kundra, US CIO, unveiled a dashboard showing metrics for US federal IT expenditures, including per-contract timelines and performance indicators. While there is no stated plan for this sort of metrics-based interface for the rest of US government expenditures, it can only be a matter of time.
• Sunlight Labs launched Transparency Corps, a mechanical-Turk application for breaking up transparency tasks that only humans can do – such as tabulating the over 9000 ways the US govt. refers to Walmart in its contract listings -- into small, bite sized chunks so that anyone can participate.

Favourite Quotes

• “I have to say I’m feeling pretty good right now.” – Ellen Miller, Sunlight Foundation E.D. after Vivek Kundra’s demo of an IT spending dashboard .
• “They [the govt.] actually do look at these things when they exist. “ -- Mike Mathieu, FrontSeat founder, talking about how many of the points voted up on ObamaCTO ended up on the president’s implementation list.
• “Technology brings power to the edges.” – Ellen Miller, Sunlight Foundation
• “We’ve got to make government by default searchable and linkable.” – Jeff Jarvis
• Alec Ross speaking to how a Columbian rally organized by four unknown young men on Facebook did more to damage the govt. than 10 years of military action.
• “Does the average person believe they can have an effect? If not, you don’t have a representative government.” – Tom Crowl, at the ‘Hacking the City’ BOF
• “You actually don’t need to regulate that much. Create software that makes it cheaper and easier for governments to publish standards compliant data, and agencies will use it. “ -- suggestion from the floor at “Redesigning .gov for Transparency and Participation” panel session
• “One of our problems is just that we have too much content. Some of our sites have over a million pages, and a team of maybe four people.” -- bracing honesty from Sheila Campbell at USA.gov, the agency that oversees US federal websites, at the “Redesigning .gov for Transparency and Participation” panel session. An EPA webmaster added during Q&A that his organization had over 500,000 pages of hand-coded HTML to manage.
• “It’s not about ‘embracing technology’, it’s about embracing people.” – Joe Raspers, on why the Obama08 internet campaign was successful.
• The hoots of applause during Jim Gilliam’s presentation on ‘Imagining White House 2.0’.
• Esther Dyson somehow ending up with a ‘Hello My Name Is’ Meetup sticker on her back.

Happy Canada Day! This Canada Day, treat your country to a beer. VisibleGovernment.ca is launching its first ever fundraising drive, ’Beers for Canada’. For the price of a beer, you can help VisibleGovernment.ca build tools to promote transparency and encourage our leaders to share more information openly.

Where will your beer money go?

The funds raised by the Beers for Canada effort will be used for:

• Creating new tools and websites that encourage more open communication between government and citizens.
• Launching the Code For Canada application design competition that awards prizes to people who build web, facebook, and iPhone apps that provide visualization, analysis, and access to federal government data sets.
• Working with other open government organizations like The Sunlight Foundation in the U.S. and MySociety in the U.K. to bring tools they’ve created to Canadian screens, and to share Canadian-made applications with others.
• Encouraging government openness in public forums, helping government organizations to share their data, showcasing examples of open government, and promoting the benefits of transparency in public office.

All it takes is a visit to www.beersforcanada.com this Canada Day holiday to help connect public officials to the general public. Have a great holiday and enjoy your beverages responsibly.

Achieving effective citizen partcipation in government -- participation that goes beyond a vote once every four years, is a hard problem. As part of being involved with Personal Democracy Form next week, I've been priviledged to hear from Jim Gillian, founder of White House 2, on his experiences building a crowd-sourced website for political policy discussion.

Here's Jim explaining his reasons for getting involved:

"I started this right before the election, as an outside effort, not sanctioned by the White House, because I knew there were all kinds of restrictions on what they could do, both legally and culturally. I figured that if I could show it working, if I could figure out how it *could* work, then it would give the people who want to do this ammunition to convince people in the bureaucracy to let them do it. It's hard to innovate in a bureaucracy, big brands have a hard time with failure. When it comes to democracy, the United States has the most important brand in the world.... whereas I'm very good at failing.

So the advantage I have is no bureaucracy. The challenge for me is publicity, but i have managed to get 8000 people involved, which is plenty to experiment with. Obviously, attention will be no problem for the White House. "

On how White House 2 works:

"At WH2, we make policy a game -- fun and competitive. As a way to encourage good talking points, and to discourage spam, we have an economic system called political capital. Massively multiplayer online games like World of Warcraft all have systems like this. It's modeled after how political capital works in real life. The more people like the things you do, the more political capital you earn.

This is a fairly common concept on social web sites, but the twist at WH2 is you can actually spend it. You can buy "ads" for priorities you want to get more attention, and track their progress. If I have an idea, I can post it on WH2, and I can write talking points explaining why it's a good idea. If people like it, I earn political capital, and then I can spend it promoting my idea."

Fascinating stuff. Jim will be talking about his plans for integrating prediction markets into White House 2 at our panel discussion at Personal Democracy Forum in New York city next Tuesday.

Our recent close brush with an election made me wonder if we can get the sort of change embodied in Obama's Memo on Transparency and Open Government on the table as an election issue in Canada.

Many of the points of that memo can be traced back to this report, Towards a 21st Century Right to Know Agenda, produced by a coalition of US non-profits with the support of the Sunlight, Ford and Carnegie Foundations.  The group held a series of round table discussions with different levels of governments, uncovering ways to systematically break down barriers to transparency.

While governments in this country will have their own unique issues, there's no question that many of the transparency roadblocks addressed in the report exist in Canada as well.   Here are 8 of the most pertinent recommendations in the report, ready for adoption by candidates who want to become the next Prime Minister:

1. The Prime Minister should issue a high level statement that he will oversee the ‘most open, honest, and accountable Canadian government ever’
   • The Prime Minister should indicate that his administration will rely on interactive technologies that can make transparency achievable in new ways and create new approaches to make government accountable to its people.
   • The Prime Minister should announce that he will immediately take action to launch a transparency initiative to ensure that government is running in an open, ethical and accountable manner.
   (Adapted from Recommendation B1)
2. The Prime Minister should immediately instruct agencies to operate in a more open style, making information available to the public in a timely manner and in searchable formats except where prohibited by law.
   • The Prime Minister should instruct agencies to actively and affirmatively disseminate all information, and not to simply wait for an Access to Information request.
   • The Prime Minister should direct agencies to evaluate their practices for handling sensitive information to ensure that the presumption of openness prevails.
   (Adapted from Recommendation B2)
3. The Prime Minister should invite the public to identify top documents and databases to make publicly available.
   • The Prime Minister should instruct the Treasury Board to use interactive technologies, such as an online survey, to involve federal employees and the public to identify high-priority information needs.
   (Adapted from Recommendation B3)
4. The Prime Minister should mandate the creation of a centralized digital system for Access to Information Requests that interacts with each agencies Access to Information office.
   • Such a system could find and manage requests more efficiently and reduce the duplication of requests. While not all requests are appropriate for publication, the default for a modern ATIP request should be both digital and public, with support for paper-based or non-public requests still available.
   (Adapted from Recommendation D5)
5. The Prime Minister should implement a process to better present information about the federal budget in an online format – tracking proposals and changes throughout the process.
   • Facts on the government’s spending of tax dollars are among the most demanded and least understood information held by the federal government. While much data is made available on proposed budgets and agency spending, the information fluctuates in format, scope, and level of details between various government sources that it becomes impossible for the average citizen to understand. The public wants simple answers to straightforward questions on government spending, and with new online tools, these answers should be easier than ever to provide.
   (Adapted from Recommendation D6)
6. The Prime Minister should instruct agencies to request sufficient resources – funding, personnel, and technical capacity – in annual budget requests to implement the vision of a more transparency government – and the Prime Minister should commit to budgeting sufficient funds. (Adapted from Recommendation E3)
7. The Prime Minister should instruct agencies to make transparency a factor in federal employee job descriptions.
   • Changing the culture of government to be more transparent will require direct individual accountability for employees and supervisors and recognition of work to improve transparency. Too often in the past, information requests have been denied, new online tools delayed, and information removed without any specific official or employee being held responsible for the action. As much as possible, this accountability should be structures as positive incentives for employees – better performance evaluations for those employees that make strong contributions to ensuring an agency or office is conducting business more transparently.
   (Adapted from Recommendation E13)
8. The Prime Minister should mandate the creation of a Transparency Scorecard, to be created by a working group, for each agency.
   • Each agency is to display their scorecard as part of their website.
   (Adapted from Recommendation E14)

I'll be in New York on June 30 moderating a panel discussion on 'Imagining White House 2.0: Making Open Collaboration Platforms Work' at Personal Democracy Forum, the world's largest conference on the intersection of technology and politics.

The panelists are:

• Craig Newmark, founder of Craigslist
• Jim Gilliam, creator of White House 2
• Fabrice Florin, executive director of NewsTrust.net
• Mark Elliott, director of Collabforge

Vivek Kundra, CIO of the United States and the man behind data.gov, will be speaking earlier in the day.

I had an amazing experience at PDF last year listening to speakers like Clay Shirky and Lawrence Lessig explore technology's impact on democracy. The only thing missing from the conference was more Canadians.  With the recent spike in public interest in the topic,  and the awareness raised in this country through events like changecamp, I'm hoping to meet more Candian political staffers at PDF this year.

If you're from Canada and attending the conference, let's get in touch.

Sir Tim Berners-Lee, who helped invent the world wide web, has been asked by UK Prime Minister Gordon Brown to help open up government information in the UK.  This post on the UK Cabinet Office Digital Engagement Blog (yes, their Cabinet has a Digital Engagement Blog), describes how Berners-Lee will be on a team tasked with:

• overseeing the creation of a single online point of access and work with departments to make this part of their routine operations.
• helping to select and implement common standards for the release of public data
• developing Crown Copyright and 'Crown Commons' licenses and extending these to the wider public sector
• driving the use of the internet to improve consultation processes.
• working with the Government to engage with the leading experts internationally working on public data and standards

Who would be the closest Canadian equivalent to Sir Berners-Lee? Are there any Canadian data and standards rock-stars the open government data world could tap into?

VisibleGovernment.ca board vice-president Andy Kaplan-Myrth will be participating in a "Roundtable on E-Democracy" for parliamentarians on Capital Hill on June 11th.  Andy's agenda for topics to discuss at the meeting is open for your ideas and comments on etherpad.

Andy says:

[I]nstead of just writing an outline for a presentation/discussion based on the topics that I think are important in terms of citizen engagement, I thought I would use some of the very technology that has sparked an interest in citizen participation, like EtherPad.com.

So if you're interested in how changing technologies, trends, behaviours, expectations and standards present challenges and solutions for citizen engagement with government in Canada, please head over to the below page at Etherpad.com and contribute your thoughts. And please pass on the link -- the more people who contribute, the more I can bring your thoughts to decision makers on the Hill.

Clay Johnson, director of Sunlight Labs, gave a call to arms to software developers at Web2.0 Expo last month. In his presentation, available here, he encourages software developers and designers to push for greater government transparency by:

• creating visualizations of government information
• creating screenshots (comps) of redesigned government websites
• becoming community organizers and convening open government hackathons

A nice quote:

"Software is faster than concensus. We can get government to change by writing code and writing websites faster than we can get members of congress to agree how to do it. So let's do it for them." (minute 14)

At around minute 27, Johnson talks about the power of a screenshot of a redesigned government website to spur change.  One of the most popular posts on the Sunlight Labs blog featured a mock-up of a redesigned FEC web site.  The post go thousands of hits -- from inside the FEC intranet.  This was promptly followed by a call from the FEC chairman about re-designing the site.

If you were to pick a federal department website in Canada to re-design, which one would it be?

Two weekends ago I was at Changecamp Ottawa with over 100 other people intent on discovering new ways to use technology to improve citizen participation, and by extension, democracy.  Some personal highlights:

• Seeing David Hume draft the beginnings of a policy on participation for the city of Ottawa.
• Watching Senator Elaine McCoy, and the very motivated StimulusWatch.ca group, run an unconference discussion on their project.
• Participating in a session run by Tracy Lariault of CivicAccess, on access to government data.  This was one of the most popular sessions -- some people from Ignatieff's technology team were in the crowd.
• Chatting with a group of coders who were thinking of taking the software base for fixmystreet.ca in a completely direction for solving citizen problems.
• Being one of the lucky few to score a changecamp t-shirt.

Congratulations to the organizers.  Particular congratulations and thanks are due to the City of Ottawa, who hosted the event at City Hall.  It shows foresight: encouraging a more open, technology-savy city government enhances Ottawa's position as a tech center.  As David Eaves argued recently, in support of an open Vancouver:

"...programmers and creative workers in all industries are attracted to places that are open because it allows them to participate in improving where they live. Having a city that is attractive to great software programmers is a strategic imperative for Vancouver. Where there are great software programmers there will be big software companies and start ups."

Reflections by a ChangeCamp Ottawa organizer, including some sweet photos of the event, are here.
Future ChangeCamps:

• Changecamp Vancouver is June 20.
• Changecamp Edmonton is being planned for September.