Before you can process and use data, you need to have data. Places that contain or store humanitarian and development data include:
Open Data is data that’s freely available to the general public. Mostly when people talk about Open Data, they mean Open Government Data – data about government services, funding etc that has been made public via internet sites. Open data initiatives are cropping up all over the world now. Places to look for these include:
- Open Knowledge Foundation – promotes open data worldwide
- OKFN mailing lists – join these to get updates on open data initiatives around the world
- How to publish open data: the Open Data Manual
- OKFN Open Data Catalogue – lists of where open data is worldwide
- The Guardian’s Open Government Data catalogue
- Open data day wiki – still one of the best lists of open data communities around
- The DataHub.org – open data search site
- Open Data Directory – seems to be defunct
- Open data is also indexed by several major data warehouse sites including Microsoft Azure datamarket and Google’s public dataset repository
Crisismapping is looking primarily at data and events that happen in real time. But this data will also often need to be put in context – although many data sources are either static or contain older data (years rather than minutes or days old), they can still be useful to us. There are now lots of sources of open data for human development. Some of the bigger ones are:
- World Bank indicators
- UN websites
- UN data repositories
- billion prices project – scraped market prices from around the world
- AI for Development datasets list
- Global Healthcare Facility Database Initiative – Proposed work from WHO et al to map health facilities around the world.
- Demographic & Health Surveys (DHS) – Representative data on Population, HIV/AIDS, Health & Nutrition covering 90 countries.
- Multiple Indicator Cluster Survey (MICS) – Comparative data on health, education, child protection and HIV/AIDS collected since 1990s covering 50 countries.
The open data movement are working hard to unlock data held in governments etc., but there are other ways to get the information you need, including citizen science and scraping. Here are some places to look:
- Grassroots mapping – balloon mapping, passenger pigeon etc for aerial images and video
- DIY Drones – for aerial images and video
- Scraperwiki – for liberating data from pdfs etc on websites
- Pachube – for networks of sensors
- World-Wide Human Geography Data Group (WWHGD)
- Map Library
- – Africa base layers various formats
There’s also a list of links to datasets in I Can Haz Datastore.