Open Data

India adopts United Nations Fundamental Principles of Official Statistics

The ten Fundamental Principles of Official Statistics, as endorsed by the UN General Assembly, are set out below:

Principle 1. Official statistics provide an indispensable element in the information system of a democratic society, serving the Government, the economy and the public with data about the economic, demographic, social and environmental situation. To this end, official statistics that meet the test of practical utility are to be compiled and made available on an impartial basis by official statistical agencies to honor citizens’ entitlement to public information.

Principle 2. To retain trust in official statistics, the statistical agencies need to decide according to strictly professional considerations, including scientific principles and professional ethics, on the methods and procedures for the collection, processing, storage and presentation of statistical data.

 Principle 3. To facilitate a correct interpretation of the data, the statistical agencies are to present information according to scientific standards on the sources, methods and procedures of the statistics.

Principle 4. The statistical agencies are entitled to comment on erroneous interpretation and misuse of statistics.

Principle 5. Data for statistical purposes may be drawn from all types of sources, be they statistical surveys or administrative records. Statistical agencies are to choose the source with regard to quality, timeliness, costs and the burden on Respondents.

Principle 6. Individual data collected by statistical agencies for statistical compilation, whether they refer to natural or legal persons, are to be strictly confidential and used exclusively for statistical purposes.

 Principle 7. The laws, regulations and measures under which the statistical systems operate are to be made public.

 Principle 8. Coordination among statistical agencies within countries is essential to achieve consistency and efficiency in the statistical system.

 Principle 9. The use by statistical agencies in each country of international concepts, classifications and methods promotes the consistency and efficiency of statistical systems at all official levels.

Principle 10. Bilateral and multilateral cooperation in statistics contributes to the improvement of systems of official statistics in all countries.

Read more at http://pib.nic.in/newsite/PrintRelease.aspx?relid=144847

Rail Radar HotSpots Allows You to Plan Your Journey without Disruption.

One of my post on Rail Radar at Govperneur blog crossed 6000 pageviews . It was an unexpected but pleasant milestone. So i went back and reviewed the article. To my utter dismay i found that the Rail Radar project which was covered in the article has been closed down and link now redirects to the National Train Enquiry Service (NTES)  website of the Indian Railways (CRIS).  On further digging i found that the project which was lauched in partnership with Rail Yatri has been shelved in favour of a new interface (supposedly user friendly) developed by CRIS  in this article.

In that article i had shared that this project marks a major milestone in the digital governance highway the Indian railways is cruising in. With the closing of this project in favour of the new inhouse interface by CRIS the project has moved backwards.  However not to be left behind, Rail Yatri (Stelling Technologies) has created Rail Radar Hotspots. This site provides train movements as per scheduled timings (since CRIS is maintaining its monopoly of train movement data) and also highlights the hotspots which are likely disrupt your travel.

Rail Radar by Rail Yatri

RailRadar™ HotSpots is a live tracker of spots of significant events and incidents called as ‘HotSpots’ occurring across the country which has or has the potential of impacting train travel in India. This live tracker is shown on an interactive map and allows users to locate these HotSpots across the country. It also helps determine if any of the scheduled train has the potential of getting impacted by it.

The effects of these HotSpots vary in terms of severity. Some of the known after-effects which we have tracked are as follows –

  • Widespread cancellation of services (partial or fully)
  • Changes in time schedule or route schedule
  • Delays
  • Rush in trains, unavailability of seats, etc.
  • Accidents/Mishaps
  • Increase in crime.
  • Increased cost of travel

It is difficult to predict a complete list of events or incidents which can impact train travel. However, one can take clues from past events/incidents over the years which have resulted in train travel getting affected in adverse manner. Some of them are –

  • Accidents, Mishaps, Breakdowns
  • Engineering & Construction work and maintenance
  • Significant public gatherings
  • Weather, Natural Calamity, etc.
  • Man-made disruptions
  • Events which cannot be explained or be attributed to – but typically are outcome of Chaos Theory or Domino Effect.

Every year thousands of such reported across the nation which has resulted in significant impact on travel.

Over the last decade, train network and traffic in India have grown tremendously. Indian Railways expects this trend to continue over the next few decades. Also increasingly Indian Railways passengers today want to know more – beyond just ‘an’ ETA of ‘a’ train at ‘a’ station. As the volume and complexity of information increases – we believe that new methods of information dissemination (associated with Trains) will be required. RailRadar™ is one such attempt on that line. We believe that value of information/directions can greatly be enhanced when presented graphically rather than just plain text.

RailRadar™ HotSpots is one such representation of information dissemination mechanism associated with events which can significantly impact train travel in India.

How this affects us?

The larger public interest is affected as CRIS is trying to maintain its monopoly of train data. They should release it as API and allow for innovation. Many innovations are possible like this Wake up call System called Pyka.

The Government has embarked on a mission to promote Open Data and to bring more information into the public domain. Much of the information published there are just statistics. It will be game changer if government takes a proactive approach in releasing such data that are relevant to large section of the population day in and day out.

This my friends represents the challenge and opportunity for Govpreneurs. What other data can be released by the Government that can have a major impact in citizen friendly government. Share your comments.

Credits – FAQs at Rail Radar Hotspots

Making Sense of Linked Data, Open Data and Visualization

Data becomes more meaningful when it is linked with a common meta data.

 

Some persistent questions that needs to be deliberated are:

Who creates a meta data that others include?

Will first movers have an advantage?

How Open Data and Linked Data will make a difference to Development.

Find educative presentations on open data, linked data, visualization and linked information.

Go here   http://linkedinfo.ikmemergent.net/

Which Country is closest to Your Utopia–in terms of MDGs

What if you could guide the destiny of our country. What if you could decide on the importance of various development choices we face as a country. Is infant mortality more important to you or health spending(as % of GDP) of the government. What will happen if you are given the choice of changing various policy options towards the achievement of the Millennium  Development Goals. What will you choose?

Well you see how your choices will workout and also find a list of countries which form the closest to your ideal priorities in achieving the MDGs. Go ahead and try this game at YourTopia – meaning Your UTOPIA. You will get a list of countries in descending order. Try this… it fun and enlightening.

yourtopia app

Yourtopia sums up human development according to your criteria. In a short quiz you choose how important different dimensions of development are to you, and the application calculates how countries do overall, according to your priorities.

Your personal measure of social progress calculated by Yourtopia makes tracking developments easy by combining large datasets into a single index.

With your anonymised quiz data you furthermore participate in constructing the world’s first multiple-dimension index of human development that overcomes the problem of arbitrary indicator-weighting and proxy choices through open public voting.

More at www.YourTopia.net/about

Technology behind the Next Generation WebApps for Government

The next generation web applications for Governments are using frameworks like Ruby of app development. Server scalability using the Amazon Cloud and other services Amazon RDS and Amazon SES. Here is latest list published by the UK Government which is working on the Gov.Uk Portal as a one place for all data. Now the scope has also been expanded to include private data sources also and is made available in an API. Licensing is an factor to think about in this emerging area. The more open the better. Here is the technology stack for Gov.Uk portal reproduced from Cabinet Office Blog, UK.

gov.uk

Hosting and Infrastructure
  • DNS hosted by Dyn.com
  • Servers are Amazon EC2 instances running Ubuntu 10.04LTS
  • Email (internal alerts) sending via Amazon SES and Gmail
  • Miscellaneous file storage on Amazon S3
  • Jetty application server
  • Nginx, Apache and mod_passenger
  • Jenkins continuous integration server
  • Caching by Varnish
  • Configuration management using Puppet
Languages, Frameworks and Plugins
  • Most of the application code is written in Ruby, running on a mixture of Rails and Sinatra
    Rails and Sinatra gave us the right balance of productivity and clean code, and were well known to the team we’ve assembled. We’ve used a range of gems along with these, full details of which can be found in the Gemfiles at https://github.com/alphagov
  • The router is written in Scala and uses Scalatra for its internal API
    The router distributes requests to the appropriate backend apps, allowing us to keep individual apps very focussed on a particular problem without exposing that to visitors. We did a bake-off between a ruby implementation and a scala implementation and were convinced that the scala version was better able to handle the high level of concurrency this app will require.
Databases
  • MongoDB
    We started out building everything using MySQL but moved to MongoDB as we realised how much of our content fitted its document-centric approach. Over time we’ve been more and more impressed with it and expect to increase our usage of it in the future.
  • MySQL hosted using Amazon’s RDS platform
    Some of the data we need to store is still essentially relational and we use MySQL to store that. Amazon RDS takes away many of the scaling and resilience concerns we had with that without requiring changes to our application code.
  • MaPit geocoding and information service from mySociety
    MaPit not only does conventional geocoding (what’s the lon/lat for a postcode) but also gives us details of all the local government areas a postcode is in, which lets us point visitors to relevant local services
Frontend
  • HTML & CSS (naturally), with elements from HTML5 & CSS3 where appropriate
  • Javascript uses jQuery, jQuery UI, Chosen, and a variety of other plugins
  • Gill Sans provided by fonts.com
  • Google web font loader
Misc

 

Source:  http://digital.cabinetoffice.gov.uk/colophon-beta/

Coming Soon…Open Data, in an office near You

open data portal india

The Open Data Portal of India went live. Its built on the Open Government Platform released by India and US jointly. The platform has been released in open source and is freely available for download and use. The new Open Data Portal has a total of 17 data sets as of now from 7 departments. It is expected that more and more data sets will be released on the site as Departments release the data. There are 5 Data Controllers who are nominated to co-ordinate data released from their departments.

Interestingly the portal also hosts four mobile sample apps which are also available for download.The portal also has communities for Developers and special focus communities on Energy,Education,Health,Rural Development and Agriculture.

It seems that NIC has really worked a lot on the portal and has departed from its tradition in many ways. For example the usage of sliders in home page using fusion charts to display Visitor Statistics .

This site has many Web 2.0 features like blogs, forum and even flaunts some user related data unlike any other government site. This is probably the first time that NIC guys are working on an open source project of this scale. This is the first Government website that uses tags for articles extensively and has a dedicated tags page. This is also the first website that has a AddThis share button.

The website provides social login options using the Janrain social login service. Government officer with NICeMail service id can login directly with their credentials.

open data community

There is a directory of world wide open data sites with 30 countries listed. Almost all countries have only one open data site except the USA that has 20 sites listed. It may be due to the face that the US Government is closely involved in the Open Data initiative in India. Ofcourse you can ask for a dataset and suggest them to be added in the open data portal.

There is no uniform copyright policy for data released on the portal and is governed by the licence set in metadata of the dataset. While the portal bears major resemblances to its USA version it is still in its infancy in terms of number of datasets, their formats and the number of departments. However its a commendable initiative and a far reaching one. Its the mark of a new era in transparency and openness in Government.

Govhashing the Digital Agenda Assembly 2012 Open Source software and content

Govhash is dedicated to creating newer smarter and agile approach to solve governance issues. With that idea in mind we will be experimenting with the various open source solutions that are freely available for use. We will try to hash them to apply for newer geographies, domains or applications.

The Digital Agenda Assembly of the EU set a fine example for the OpenGov movement by making open the content and the platform. It is based on Drupal 7. The discussions are also available as a zip archive.

I tried installing the software on my shared hosting server, created the database table and tried to run the install script. It showed a MySQL error with PDO. On further enquiry I found that the error is because my shared host not providing the required MySQL version as per Drupal Requirements.

drupal daa12 pdo error

Lessons Learnt:

Check up the requirements and compatibility before installing any software.

When signing up for hosting be clear about your requirements and ensure that the service provider is supporting the specs.

Data.Gov celebrates three years of Openness

Data.Gov the open data portal of USA celebrates its three of openness.

Growing from 47 datasets in 2009 to nearly 450,000 datasets today, Data.gov reaches across 172 federal agencies to bring data to innovators, developers, analysts and citizens across the nation. The data shows up in smart phone apps, websites, and information that lets people buy smarter, use energy more efficiently, and find better health-care solutions each day.

Source: Data.gov

Data.gov had become a vibrant community of experts from the public, academia, industry, and government to address the national challenges in energy, health, and law, and this year new communities launched on safety, education, manufacturing, oceans, ethics, developers, and business. These topic-based communities drive open data to higher levels by organizing challenges to inspire new innovations to supporting code-a-thons in cities, to building platforms for entrepreneurs to find new technologies and grow their businesses—Data.gov is putting federal data to work for Americans.

Data.gov is managed by the General Services Administration’s Office of Citizen Services and Innovative Technologies working with the U.S. Chief Information Officer and U.S. Chief Technology Officer.

At this juncture I couldn’t resist the temptation to compare the Indian response to the #OpenGov movement. India has released the Open Government Platform in collaboration with US . Though India is not a signatory to the Open Government Partnership this is a welcome step towards Open Data. It’s a challenge and an opportunity for India to use open data a driver of growth, economic equity and sustainable development.