OGP Products

Data.gov.sg

data.gov.sg

data.gov.sg is Singapore’s open data platform where businesses, researchers and policy makers can access data for decision-making at scale. By centralising key national datasets, improving discovery and providing universal access to data distribution channels, data.gov.sg improves government transparency, public service as well as drives socio-economic value in a cost efficient manner.

SINCE 2015

Active

Product Overview

Team Members

1PM

6Engineer

1Design

1½Ops

⅓Policy

Cost Per Quarter

$935,085

Show breakdown

$364,127

Salary

$179,025

Infrastructure

$276,229

Corporate Overhead

$80,918

Equipment, Software & Office

$34,786

Others

Usage

484K

Page views

+1%

286M

Automated downloads (API calls)

-2%

152K

Manual downloads

+24%

Satisfaction

5.26/ 7

Ease of use

+4%

Key Highlights

Metric Definitions

Page views: number of user visits across all pages
Automated downloads (API calls): number of times data is downloaded remotely, excluding manual downloads
Manual downloads: number of time the download button is clicked
Ease of use: Single Ease of use Question (SEQ) is posed to the user after each task, to gauge the ease of using the platform from a scale of 1 (lowest) to 7 (highest)

Q2 2024 Apr-Jun

Key Features Added:
- Revamped the data.gov.sg platform, including the adding of data reuse and usage metrics on the landing page
- Added semantic search, filters, sorting and redesigned search to a split view for easier browsing of datasets
- Implemented a new dataset details page, including a dataset preview, filtering and API code snippet for querying of datasets via the platform apis
- Added a data viewer for viewing of sample data, including the summary statistics on categorical variables and null values, as well as filtering and search capabilities
Metrics:
- Backdated download metrics since Q1 2024 (changed from 193k to 123k) to correct for a miscalculation due to internal API calls for the real-time API datasets

Q1 2024 Jan-Mar

Key Features Added:
- Added caching to datastore APIs, resulting in an 80% reduction of API call volume to the server
- Write APIs deployed to allow government agencies to automate their data uploading workflow
Growth:
- Added NEA’s real-time weather readings for lightning forecasts
Metrics:
- Q1 2024 infrastructure costs were previously reported as $96707, but have been corrected on 10th Jul 2024 to $191919 to include additional product costs

Q4 2023 Oct-Dec

Key Features Added:
- Ability to generate API keys
- Scaled up API infrastructure to handle higher QPS and reduce CPU load
- Deployment of write APIs for automated data ingestions
Growth:
- Implemented datastore for E-Gazette Website (https://www.egazette.gov.sg/)
- Automated integration with Singstat using Singstat Table Builder APIs
- Upgraded integration infrastructure with NEA to message queues

Q3 2023 Jul-Sep

100% Human traffic migrated
100% Charts migrated
91% API traffic migrated
Scheduled deprecation of metadata APIs for Q4
Shut down legacy.data.gov.sg application
Added the API calls from real-time APIs to the metrics
Key Features Released:
- Launched the admin panel, Admins now able to create, edit, publish and unpublish datasets
- Admin OTP login implemented
- CSV ingestion and validation based on canonical data formats (SGDRM)
- Metadata APIs (dataset/collections listing and metadata) released

Q2 2023 Apr-Jun

Rewrote public application for data.gov.sg and launched the beta
46% Human traffic migrated
Key Features Released:
- User interface redesign, including landing page, search and dataset pages
- Migration away from CKAN data infrastructure towards modern AWS stack (DynamoDB, RDS, S3)
- Upgraded read API infrastructure
  - Deployment of backwards compatible read APIs (96% of total API traffic)
  - Testing of upgraded metadata APIs (4% of total API traffic)

Built by