What did we work on?

• Embedded Elections map in local and international media agencies for elections coverage
• Polished the product, reducing search load time from 3.4s to 1.9s and improving responsiveness by 20% on average
• Onboarded new datasets, including lightning and WBGT real-time APIs and the Dengue Cluster geospatial dataset

Key Features Added:

• Launched elections microsite with vernacular support (https://elections.data.gov.sg/en)
• Added product status page for monitoring (https://status.data.gov.sg/)
• Added Citations snippet for easy reference of citation instructions
• Added Google Colab notebook for direct exploration of datasets
• Enhanced the data viewer table on dataset pages to allow for iframe embeds, enable sorting of numeric columns and API code snippets based on active filters
• Revamped the admin experience, improving the dataset validation and ingestion process, optimising login and providing admin metrics

Metrics:

• Started tracking number of engaged sessions, through dataset consumption or interactions with data assets

Key Features Added:

• Removed beta prefix from the url
• Updated the Real-Time API dataset details page for API querying within the user-interface, added API schemas, status codes and examples
• Added map-based visualisations on Mapbox for GeoJSON datasets, providing previews of the data and metadata viewers on the user-interface
• Optimised core web vitals on both desktop and mobile to ensure optimised user experience (>90% good URLs and 0% bad URLs)
• Deprecated legacy products, saving ~9.9K USD per month, migrated from Open Search to Typesense, saving ~3k USD per month

Metrics:

• Decrease in page views due to changes in the user journey wherein users may now browse datasets directly on the search results page
• Removed internal data sharing platform metrics due to scheduled deprecation on 31st Dec 2024
• Added intermediate metrics to measure progress towards search and discovery of data.gov.sg
• Switched from Single Ease of use Question to User satisfaction scores to measure overall user satisfaction with data.gov.sg from a scale of 1 (lowest) to 5 (highest)

• Revamped the data.gov.sg platform, including the adding of data reuse and usage metrics on the landing page
• Added semantic search, filters, sorting and redesigned search to a split view for easier browsing of datasets
• Implemented a new dataset details page, including a dataset preview, filtering and API code snippet for querying of datasets via the platform apis
• Added a data viewer for viewing of sample data, including the summary statistics on categorical variables and null values, as well as filtering and search capabilities

• Backdated download metrics since Q1 2024 (changed from 193k to 123k) to correct for a miscalculation due to internal API calls for the real-time API datasets

• Added caching to datastore APIs, resulting in an 80% reduction of API call volume to the server
• Write APIs deployed to allow government agencies to automate their data uploading workflow

• Added NEA’s real-time weather readings for lightning forecasts

• Q1 2024 infrastructure costs were previously reported as $96707, but have been corrected on 10th Jul 2024 to $191919 to include additional product costs

• Ability to generate API keys
• Scaled up API infrastructure to handle higher QPS and reduce CPU load
• Deployment of write APIs for automated data ingestions

• Implemented datastore for E-Gazette Website (https://www.egazette.gov.sg/)
• Automated integration with Singstat using Singstat Table Builder APIs
• Upgraded integration infrastructure with NEA to message queues

• Launched the admin panel, Admins now able to create, edit, publish and unpublish datasets
• Admin OTP login implemented
• CSV ingestion and validation based on canonical data formats (SGDRM)
• Metadata APIs (dataset/collections listing and metadata) released

• User interface redesign, including landing page, search and dataset pages
• Migration away from CKAN data infrastructure towards modern AWS stack (DynamoDB, RDS, S3)
• Upgraded read API infrastructure
  • Deployment of backwards compatible read APIs (96% of total API traffic)
  • Testing of upgraded metadata APIs (4% of total API traffic)