Data sources

Out of the box, the data starter includes the data and metadata as CSV and YAML files, respectively.

SDG Build

This data and metadata is processed by a Python library called sdg-build. This library can input data/metadata from a variety of sources, and then outputs them in the format expected by Open SDG.

Data repository config file

If you are using the out-of-the-box CSV/YAML approach mentioned above, you can take advantage of using a configuration file in your data repository. For an example of using this approach, see this example in the sdg-build documentation.

Alternative data sources

You may want to maintain your data and/or metadata in some other way. With the help of the sdg-build library, there are additional options available for Open SDG.

NOTE: If using any of these alternative data sources, the "config file" approach mentioned above will not work.

SDMX

Your data repository can contain SDMX files, either of the SDMX-ML or the SDMX-JSON format. Alternatively, your data repository can "point" to a remote SDMX API endpoint. Examples can be found in the sdg-build documentation.

UN SDG API

What is the UN SDG API?

An API stands for Application Programming Interface, which is a software intermediary that allows two applications to talk to each other.

With Open SDG, you can choose to use the UN SDG API to pull data onto your site from the United Nations (UN) Global SDG Database in the SDMX standard (Statistical Data and Metadata eXchange).

When to use the UN SDG API

  • The UN SDG API is a useful tool for those organisations that want to display their SDG data, but do not have much or any of their own data files to upload. As such, they could use the API to put the data already on the UN Global SDG Database onto their platform.

  • You can use a combination of inputting your own data files, and inputting data using the API if you need to fill some gaps, but the data will need to be structured in the same way to do so.

  • You should not use the API for data that you already have.

How to use the UN SDG API

For a basic overview of how to implement the UN SDG API on your platform, see below in conjunction with the UN SDG API manual:

You will need to build a query by following the steps in the UN SDG API manual to source the data that you want to be on your platform.

For this data to appear on your platform, you need to add some code to the config_data.yml file in your data repository. There are two ways of doing this, one using the InputSdmxMl_UnitedNationsApi class and one using the InputSdmxMl_Structure class.

See two examples below:

InputSdmxMl_UnitedNationsApi

inputs: 
  - class: InputSdmxMl_UnitedNationsApi 
    reference_area: 231 
    dimension_query: 
      REPORTING_TYPE: N 

This class handles the URL generated as the query for you, so you don’t need to obtain it using the UN data website as per the UN SDG API manual instructions. This method is simple which means you may not be able to add all the details that you want.

Note that in the example above, the REPORTING_TYPE has been specified to be ‘N’ which means it is specified to pull national data from the Database.

InputSdmxMl_Structure

inputs: 
  - class: InputSdmxMl_Structure 
    source: https://data.un.org/ws/rest/data/IAEG-SDGs,DF_SDG_GLH/...231...........?startPeriod=2000&endPeriod=2022 

If you have used this method, then you will need the URL, which you can generate as a query from the UN data website as per the manual instructions, or construct/adjust yourself using the structure detailed below. You can make more modifications with this method, such as changing the time period etc., as you will be using the API directly by building a key.

The first part of the URL should be:

https://data.un.org/ws/rest/data/IAEG-SDGs,DF_SDG_GLH/  

This is then followed by the second part, which is the key. The key needs to follow the below structure and needs to be composed of 15 parts. The 15 parts should be separated by a dot (.):

[FREQ].[REPORTING_TYPE].[SERIES].[REF_AREA].[SEX].[AGE].[URBANISATION].[INCOME_WEALTH_QUANTILE].[EDUCATION_LEV].[OCCUPATION].[CUST_BREAKDOWN].[COMPOSITE_BREAKDOWN].[DISABILITY_STATUS].[ACTIVITY].[PRODUCT]  

However, you do not need to include details for all 15 parts as shown in the code example above where 231 has been entered as the [REF_AREA] part of the key (in the fourth part between the dots (.)), but no other parts are specified. To mirror the first example using the InputSdmxMl_UnitedNationsApi to specify national data, you would need to add ‘N’ to the second part between the dots (.) where [REPORTING_TYPE] is in the key. E.g. https://data.un.org/ws/rest/data/IAEG-SDGs,DF_SDG_GLH/.N..231...........?startPeriod=2000&endPeriod=2022

This example also demonstrates adding a custom time period of 2000-2022, which is achieved by adding the following details after the key:

?startPeriod=yyyy&endPeriod=yyyy 

To know what to populate the 15 parts of the key with, refer to the SDG Data Structure Definition (DSD) here.

CKAN

Your data repository can point to a CKAN instance containing data in a compatible format. Examples can be found in the sdg-build documentation.