User Tools

Site Tools


Menu

 ?

About

Basic concepts and guidelines

User Guides

For users

For mappers

Technical

Learn about Dokuwiki

public:using_the_rdf_endpoints_to_pull_data_from_ckan

This is an old revision of the document!


Using the API to pull/push data from/to CKAN

Who is this guide for

  • Developers willing to access data in LOD formats
  • Partners willing to integrate ODM data into their platforms

What this guide teaches

  • How to call the catalog endpoint to obtain a list of datasets
  • How to get a particular dataset in various LOD formats

Things to know in forehand

What is Linked Open Data (LOD) and what is the difference from Open Data

Quoted from linkeddata.org:

Linked Data is about using the Web to connect related data that wasn't previously linked, or using the Web to lower the barriers to linking data currently linked using other methods. More specifically, Wikipedia defines Linked Data as “a term used to describe a recommended best practice for exposing, sharing, and connecting pieces of data, information, and knowledge on the Semantic Web using URIs and RDF.”

RDF is not a format, it is a specification

The Resource Description Framework (RDF) is a family of World Wide Web Consortium (W3C) specifications originally designed as a metadata data model. It is used for describing graph-like ( nodes and vertices ) relationships between pieces of information by using RDF Triples.

An RDF triple contains three components:

- the subject, which is an RDF URI reference or a blank node. - the predicate, which is an RDF URI reference. - the object, which is an RDF URI reference, a literal or a blank node.

It is important to distinguish the RDF/XML format from the abstract RDF model itself. Although the RDF/XML format is still in use, other RDF serializations are now preferred by many RDF users, both because they are more human-friendly,[32] and because some RDF graphs are not representable in RDF/XML due to restrictions on the syntax of XML QNames.

The most common RDF serialization formats

From RDF definition on wikipedia

  • Turtle: a compact, human-friendly format.
  • N-Triples: a very simple, easy-to-parse, line-based format that is not as compact as Turtle.
  • N-Quads: a superset of N-Triples, for serializing multiple RDF graphs.
  • JSON-LD: a JSON-based serialization.
  • N3 or Notation3: a non-standard serialization that is very similar to Turtle, but has some additional features, such as the ability to define inference rules.
  • RDF/XML: an XML-based syntax that was the first standard format for serializing RDF.

This feature is based on CKAN DCAT extension

ODM's CKAN instance exposes this functionality thanks to the great job done by the contributors of https://github.com/ckan/ckanext-dcat, a CKAN extension for exposing data in RDF, and many more things.

RDF DCAT endpoints

RDF representations of a particular dataset can accessed using the following endpoint:

https://data.opendevelopmentmekong.net/dataset/{dataset-id}.{format}

The extension will determine the RDF serialization format returned. The currently supported values are:

Extension Format
xml RDF/XML
ttl Turtle
n3 Notation3
jsonld JSON-LD

The fallback rdf format defaults to RDF/XML.

Here's an example of the different formats available:

The catalog endpoint

Additionally to the individual dataset representations, there is a catalog-wide endpoint for retrieving multiple datasets at the same time (the datasets are paginated, see below for details):

https://data.opendevelopmentmekong.net/catalog.{format}?[page={page}]&[modified_date={date}]

As described previously, the extension will determine the RDF serialization format returned.

  http://data.opendevelopmentmekong.net/catalog.rdf
  http://data.opendevelopmentmekong.net/catalog.xml
  http://data.opendevelopmentmekong.net/catalog.ttl

The number of datasets returned is limited. The response will include paging info, serialized using the Hydra vocabulary. The different terms are self-explanatory, and can be used by clients to iterate the catalog:

@prefix hydra: <http://www.w3.org/ns/hydra/core#> .

<http://example.com/catalog.ttl?page=1> a hydra:PagedCollection ;
    hydra:firstPage "http://example.com/catalog.ttl?page=1" ;
    hydra:itemsPerPage 100 ;
    hydra:lastPage "http://example.com/catalog.ttl?page=3" ;
    hydra:nextPage "http://example.com/catalog.ttl?page=2" ;
    hydra:totalItems 283 .

The default number of datasets returned is 100.

The catalog endpoint also supports a modified_date parameter to restrict datasets to those modified from a certain date. The parameter value should be a valid ISO-8601 date:

http://data.opendevelopmentmekong.net/catalog.xml?modified_since=2015-07-24

Metadata terms being used

Datasets

Library records

Laws records

Visualizing ODM's datasets

TBD

To improve

Split multilingual versions using lang="CODE"

Currently, multilingual values are not exposed following RDF standards but as json object.

Use URIs for standard values such as odm_spatial_range ( gn:countryCode ) or taxonomy ( foaf:topics )

Instead of using just literals for the object of the different predicates, as is the case now, standard values (such as a country code) should be exposed via an URI.

Do not generate Empty/None triples

In case a triple has the value None, it should not be generated.

public/using_the_rdf_endpoints_to_pull_data_from_ckan.1463064213.txt.gz · Last modified: 2020/06/23 15:03 (external edit)