This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
public:using_the_rdf_endpoints_to_pull_data_from_ckan [2016/05/04 14:26] acorbi |
public:using_the_rdf_endpoints_to_pull_data_from_ckan [2020/01/14 16:12] admin Added fields from https://docs.google.com/spreadsheets/d/1KbUbH4KOgqtkcR5AGpbcl362GeR3zt2MX8AcL-bE_2g/edit#gid=230201128 |
||
---|---|---|---|
Line 3: | Line 3: | ||
===== Who is this guide for ===== | ===== Who is this guide for ===== | ||
- | * Developers willing to access data in OLD formats | + | * Developers willing to access data in LOD formats |
* Partners willing to integrate ODM data into their platforms | * Partners willing to integrate ODM data into their platforms | ||
Line 9: | Line 9: | ||
* How to call the catalog endpoint to obtain a list of datasets | * How to call the catalog endpoint to obtain a list of datasets | ||
- | * How to get a particular dataset in various OLD formats | + | * How to get a particular dataset in various LOD formats |
===== Things to know in forehand ===== | ===== Things to know in forehand ===== | ||
- | ==== What is Open Linked Data (OLD) and what is the difference from Open Data ==== | + | ==== What is Linked Open Data (LOD) and what is the difference from Open Data ==== |
Quoted from [[http://linkeddata.org|linkeddata.org]]: | Quoted from [[http://linkeddata.org|linkeddata.org]]: | ||
Line 24: | Line 24: | ||
An RDF triple contains three components: | An RDF triple contains three components: | ||
- | - the subject, which is an RDF URI reference or a blank node. | + | - the **subject**, which is an RDF URI reference or a blank node. |
- | - the predicate, which is an RDF URI reference. | + | - the **predicate**, which is an RDF URI reference. |
- | - the object, which is an RDF URI reference, a literal or a blank node. | + | - the **object**, which is an RDF URI reference, a literal or a blank node. |
It is important to distinguish the RDF/XML format from the abstract RDF model itself. Although the RDF/XML format is still in use, other RDF serializations are now preferred by many RDF users, both because they are more human-friendly,[32] and because some RDF graphs are not representable in RDF/XML due to restrictions on the syntax of XML QNames. | It is important to distinguish the RDF/XML format from the abstract RDF model itself. Although the RDF/XML format is still in use, other RDF serializations are now preferred by many RDF users, both because they are more human-friendly,[32] and because some RDF graphs are not representable in RDF/XML due to restrictions on the syntax of XML QNames. | ||
Line 44: | Line 44: | ||
ODM's CKAN instance exposes this functionality thanks to the great job done by the contributors of https://github.com/ckan/ckanext-dcat, a CKAN extension for exposing data in RDF, and many more things. | ODM's CKAN instance exposes this functionality thanks to the great job done by the contributors of https://github.com/ckan/ckanext-dcat, a CKAN extension for exposing data in RDF, and many more things. | ||
+ | |||
+ | ===== RDF DCAT endpoints ===== | ||
+ | |||
+ | RDF representations of a particular dataset can accessed using the following endpoint: | ||
+ | |||
+ | <code>https://data.opendevelopmentmekong.net/dataset/{dataset-id}.{format}</code> | ||
+ | |||
+ | The extension will determine the RDF serialization format returned. The currently supported values are: | ||
+ | |||
+ | ^ Extension ^ Format ^ Media Type | ||
+ | | xml | RDF/XML | application/rdf+xml | ||
+ | | ttl | Turtle | text/turtle | ||
+ | | n3 | Notation3 | text/n3 | ||
+ | | jsonld | JSON-LD | application/ld+json | ||
+ | |||
+ | |||
+ | The fallback rdf format defaults to RDF/XML. | ||
+ | |||
+ | Here's an example of the different formats available: | ||
+ | |||
+ | * https://data.opendevelopmentmekong.net/dataset/economic-land-concessions.rdf | ||
+ | * https://data.opendevelopmentmekong.net/dataset/economic-land-concessions.ttl | ||
+ | * https://data.opendevelopmentmekong.net/dataset/economic-land-concessions.n3 | ||
+ | * https://data.opendevelopmentmekong.net/dataset/economic-land-concessions.jsonld | ||
===== The catalog endpoint ===== | ===== The catalog endpoint ===== | ||
- | ===== Endpoint for a particular dataset ===== | + | Additionally to the individual dataset representations, there is a catalog-wide endpoint for retrieving multiple datasets at the same time (the datasets are paginated, see below for details): |
+ | |||
+ | <code>https://data.opendevelopmentmekong.net/catalog.{format}?[page={page}]&[modified_date={date}]</code> | ||
+ | |||
+ | As described previously, the extension will determine the RDF serialization format returned. | ||
+ | |||
+ | http://data.opendevelopmentmekong.net/catalog.rdf | ||
+ | http://data.opendevelopmentmekong.net/catalog.xml | ||
+ | http://data.opendevelopmentmekong.net/catalog.ttl | ||
+ | |||
+ | The number of datasets returned is limited. The response will include paging info, serialized using the Hydra vocabulary. The different terms are self-explanatory, and can be used by clients to iterate the catalog: | ||
+ | |||
+ | <code> | ||
+ | @prefix hydra: <http://www.w3.org/ns/hydra/core#> . | ||
+ | |||
+ | <http://example.com/catalog.ttl?page=1> a hydra:PagedCollection ; | ||
+ | hydra:firstPage "http://example.com/catalog.ttl?page=1" ; | ||
+ | hydra:itemsPerPage 100 ; | ||
+ | hydra:lastPage "http://example.com/catalog.ttl?page=3" ; | ||
+ | hydra:nextPage "http://example.com/catalog.ttl?page=2" ; | ||
+ | hydra:totalItems 283 . | ||
+ | </code> | ||
+ | |||
+ | The default number of datasets returned is 100. | ||
+ | |||
+ | The catalog endpoint also supports a modified_date parameter to restrict datasets to those modified from a certain date. The parameter value should be a valid ISO-8601 date: | ||
+ | |||
+ | http://data.opendevelopmentmekong.net/catalog.xml?modified_since=2015-07-24 | ||
+ | |||
+ | ===== Metadata terms being used ===== | ||
+ | |||
+ | ==== Datasets ==== | ||
+ | |||
+ | ^ Metadata field name ^ ODM Ckan internal id ^ Standard term ^ Standard term uri ^ | ||
+ | | Title | title_translated | dcterms:title | http://purl.org/dc/terms/title | | ||
+ | | Description | notes_translated | dcterms:description | http://purl.org/dc/terms/description | | ||
+ | | Topics | taxonomy | foaf:topic | http://xmlns.com/foaf/0.1/topic | | ||
+ | | License | odm_license | dcterms:licence | http://purl.org/dc/terms/licence | | ||
+ | | Copyright | odm_copyright | cro:Copyright | http://rhizomik.net/ontologies/copyrightonto.owl#copyright | | ||
+ | | Access and use constraints | odm_access_and_use_constraints | md:useconstraints | http://def.seegrid.csiro.au/isotc211/iso19115/2003/metadata#useConstraints | | ||
+ | | Organization | owner_org | foaf:Organization | http://xmlns.com/foaf/0.1/organization| | ||
+ | | Version | version | doap:version | http://usefulinc.com/ns/doap#version | | ||
+ | | Contact | odm_contact | ebucore:Contact | https://www.ebu.ch/metadata/ontologies/ebucore/index.html#Contact | | ||
+ | | Language | odm_language | dcterms:language | http://dublincore.org/documents/2012/06/14/dcmi-terms/?v=terms#language | | ||
+ | | Date created | odm_date_created | dcterms:created | http://dublincore.org/documents/2012/06/14/dcmi-terms/?v=terms#created | | ||
+ | | Date uploaded | odm_date_uploaded | schema:uploadDate | http://schema.org/uploadDate | | ||
+ | | Date modified | odm_date_modified | dcterms:modified | http://dublincore.org/documents/2012/06/14/dcmi-terms/?v=terms#modified | | ||
+ | | Geographical area (Spatial range) | odm_spatial_range | gn:countryCode | http://www.geonames.org/ontology#countryCode | | ||
+ | | Accuracy | odm_accuracy | dqm:accuracy | http://semwebquality.org/dqm-vocabulary/v1/dqm#Accuracy | | ||
+ | | Logical consistency | odm_logical_consistency | dq:LogicalConsistency | http://def.seegrid.csiro.au/isotc211/iso19115/2003/dataquality#LogicalConsistency | | ||
+ | | Completeness | odm_completeness | dq:Completeness | http://def.seegrid.csiro.au/isotc211/iso19115/2003/dataquality#Completeness | | ||
+ | | Source(s) | odm_source | dcterms:source | http://dublincore.org/documents/2012/06/14/dcmi-terms/?v=terms#source | | ||
+ | | Attributes | odm_attributes | omn:Attribute | http://open-multinet.info/ontology/omn#isAttributeOf | | ||
+ | ==== Library records ==== | ||
+ | |||
+ | ^ Metadata field name ^ ODM Ckan internal id ^ Standard term ^ Standard term uri ^ | ||
+ | | Document type | document_type | agls:documentType | http://www.agls.gov.au/agls/terms/documentType | | ||
+ | | Language of document | odm_language | dcterms:language | http://dublincore.org/documents/2012/06/14/dcmi-terms/?v=terms#language | | ||
+ | | Formal full title | title_translated | dcterms:title | http://purl.org/dc/terms/title | | ||
+ | | Short title (alternative/varying form of title) | marc21_246 | gc:shortTitle | http://www.oegov.org/core/owl/gc#shortTitle | | ||
+ | | Topics | taxonomy | foaf:topic | http://xmlns.com/foaf/0.1/topic | | ||
+ | | Short summary (contents) | notes_translated | dcterms:description | http://dublincore.org/documents/2012/06/14/dcmi-terms/?v=terms#description | | ||
+ | | Geographic area (spatial range) | odm_spatial_range | gn:countryCode | http://www.geonames.org/ontology#countryCode | | ||
+ | | Copyright | odm_copyright | cro:Copyright | http://rhizomik.net/ontologies/copyrightonto.owl#copyright | | ||
+ | | Access and use constraints | odm_access_and_use_constraints | md:useconstraints | http://def.seegrid.csiro.au/isotc211/iso19115/2003/metadata#useConstraints | | ||
+ | | Version / Edition | version | doap:version | http://usefulinc.com/ns/doap#version | | ||
+ | | Organization | owner_org | foaf:Organization | http://xmlns.com/foaf/0.1/organization| | ||
+ | | Date uploaded | odm_date_uploaded | schema:uploadDate | http://schema.org/uploadDate | | ||
+ | | License | license_id | dcterms:licence | http://purl.org/dc/terms/licence | | ||
+ | | Contact | odm_contact | ebucore:Contact | https://www.ebu.ch/metadata/ontologies/ebucore/index.html#Contact | | ||
+ | | Author (individual) | marc21_100 | opus:author | http://lsdis.cs.uga.edu/projects/semdis/opus#author | | ||
+ | | Author (corporate) | marc21_110 | opus:author | http://lsdis.cs.uga.edu/projects/semdis/opus#author | | ||
+ | | Co-author (individual) | marc21_700 | opus:coauthor | http://lsdis.cs.uga.edu/projects/semdis/opus#coauthor | | ||
+ | | Co-Author (corporate) | marc21_710 | opus:coauthor | http://lsdis.cs.uga.edu/projects/semdis/opus#coauthor | | ||
+ | | ISBN number | marc21_020 | opus:isbn | http://lsdis.cs.uga.edu/projects/semdis/opus#isbn | ||
+ | | ISSN number | marc21_022 | dbpedia-owl:issn | http://dbpedia.org/ontology/issn | | ||
+ | | Publication place | marc21_260a | mrel:pup | http://id.loc.gov/vocabulary/relators/pup | | ||
+ | | Publisher | marc21_260b | dcterms:publisher | http://purl.org/dc/terms/publisher | | ||
+ | | Publication date | marc21_260c | dcterms:issued | http://purl.org/dc/terms/issued | | ||
+ | | Pagination | marc21_300 | bibo:numPages | http://bibliontology.com/bibo/bibo.php#numPages| | ||
+ | | General note | marc21_500 | skos:note | https://www.w3.org/2009/08/skos-reference/skos.html#note | | ||
+ | | Legacy reference document | odm_reference_document | pproc:documentReference | http://contsem.unizar.es/def/sector-publico/pproc.html#documentReference | | ||
+ | | Province(s) | odm_province | | | | ||
+ | | Date modified | odm_date_modified | | | | ||
+ | | Keywords | odm_keywords | | | | ||
+ | ==== Laws records ==== | ||
+ | |||
+ | ^ Metadata field name ^ ODM Ckan internal id ^ Standard term ^ Standard term uri ^ | ||
+ | | Document type | odm_document_type | agls:documentType | http://www.agls.gov.au/agls/terms/documentType | | ||
+ | | Document reference number | odm_document_number | dbpedia-owl:documentNumber | http://dbpedia.org/ontology/documentNumber | | ||
+ | | Language | odm_language | dcterms:language | http://dublincore.org/documents/2012/06/14/dcmi-terms/?v=terms#language | | ||
+ | | Formal full title | title_translated | dcterms:title | http://purl.org/dc/terms/title | | ||
+ | | Alternative/short title | odm_short_title | gc:shortTitle | http://www.oegov.org/core/owl/gc#shortTitle | | ||
+ | | Topics | taxonomy | foaf:topic | http://xmlns.com/foaf/0.1/topic | | ||
+ | | Short summary | notes_translated | dcterms:description | http://purl.org/dc/terms/description | | ||
+ | | Geographic area (spatial range) | odm_spatial_range | gn:countryCode | http://www.geonames.org/ontology#countryCode | | ||
+ | | Organization | owner_org | foaf:Organization | http://xmlns.com/foaf/0.1/organization| | ||
+ | | License | license_id | dcterms:licence | http://purl.org/dc/terms/licence | | ||
+ | | Copyright | odm_copyright | cro:Copyright | http://rhizomik.net/ontologies/copyrightonto.owl#copyright | | ||
+ | | Adoption date/Promulgation date/Signing | odm_promulgation_date | bibframe:legalDate | http://id.loc.gov/ontologies/bibframe.html#p_legalDate | | ||
+ | | Short notes of change | odm_laws_previous_changes_notes | skos:changeNote | https://www.w3.org/2009/08/skos-reference/skos.html#changeNote | | ||
+ | | Contact | odm_contact | ebucore:Contact | https://www.ebu.ch/metadata/ontologies/ebucore/index.html#Contact | | ||
+ | | Notes | odm_laws_notes | skos:note | https://www.w3.org/2009/08/skos-reference/skos.html#note | | ||
+ | | Legacy reference document | odm_reference_document | pproc:documentReference | http://contsem.unizar.es/def/sector-publico/pproc.html#documentReference | | ||
+ | | Province(s) | odm_province | | | | ||
+ | | Date modified | odm_date_modified | | | | ||
+ | | Date updated | odm_date_updated | | | | ||
+ | | Keywords | odm_keywords | | | | ||
+ | | Issuing agency/parties | odm_laws_issuing_agency_parties | | | | ||
+ | | Implementing agencies | odm_laws_implementing_agencies | | | | ||
+ | | | odm_laws_primary_policy_reference_point | | | | ||
+ | | | odm_access_and_use_constraints | | | | ||
+ | | Status | odm_laws_status | | | | ||
+ | | Version date (of draft) | odm_laws_version_date | | | | ||
+ | | Effective/Enforced Date | odm_effective_date | | | | ||
+ | | Previous legal document | odm_laws_previous_legal_document | | | | ||
+ | | Parent document | odm_laws_parent_document | | | | ||
+ | | Child Document | odm_laws_child_document | | | | ||
+ | | Other reference or supporting documents | odm_laws_other_references | | | | ||
+ | | Publication reference | odm_laws_official_publication_reference | | | | ||
+ | | Links to source | odm_laws_source | | | | ||
+ | ===== Visualizing ODM's datasets ===== | ||
+ | |||
+ | TBD | ||
+ | |||
+ | ===== To improve ===== | ||
+ | |||
+ | ==== Split multilingual versions using lang="CODE" ==== | ||
+ | |||
+ | Currently, multilingual values are not exposed following RDF standards but as json object. | ||
+ | |||
+ | ==== Use URIs for standard values such as odm_spatial_range ( gn:countryCode ) or taxonomy ( foaf:topics ) ==== | ||
+ | |||
+ | Instead of using just literals for the object of the different predicates, as is the case now, standard values (such as a country code) should be exposed via an URI. | ||
+ | |||
+ | ==== Do not generate Empty/None triples ==== | ||
+ | |||
+ | In case a triple has the value **None**, it should not be generated. | ||
+ | ==== Implement mapping between ODM taxonomy terms and terms in standard vocabularies (LandVoc, etc...) ==== | ||
+ | In order to expose topics using standard terms |