User Tools

Site Tools


public:metadata

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
public:metadata [2017/11/16 10:02]
penhleak [What this guide teaches]
public:metadata [2020/10/09 02:55] (current)
mchung [Guidelines for creating accurate metadata]
Line 8: Line 8:
 ==== What this guide teaches ==== ==== What this guide teaches ====
  
-  * What is medata data, and why it is important?+  * What is metadata, and why it is important?
   * Guidelines for creating an accurate metadata   * Guidelines for creating an accurate metadata
   * Open Development metadata templates ​   * Open Development metadata templates ​
Line 18: Line 18:
 Without having to tear the wrapping papers and open the box, a label with written note attached can tell you what they are going to get.  Without having to tear the wrapping papers and open the box, a label with written note attached can tell you what they are going to get. 
  
-Similarly, ​for CKAN purposes, data is published in units called “datasets”. A dataset is a parcel of data - for example, it could be the crime statistics for a region, the spending figures for a government department, or temperature readings from various weather stations or a reference document. ​+A carefully wrapped present with out a label might add excitement ​for the gift recipient, but data without a metadata is not usable. Data is simply number and figures. Data doesn’t mean anything without a description,​ a metadata. ​  
 + 
 +For CKAN purposes, data is published in units called “datasets”. A dataset is a parcel of data - for example, it could be the crime statistics for a region, the spending figures for a government department, or temperature readings from various weather stations or a reference document. ​
  
 A dataset contains two things: A dataset contains two things:
Line 28: Line 30:
 Example: [[https://​opendevelopmentmekong.net/​dataset/?​id=hydro-basins-level-6-greater-mekong-subregion-laos-myanmar-thailand-vietnam-cambodia&​search_query=P3M9aHlkcm9iYXNpbiZ0eXBlPWRhdGFzZXQmcGFnZT0w|Hydrobasins level 6 dataset on OD Mekong]] Example: [[https://​opendevelopmentmekong.net/​dataset/?​id=hydro-basins-level-6-greater-mekong-subregion-laos-myanmar-thailand-vietnam-cambodia&​search_query=P3M9aHlkcm9iYXNpbiZ0eXBlPWRhdGFzZXQmcGFnZT0w|Hydrobasins level 6 dataset on OD Mekong]]
  
-{{ :public:​hydrobasin_data.png?​nolink&​600 |}}+{{ :hydro_basin_level_6_mekong.png?​nolink&​600 |}}
  
 Metadata provides important context about an informational asset’s source and manner of creation, as well as in what applications or environments the asset is relevant. ​ Metadata provides important context about an informational asset’s source and manner of creation, as well as in what applications or environments the asset is relevant. ​
Line 44: Line 46:
 **How may we create an accurate and useful metadata when the information it is describing might be flawed?** **How may we create an accurate and useful metadata when the information it is describing might be flawed?**
  
-We aim to produce, to the best of our ability, an accurate metadata by describing the extent of our knowledge about asset/​resource. ​It should clearly state what is known about the resource and what is not known or problematic. Metadata changes when the asset itself or knowledge about its condition changes+We aim to produce, to the best of our ability, an accurate metadata by describing the extent of our knowledge about the asset/​resource. ​A good metadata ​should clearly state what is known about the resource and what is not known or problematic. Metadata changes when the asset itself or knowledge about its condition changes.
- +
-If information is missing or inconsistent,​ describe the known inconsistencies or gaps instead of disregarding the resource. Mention any steps being taken to address these issues, along with an expected timeline.+
  
 +If information is missing or inconsistent,​ describe the known inconsistencies or gaps instead of disregarding the resource. Mention any steps being taken to address these issues, along with an expected timeline. **[THIS INSTRUCTION IS UNDER REVIEW]
 +**
 ==== Open Development Platform metadata templates ==== ==== Open Development Platform metadata templates ====
    
-On the Datahub 4 different types of datasets, each requires its own metadata template, are currently stored/​administered:​+For each different type of data, there are specific terms that relate to that type of data. On the Datahub 4 different types of datasets, each requires its own metadata template, are currently stored/​administered:​
  
   * Dataset (both spatial and non-spatial)   * Dataset (both spatial and non-spatial)
Line 57: Line 59:
   * Agreement records (for contracts)   * Agreement records (for contracts)
  
-These metadata templates ​are developed by adapting and enhancing the standard CKAN's metadata template. Each template contains metadata fields common for all dataset types on CKAN and a set of fields that are only applicable to the dataset type. For example, metadata about a research report (Library records) will have information about author(s) and publishers (s); metadata for laws and policies (Law records) will instead have information about the drafting agency, issuing agency, and promulgation date etc.+These metadata templates ​were developed by adapting and enhancing the standard CKAN's metadata template. Each template contains metadata fields common for all dataset types on CKAN and a set of fields that are only applicable to the dataset type. For example, metadata about a research report (Library records) will have information about author(s) and publishers (s); metadata for laws and policies (Law records) will instead have information about the drafting agency, issuing agency, and promulgation date etc.
  
 The templates below outline information that should be included and offer instruction for each metadata field. The templates below outline information that should be included and offer instruction for each metadata field.
  
-==== Metadata for dataset (both spatial and non-spatial) ==== +==== Metadata for dataset (both spatial and non-spatial) ==== 
- +
-<WRAP center round info 90%> +
-  * Fields marked with * are mandatory +
-  * Fields marked with [ML] are multilingual +
-  * Fields marked with [M] can have more than 1 value +
-  * Fields with contents being <​del>​striked through</​del>​ are marked for removal +
-</​WRAP>​+
  
-^ **Label** ^ **Fieldname** ^ **Definition and guideline** ​ ^ +[[public:geospatial_metadata|public:geospatial_metadata]]
-| Title * [ML] | title_translated | Name given to the dataset. ​ | +
-| Description | notes_translated | Short description explaining the content and its origins. ​ | +
-| Topics * | taxonomy | e.g. economy, mental health, government. See [[https://​wiki.opendevelopmentmekong.net/​partners:keywords_and_taxonomic_tagging_guidelines#​guideline_to_determining_odm_taxonomic_terms| Taxonomic tagging guide]]  | +
-| License * | license_id | License definitions and additional information can be found at http://​opendefinition.org/ ​ | +
-| Copyright * | odm_copyright | Select '​Yes',​ '​No',​ '​Unclear copyright'​ or 'To be determined'​ about the copyright of the dataset. If copyright of any type is present, describe further in Access and User Constraints. ​ | +
-| Access and use constraints | odm_access_and_use_constraints | A few sentences describing legal considerations for people who access the website and/or use its contents. ​ | +
-| Organization * | owner_org |   | +
-| Version * | version | Dataset'​s version (eg. 1.0)  | +
-| Contact | odm_contact | Contact information for the individual or organization that is responsible for or most knowledgeable about the dataset. This could be the author of a report, the contact information for the relevant department of an organization that produced a report, or the data analyst, mapper or researcher that produced a dataset or report. ​ | +
-| Language * | odm_language | Language(s) of the dataset, including resources within dataset. ​ | +
-| Date created * | odm_date_created | Date the dataset was first Published by its creator. ​ | +
-| Date uploaded | odm_date_uploaded | Date a new version or update of the dataset was uploaded. ​ | +
-| Date modified | odm_date_modified | Date a new version or update of the dataset was uploaded. ​ | +
-| Temporal range | odm_temporal_range | The period of time for which the dataset is relevant (i.e. 2011-01-01:​2011-12-31). ​ | +
-| Spatial data | spatial | A valid GEOJSON string describing the dataset boundaries ​ | +
-| Geographic area (spatial range) * | odm_spatial_range | The geographic area that the dataset is relevant to (i.e. Cambodia, Laos). ​ | +
-| Province(s) | odm_province | The province(s) this dataset relates to  | +
-| Accuracy | odm_accuracy | Details on the level of accuracy of the dataset and any existing issues. ​ | +
-| Logical Consistency | odm_logical_consistency | Issues with logical consistency in the dataset and the steps, if any, being taken to validate its content. ​ | +
-| Completeness | odm_completeness | Brief description of the level of completeness of the dataset'​s contents and the steps, if any, being taken to make the dataset more complete. ​ | +
-| Process(s) * | odm_process | The steps taken to acquire, aggregate, or transform any of the resources in the dataset. ​ | +
-| Source(s) * | odm_source | Ordered citations for all information sources that went into producing the dataset. ​ | +
-| Metadata Reference Information | odm_metadata_reference_information | Information about how up-to-date the metadata is and who is responsible for maintaining it.  | +
-| Attributes | odm_attributes | Details about the information content of the dataset. ​ | +
-| Legacy reference document | odm_reference_document | For internal use only.  | +
-| Database table? | odm_db_table | INTERNAL USE ONLY: Select true if this record contains CSV and/or XLS resources available in the datastore. ​ | +
-| Keywords | odm_keywords | INTERNAL USE ONLY: Enter keywords for improving the discoverability of this record via search ​ |+
  
   
 ==== Metadata template for library publications ==== ==== Metadata template for library publications ====
  
-<WRAP center round info 90%> +[[public:​library_metadata|public:​library_metadata]]
-  * Fields marked with * are mandatory +
-  * Fields marked with [ML] are multilingual +
-  * Fields marked with [Mcan have more than 1 value +
-  * Fields with contents being <​del>​striked through</​del>​ are marked for removal +
-</​WRAP>​+
  
-^ **Label** ^ **Fieldname** ^ **Definition and guideline** ​ ^ 
-| Document type * | document_type | Select pre-defined OD document types from the drop-down list.  | 
-| Language of document * | odm_language | Language(s) of the dataset, including resources within dataset. ​ | 
-| Formal full title [ML] | title_translated | Main title  | 
-| Short title (alternative/​varying form of title) [ML] | marc21_246 | Parallel title or translation. ​ | 
-| Topics * | taxonomy | e.g. economy, mental health, government. See [[https://​wiki.opendevelopmentmekong.net/​partners:​keywords_and_taxonomic_tagging_guidelines#​guideline_to_determining_odm_taxonomic_terms| Taxonomic tagging guide]] ​ | 
-| Short summary (contents) [ML] | notes_translated | Abstract or summary of book or articlee ​ | 
-| Geographic area (spatial range) * | odm_spatial_range | The geographic area that the dataset is relevant to (i.e. Cambodia, Laos). ​ | 
-| Province(s) | odm_province | The province(s) this dataset relates to  | 
-| Copyright * | odm_copyright | Select '​Yes',​ '​No',​ '​Unclear copyright'​ or 'To be determined'​ about the copyright of the dataset. If copyright of any type is present, describe further in Access and User Constraints. ​ | 
-| Access and use constraints | odm_access_and_use_constraints | A few sentences describing legal considerations for people who access the website and/or use its contents. ​ | 
-| Version / Edition * | version | Version of publication ​ | 
-| Organization * | owner_org |   | 
-| Date uploaded | odm_date_uploaded | Date a new version or update of the dataset was uploaded. ​ | 
-| Date modified | odm_date_modified | Date a new version or update of the dataset was uploaded. ​ | 
-| License * | license_id | License definitions and additional information can be found at http://​opendefinition.org/ ​ | 
-| Contact [ML] | odm_contact | Contact information for the individual or organization that is responsible for or most knowledgeable about the dataset. This could be the author of a report, the contact information for the relevant department of an organization that produced a report, or the data analyst, mapper or researcher that produced a dataset or report. ​ | 
-| Author (individual) | marc21_100 | Main Entry-Personal Name (author). ​ | 
-| Author (corporate) | marc21_110 | Main Entry-Corporate Name (corporate author) or title of journal. ​ | 
-| Co-author (individual) | marc21_700 | Personal Name (co-author),​ more than one author. ​ | 
-| Co-author (coorporate) | marc21_710 | Corporate Name, more than one Corporate. ​ | 
-| ISBN number | marc21_020 | 13-digit unique number that uniquely identify a commercial book  | 
-| ISSN number | marc21_022 | 8-digit number that uniquely identify a serial publication ​ | 
-| Publication place [ML] | marc21_260a | Place of publisher ​ | 
-| Publisher [ML] | marc21_260b | Name of publishing organization ​ | 
-| Publication date | marc21_260c | Date published (YYYY) ​ | 
-| Pagination [ML] | marc21_300 | Physical description (pagination) ​ | 
-| General note [ML] | marc21_500 | General note.  | 
-| Legacy reference document | odm_reference_document | For internal use only.  | 
-| Keywords | odm_keywords | INTERNAL USE ONLY: Enter keywords for improving the discoverability of this record via search ​ | 
  
 ==== Metadata template for law and policy documents ==== ==== Metadata template for law and policy documents ====
  
-<WRAP center round info 90%> +[[public:​laws_metadata|public:​laws_metadata]]
-  * Fields marked with * are mandatory +
-  * Fields marked with [ML] are multilingual +
-  * Fields marked with [Mcan have more than 1 value +
-  * Fields with contents being <​del>​striked through</​del>​ are marked for removal +
-</​WRAP>​+
  
-^ **Label** ​ ^ **Fieldname** ​ ^ **Definition and guideline** ​ ^ 
-| Geographic area (spatial range)* | odm_spatial_range | The geographic area that the dataset is relevant to (i.e. Cambodia, Laos). ​ | 
-| Province(s) | odm_province | The province(s) this dataset relates to  | 
-| Document reference # [ML] | odm_document_number | The legal reference document number as used by the internal governing agency. ​ | 
-| Issuing agency/​parties * | odm_laws_issuing_agency_parties | The jurisdictional agency responsible for drafting and issuing the (law) legal document. ​ | 
-| Implementing agencies | odm_laws_implementing_agencies | The jurisdictional agency responsible for the enforcing and implementing the (law) legal document. ​ | 
-| Language [ML] | odm_language | Language(s) of the dataset, including resources within dataset. ​ | 
-| Formal full title [ML] | title_translated | Full title of document. Please do not repeat the document type or number in this field. ​ | 
-| Formal type of document | odm_document_type | The type of document this is.  | 
-| Alternative/​short title [ML] | odm_short_title | Commonly used label, e.g. Cambodia Labor Law.  | 
-| Topics * | taxonomy | e.g. economy, mental health, government. See [[https://​wiki.opendevelopmentmekong.net/​partners:​keywords_and_taxonomic_tagging_guidelines#​guideline_to_determining_odm_taxonomic_terms| Taxonomic tagging guide]] ​ | 
-| Short summary [ML] | notes_translated | Describe general purpose and scope, preamble will often provide a useful statement of objective. ​ | 
-| Primary policy reference point [ML] | odm_laws_primary_policy_reference_point | References are generally in the preamble or opening sections of a legal authority as the legitimacy of the document is derived from the source it references. ​ | 
-| Organization * | owner_org |   | 
-| License * | license_id | License definitions and additional information can be found at http://​opendefinition.org/ ​ | 
-| Copyright * | odm_copyright | Select '​Yes',​ '​No',​ '​Unclear copyright'​ or 'To be determined'​ about the copyright of the dataset. If copyright of any type is present, describe further in Access and User Constraints. ​ | 
-| Access and use constraints [ML] | odm_access_and_use_constraints | A few sentences describing legal considerations for people who access the website and/or use its contents. ​ | 
-| Status * | odm_laws_status | Current operational state of the legal document. ​ | 
-| Version date (of draft) | odm_laws_version_date | The version date of when the law was drafted. ​ | 
-| Adoption date/​Enacted/​Promulgation date/​Signing date * | odm_promulgation_date | The date the law was officially authorised. ​ | 
-| Effective/​Enforced date * | odm_effective_date | Date the law is to take effect. ​ | 
-| Previous legal document | odm_laws_previous_legal_document | Does this law replace, amend or supplement previous law?  | 
-| Short notes of change [ML] | odm_laws_previous_changes_notes | A short statement describing what changed. ​ | 
-| Parent document | odm_laws_parent_document | The law that directly supersedes this law  | 
-| Child Document | odm_laws_child_document | The law that directly precedes this law  | 
-| Other reference or supporting documents | odm_laws_other_references | Any other supporting documents or references that relate to this law; i.e. reports, policy briefs etc.  | 
-| Publication reference * | odm_laws_official_publication_reference | The official gazette or other official promulgation of policy, referencing issue #, date and page  | 
-| Links to source | odm_laws_source | Official URLs where the document is made available. ​ | 
-| Contact [ML] | odm_contact | Contact information for the individual or organization that is responsible for or most knowledgeable about the document. ​ | 
-| Notes [ML] | odm_laws_notes | Any additional notes regarding this document. ​ | 
-| Legacy reference document | odm_reference_document | For internal use only.  | 
-| Keywords | odm_keywords | INTERNAL USE ONLY: Enter keywords for improving the discoverability of this record via search ​ | 
  
 ==== Metadata for agreement documents (contracts) ==== ==== Metadata for agreement documents (contracts) ====
  
-<WRAP center round info 90%> +[[public:​agreement_metadata|public:​agreement_metadata]]
-  * Fields marked with * are mandatory +
-  * Fields marked with [ML] are multilingual +
-  * Fields marked with [Mcan have more than 1 value +
-  * Fields with contents being <​del>​striked through</​del>​ are marked for removal +
-</​WRAP>​+
  
-^ **Label** ^ **Fieldname** ^ **Definition and guideline** ​ ^ 
-| Contract name [ML] | title_translated | Main title  | 
-| Objective [ML] | notes_translated | Abstract or summary of book or articlee ​ | 
-| Geographic area (spatial range) * | odm_spatial_range | The geographic area that the dataset is relevant to (i.e. Cambodia, Laos). ​ | 
-| Topics * | taxonomy | e.g. economy, mental health, government. [[https://​wiki.opendevelopmentmekong.net/​partners:​keywords_and_taxonomic_tagging_guidelines#​guideline_to_determining_odm_taxonomic_terms| Taxonomic tagging guide]] ​ | 
-| Document type * | odm_agreement_document_type |   | 
-| Participating share | odm_agreement_participating_share | Comma separated values with following format: company1, %; company2, %; company3, %;  | 
-| Contracting Parties * | odm_agreement_contracting_parties |   | 
-| Government Entity (multiple) | odm_agreement_government_entity | The jurisdictional agency responsible for the enforcing and implementing the (law) legal document. ​ | 
-| Concession/​License name | odm_agreement_concession_name | The concession or license name, is meant for the name of the license area or block which is covered by the contract. ​ | 
-| Disclosure Mode | odm_agreement_disclosure_mode |   | 
-| Document reference no. | odm_agreement_document_reference_number | Unknown default if not listed on the document. ​ | 
-| LandMatrix deal No. | odm_agreement_landmatrix_no | Look up land matrix number. ​ | 
-| License * | license_id | License definitions and additional information can be found at http://​opendefinition.org/ ​ | 
-| Copyright * | odm_copyright | Select '​Yes',​ '​No',​ '​Unclear copyright'​ or 'To be determined'​ about the copyright of the dataset. If copyright of any type is present, describe further in Access and User Constraints. ​ | 
-| Access and use constraints | odm_access_and_use_constraints | A few sentences describing legal considerations for people who access the website and/or use its contents. ​ | 
-| Organization * | owner_org |   | 
-| Version / Edition * | version | Version of publication ​ | 
-| Language of document * | odm_language | Language(s) of the dataset, including resources within dataset. ​ | 
-| Date created | odm_date_created | Date a new version or update of the dataset was created. ​ | 
-| Date uploaded | odm_date_uploaded | Date a new version or update of the dataset was uploaded. ​ | 
-| Date modified | odm_date_modified | Date a new version or update of the dataset was uploaded. ​ | 
-| Province(s) | odm_province | The province(s) this dataset relates to  | 
-| Metadata Reference Information | odm_metadata_reference_information | Information about how up-to-date the metadata is and who is responsible for maintaining it.  | 
-| Amendment to agreement | odm_agreement_amendment_to_contract |   | 
-| Signature date | odm_agreement_signature_date | Date a new version or update of the dataset was uploaded. ​ | 
-| Short notes of change | odm_agreement_short_notes_of_change | Only if the document is an amendent or annex  | 
-| Links to source | odm_agreement_source | Official URLs where the document is made available. ​ | 
-| Contact | odm_contact | Contact information for the individual or organization that is responsible for or most knowledgeable about the dataset. This could be the author of a report, the contact information for the relevant department of an organization that produced a report, or the data analyst, mapper or researcher that produced a dataset or report. ​ | 
-| Notes | odm_agreement_notes |   | 
-| Granted area | odm_agreement_granted_area | in hectares ​ | 
-| Contract term | odm_agreement_contract_term | Period of time, please specify unit below. ​ | 
-| Contract term unit | odm_agreement_contract_term_unit |   | 
-| Payment for concession fees | odm_agreement_payment_for_concession_fees | Comma separated values with following format: Year 1, $5; Year 2, $7; Year 3, $9  | 
-| Payment for concession fees unit | odm_agreement_payment_for_concession_fees_unit |   | 
-| Guaranty deposit for contract implementation ($) | odm_agreement_guaranty_deposit_for_contract_implementation | in USD  | 
-| Land use planning | odm_agreement_land_use_planning | Comma separated values with following format: Year 1, 1000; Year 2, 2000; Year 3, 3000  | 
-| Land use planning unit | odm_agreement_land_use_planning_unit |   | 
-| Party'​s obligation in the implementation of the project | odm_agreement_parties_obligations | Short summary of the general obligations for the private company ​ | 
-| Job creation summary | odm_agreement_job_creation_summary | Obligations of the private company in recruiting local people for work, instead of foreigners. ​ | 
-| Number of created jobs | odm_agreement_job_creation_number | Figures can be mentioned occasionally ​ | 
-| Training summary | odm_agreement_training_summary | Obligations of the private company in recruting local people for work, instead of foreigners. If there is no technical staff from the local people, they shall be guided by foreigner technical staff; or financial resource allocation shall be planed and iimplemented. ​ | 
-| Fund allocation for training | odm_agreement_training_number | in USD/​year ​ | 
-| Environmental protection obligation | odm_agreement_environmental_protection | The private company has obligations to mitigate the environmental impact of its project implementation to the maximum. Any logging activities are banned. Recommendations from the relevant authorities shall be condisered thoughtly. ​ | 
-| Socio-cultural protection | odm_agreement_sociocultural_protection | Obligations of the private companies in the relocation and restoration of the people livelihood, and no harmful activities to the people'​s livelihood and culture. Eg, the company is taking care on the livehood of the workers and their familities: residence, hopital, schools (ELCs). The compay shall not make any harms to the heritage sites, and spiritual areas (mining). ​ | 
-| Fiscal duties summary | odm_agreement_fiscal_duties_summary | The private company has obligations to pay tax to the state. In general, there is no figure. ​ | 
-| Total amount of fiscal duties | odm_agreement_fiscal_duties_number | in USD  | 
-| Environmental impact assessment (EIA) | odm_agreement_eia | Company'​s obligations to comply with planning for mining feasibility and restoration planning (of the nature, especially land in the area). In some case, if it is mentioned, EIA with a particular date may be left here.  | 
-| Total amount of social fund | odm_agreement_social_fund | in USD  | 
-| Environmental fund summary | odm_agreement_environmental_fund_summary | Activities of the area and nature/​environment preservation and restoration,​ and how funding is managed and spent. ​ | 
-| Total amount of environmental funds | odm_agreement_environmental_funds | in USD  | 
-| Suspension and/or revocation or termination | odm_agreement_suspension_revocation_termination | Suspension and/or revocation of the mining licenses when certain conditions are met. Termination of the contract (ELCs) may take place on a part of or the entire project upon the agreement of the party or the sole previledge of the state. ​ | 
-| Related documents | odm_agreement_related_documents |   | 
-| Related Project | odm_agreement_suspension_related_project | Link to Profile map ID for map or concession coordinates. eg. https://​opendevelopmentcambodia.net/​profiles/​economic-land-concessions/?​feature_id=elc_gdc_10 ​ | 
-| Legacy reference document | odm_reference_document | For internal use only.  | 
-| Maintainer * | maintainer |   | 
-| Maintainer email * | maintainer_email |   | 
-| Author * | author |   | 
-| Author email * | author_email |   | 
-| Keywords | odm_keywords | INTERNAL USE ONLY: Enter keywords for improving the discoverability of this record via search ​ | 
-| Open Contracting Identifier | open_contracting_id | This field is autogenerated with the following schema: ocds-miumsd-CKAN_UNIQUE_ID ​ | 
-==== Other metadata fields ==== 
  
 Other metadata fields exposed by the CKAN API: Other metadata fields exposed by the CKAN API:
public/metadata.1510826529.txt.gz · Last modified: 2020/06/23 15:03 (external edit)