User Tools

Site Tools


Menu

 ?

About

Basic concepts and guidelines

User Guides

For users

For mappers

Technical

Learn about Dokuwiki

public:setting_up_test_development_environment

FIXME This page is outdated. Please help completing the wiki updates.
(remove this paragraph once the page is updated)

Setting up test & development environment

Who is this guide for

  • Developers contributing on the development of ODM's modules
  • ODM Staff testing the actual development
  • System administrators willing to understand better ODM'S architecture

What this guide teaches

  • How to replicate the test and development environment using both Vagrant and Webfaction.

Things to know in forehand

What is Vagrant

Vagrant is free and open-source software for creating and configuring virtual development environments. It can be seen as a wrapper around virtualization software such as VirtualBox, and around configuration management software such as Chef, Salt or Puppet.

What is Ansible

Ansible is an open-source software platform for configuring and managing computers. It combines multi-node software deployment, ad hoc task execution, and configuration management. It manages nodes over SSH and requires Python (2.4 or later) to be installed on them. Modules work over JSON and standard output and can be written in any programming language. The system uses YAML to express reusable descriptions of systems.

Using Virtualized environment in local machine (Vagrant)

ODM developers can choose Vagrant for managing Virtual Machines where testing and development tasks can be undertaken. For this, following software needs to be installed and configured:

Provisioning a Vagrant VM with Ansible

Step 1: get deployment scripts

TBC

Step 2: Edit the configuration files

TBC

Step 3: Starting the box for the first time provisions it

In order to start the development box, change directory to where the Vagrantfile is and type:

cd odm-deployment
vagrant up   

This will take some time because the virtual box will be provisioned with the instructions specified in the bootstrap.sh. This script basically install some needed dependencies such as git and ansible and runs the set of Ansible instructions available on deployment/ansible in order to provision the VM.

The provisioning process might take between 5 and 15 minutes, depending on the power of your machine and speed of your internet connection.

For more information about provisioning Vagrant, refer to Vagrant documentation

Aparently, the VM needs to be restarted once after the deployment is ready in order for the different services to start properly. Following command needs to be run:

vagrant reload

If Errors occur during vagrant up

Sometimes, the deployment of the Development Environment will be stuck at some points that caused by wrong configuration or the speed of internet connection.

Double Check the Configuration

Change the location of Ubuntu servers

  • Next vagrant destroy and up again.

Fail due to Internet connection issue

Log in with ssh once provisioning finish.

Once the process is finished and no errors have occurred, it is time to access the machine via ssh. There are still some necessary steps to be done.

vagrant ssh  

Stopping the box

Exit the ssh session and call vagrant halt, this way the box will be stopped, you can start it any time by calling vagrant up again.

exit
vagrant halt

Step 3: Run the ansible playbooks

In order to start the provisioning, we need to run the ansible-playbook command:

cd ansible
ansible-playbook site.yml -i stage 

After provisioning

Finish configuring CKAN

The provisioning scripts installs CKAN and its dependencies plus some extensions, but there are still some actions to be taken in order to configure the development environment completely:

Creating a system administrator user

After activating the python environment with

. ~/.virtualenvs/ckan/bin/activate

, you can execute

paster --plugin=ckan sysadmin add USERNAME ~/etc/ckan/default/development.ini

OR

paster --plugin=ckan sysadmin add USERNAME -c ~/etc/ckan/default/development.ini

for creating a new user with sysadmin permissions, or attribute those permissions to an existing user. USERNAME is the account name, you can name whatever eg. admin or user1

Once this is done and password has been set, you can login into CKAN under http://192.168.33.10:8081/user/login.

Inserting initial data using ckanext-odm_migration

Since the ckanext-odm_migration extension gets installed automatically while provisioning the vagrant machine. Once logged in, you can access http://192.168.33.10:8081/migration where you will find a series of utilities for inserting initial data and importing taxonomy terms and their translations and/or law records.

Finish configuring WP

Accessing the code on the Development Box

Troubleshooting

Failed to fetch mirror error

Sometimes it can happen that the during the “Update packages list” step of the common Ansible playbook, the following error is thrown, stoping the execution of the provisioning.

In order to solve this, a failsafe sources list can be selected. For that, it is just necessary to change the following line in the deployment/ansible/roles/common-ubuntu/tasks/main.yml file:

- name: add new sources.list that checks for mirror
  copy: src=sources.list dest=/etc/apt/sources.list owner=root group=root mode=0644
  sudo: yes

And replace src=sources.list with one of the available failsafe sources lists in deployment/ansible/roles/common/tasks/main.yml

On start, CKAN and Tomcat not started

If the machine is provisioned and started but CKAN and/or Tomcat are not accesible through their respective URLs, it can be that supervisor has not being able to start them for some reason. In order to manually start them:

  1. Activate virtualenv:
    . ~/.virtualenvs/ckan/bin/activate
  2. Run supervisorctl:
    supervisorctl
  3. If following error message appears: unix:/home/vagrant/tmp/supervisor.sock no such file run: supervisord.
public/setting_up_test_development_environment.txt · Last modified: 2020/08/13 03:27 by arkyarky