Version 26 vs 27
Version 26 vs 27
Content Changes
Content Changes
* Our build system is Jenkins and hosted at https://jenkins.c4science.ch
* Phabricator also includes support for CircleCI
WARNING: Jenkins only works for public repositories for now (Credentials are global), we're working on it
## How to do continuous integration
There are two ways of doing unit testing in c4science
* [[https://c4science.ch/w/c4science/unit_tests/#using-arcanist-on-your-l|With Arcanist when doing code review or manually (simple)]]
* [[https://c4science.ch/w/c4science/unit_tests/#using-jenkins-and-harbor|With Jenkins or external build system (expert)]]
### Using Arcanist on your local machine
We'll be using a community extensions to integrate your custom unit test engine with Phabricator. This extensions allows any [[https://testanything.org/|TAP]] compatible engine to be used. You can also write your own engine, [[https://secure.phabricator.com/book/phabricator/article/arcanist_lint_unit/|take a look at the documentation]].
* In your repository
* Add the lib as a submodule
```
git submodule add https://github.com/tagview/arcanist-extensions.git .arcanist-extensions
git submodule update --init
```
* Create an `.arcconfig` file with the following content
```{
"load": [
".arcanist-extensions/tap_test_engine"
],
"unit.engine": "TAPTestEngine",
"unit.engine.tap.command": "your_unit_test_script.sh"
}
```
* Replace `your_unit_test_script.sh` by a script in your repository that do the unit testing and returns the result in a [[https://testanything.org/producers.html|TAP format]]
* For instance if your code is in python, you can use `tap.py`, see the example in rPHTEST
### Using Jenkins and Harbormaster on c4science
Here are the step that you have to setup, detailed in the following sections.
* Create a Jenkins Job with parameters
* Create a Build Plan on Harbormaster which will trigger the Job
* Create an Herald rule which will trigger the Build Plan
### Create a new Job
* Create a new Item of type "Freestyle project", give it a name
{F17675}
* If you want to trigger the build from Herald on a new Diffusion or Commit: choose "This build is parameterized" and add a new String Parameter with the name PHID
{F17669}
* Select Git as source code management and put the URL of the repository
** If your repository is private, you have to create a new Creadential for it, for instance using an SSH key with your Phabricator username, You'll add the public key to your phabricator profile: https://c4science.ch/settings/panel/ssh/
{F17663}
* If you want to periodically build the project, select the option under Build triggers and use this syntax `H/30 * * * *` to build every 30 mn. See complete documentation about the syntax by clicking on the question mark on the right of the field
{F17651}
* To run your unit test, add a Build step, select Execute Shell and provide the command to run your tests
{F17645}
* If you want to specify the docker image, you can choose on by clicking on "Restrict where this project can be run" with the following values
** **ubuntu:trusty** Ubuntu Trusty 14.04.4 LTS x86_64
** **centos:7** Centos 7.2 x86_64
** More to come
{F17639}
* If you want to trigger the build from Phabricator, under Build Triggers, tick "Trigger builds remotely" and create a token.
** The best would be some random string, you can generate one like this on a linux shell: `head -c 16 /dev/urandom | md5sum`
* To have the status reported back to Phabricator, add the "Post to Phabricator" post-build step
#### Trigger build on commit
* You can trigger a build from Phabricator using Harbormaster and Herald applications, either on commit or on differential
##### Harbormaster (run build)
* Create a Build Plan for your repository, https://c4science.ch/harbormaster/plan/
* You'll have to create a custom policy so yourself can view and edit, as well as the @jenkins user
{F46021}
NOTE: This is a hack and potentially gives access to other users to your build plan. Only use Jenkins for public projects for now
* Add a new Build Step of type "Make HTTP Request"
{F16204}
* Add the following URL, `https://jenkins.c4science.ch/buildByToken/buildWithParameters?PHID=${target.phid}&token=<your token>&job=<your job name>`
* Select "GET" on the //HTTP Method// field
* Select "Wait For Message" on the //When Complete// field.
* Save your step
{F16207}
##### Herald (Notification and action)
* Go to Herald, https://c4science.ch/herald/
* Create a new Rule on Commits of type Object
* Put the reference of your object, the repository ID (rXX) or callsign (rCALLSIGN)
* Give it a name and select the condition as "Repository is any of <your repository>"
* Then select the "Run build plans" action with your newly created Build plan
* Save the rule
* Now when a commit is pushed on your repository, a build will be generated on your Jenkins job and the result will be reported back to phabricator.
* You can access the jenkins job build in the harbormaster build, under artifacts, as shown on this image
{F16202, size=full}
* Our build system is Jenkins and hosted at https://jenkins.c4science.ch
* Phabricator also includes support for CircleCI
WARNING: Jenkins only works for public repositories for now (Credentials are global), we're working on it
## How to do continuous integration
There are two ways of doing unit testing in c4science
* [[https://c4science.ch/w/c4science/unit_tests/#using-arcanist-on-your-l|With Arcanist when doing code review or manually (simple)]]
* [[https://c4science.ch/w/c4science/unit_tests/#using-jenkins-and-harbor|With Jenkins or external build system (expert)]]
### Using Arcanist on your local machine
We'll be using a community extensions to integrate your custom unit test engine with Phabricator. This extensions allows any [[https://testanything.org/|TAP]] compatible engine to be used. You can also write your own engine, [[https://secure.phabricator.com/book/phabricator/article/arcanist_lint_unit/|take a look at the documentation]].
* In your repository
* Add the lib as a submodule
```
git submodule add https://github.com/tagview/arcanist-extensions.git .arcanist-extensions
git submodule update --init
```
* Create an `.arcconfig` file with the following content
```{
"load": [
".arcanist-extensions/tap_test_engine"
],
"unit.engine": "TAPTestEngine",
"unit.engine.tap.command": "your_unit_test_script.sh"
}
```
* Replace `your_unit_test_script.sh` by a script in your repository that do the unit testing and returns the result in a [[https://testanything.org/producers.html|TAP format]]
* For instance if your code is in python, you can use `tap.py`, see the example in rPHTEST
* Those tests can be run by using the `arc unit` command or it will be run automatically when you create a Differential Revision using Arcanist (`arc diff`)
### Using Jenkins and Harbormaster on c4science
Here are the step that you have to setup, detailed in the following sections.
* Create a Jenkins Job with parameters
* Create a Build Plan on Harbormaster which will trigger the Job
* Create an Herald rule which will trigger the Build Plan
### Create a new Job
* Create a new Item of type "Freestyle project", give it a name
{F17675}
* If you want to trigger the build from Herald on a new Diffusion or Commit: choose "This build is parameterized" and add a new String Parameter with the name PHID
{F17669}
* Select Git as source code management and put the URL of the repository
** If your repository is private, you have to create a new Creadential for it, for instance using an SSH key with your Phabricator username, You'll add the public key to your phabricator profile: https://c4science.ch/settings/panel/ssh/
{F17663}
* If you want to periodically build the project, select the option under Build triggers and use this syntax `H/30 * * * *` to build every 30 mn. See complete documentation about the syntax by clicking on the question mark on the right of the field
{F17651}
* To run your unit test, add a Build step, select Execute Shell and provide the command to run your tests
{F17645}
* If you want to specify the docker image, you can choose on by clicking on "Restrict where this project can be run" with the following values
** **ubuntu:trusty** Ubuntu Trusty 14.04.4 LTS x86_64
** **centos:7** Centos 7.2 x86_64
** More to come
{F17639}
* If you want to trigger the build from Phabricator, under Build Triggers, tick "Trigger builds remotely" and create a token.
** The best would be some random string, you can generate one like this on a linux shell: `head -c 16 /dev/urandom | md5sum`
* To have the status reported back to Phabricator, add the "Post to Phabricator" post-build step
#### Trigger build on commit
* You can trigger a build from Phabricator using Harbormaster and Herald applications, either on commit or on differential
##### Harbormaster (run build)
* Create a Build Plan for your repository, https://c4science.ch/harbormaster/plan/
* You'll have to create a custom policy so yourself can view and edit, as well as the @jenkins user
{F46021}
NOTE: This is a hack and potentially gives access to other users to your build plan. Only use Jenkins for public projects for now
* Add a new Build Step of type "Make HTTP Request"
{F16204}
* Add the following URL, `https://jenkins.c4science.ch/buildByToken/buildWithParameters?PHID=${target.phid}&token=<your token>&job=<your job name>`
* Select "GET" on the //HTTP Method// field
* Select "Wait For Message" on the //When Complete// field.
* Save your step
{F16207}
##### Herald (Notification and action)
* Go to Herald, https://c4science.ch/herald/
* Create a new Rule on Commits of type Object
* Put the reference of your object, the repository ID (rXX) or callsign (rCALLSIGN)
* Give it a name and select the condition as "Repository is any of <your repository>"
* Then select the "Run build plans" action with your newly created Build plan
* Save the rule
* Now when a commit is pushed on your repository, a build will be generated on your Jenkins job and the result will be reported back to phabricator.
* You can access the jenkins job build in the harbormaster build, under artifacts, as shown on this image
{F16202, size=full}
* Our build system is Jenkins and hosted at https://jenkins.c4science.ch
* Phabricator also includes support for CircleCI
WARNING: Jenkins only works for public repositories for now (Credentials are global), we're working on it
## How to do continuous integration
There are two ways of doing unit testing in c4science
* [[https://c4science.ch/w/c4science/unit_tests/#using-arcanist-on-your-l|With Arcanist when doing code review or manually (simple)]]
* [[https://c4science.ch/w/c4science/unit_tests/#using-jenkins-and-harbor|With Jenkins or external build system (expert)]]
### Using Arcanist on your local machine
We'll be using a community extensions to integrate your custom unit test engine with Phabricator. This extensions allows any [[https://testanything.org/|TAP]] compatible engine to be used. You can also write your own engine, [[https://secure.phabricator.com/book/phabricator/article/arcanist_lint_unit/|take a look at the documentation]].
* In your repository
* Add the lib as a submodule
```
git submodule add https://github.com/tagview/arcanist-extensions.git .arcanist-extensions
git submodule update --init
```
* Create an `.arcconfig` file with the following content
```{
"load": [
".arcanist-extensions/tap_test_engine"
],
"unit.engine": "TAPTestEngine",
"unit.engine.tap.command": "your_unit_test_script.sh"
}
```
* Replace `your_unit_test_script.sh` by a script in your repository that do the unit testing and returns the result in a [[https://testanything.org/producers.html|TAP format]]
* For instance if your code is in python, you can use `tap.py`, see the example in rPHTEST
* Those tests can be run by using the `arc unit` command or it will be run automatically when you create a Differential Revision using Arcanist (`arc diff`)
### Using Jenkins and Harbormaster on c4science
Here are the step that you have to setup, detailed in the following sections.
* Create a Jenkins Job with parameters
* Create a Build Plan on Harbormaster which will trigger the Job
* Create an Herald rule which will trigger the Build Plan
### Create a new Job
* Create a new Item of type "Freestyle project", give it a name
{F17675}
* If you want to trigger the build from Herald on a new Diffusion or Commit: choose "This build is parameterized" and add a new String Parameter with the name PHID
{F17669}
* Select Git as source code management and put the URL of the repository
** If your repository is private, you have to create a new Creadential for it, for instance using an SSH key with your Phabricator username, You'll add the public key to your phabricator profile: https://c4science.ch/settings/panel/ssh/
{F17663}
* If you want to periodically build the project, select the option under Build triggers and use this syntax `H/30 * * * *` to build every 30 mn. See complete documentation about the syntax by clicking on the question mark on the right of the field
{F17651}
* To run your unit test, add a Build step, select Execute Shell and provide the command to run your tests
{F17645}
* If you want to specify the docker image, you can choose on by clicking on "Restrict where this project can be run" with the following values
** **ubuntu:trusty** Ubuntu Trusty 14.04.4 LTS x86_64
** **centos:7** Centos 7.2 x86_64
** More to come
{F17639}
* If you want to trigger the build from Phabricator, under Build Triggers, tick "Trigger builds remotely" and create a token.
** The best would be some random string, you can generate one like this on a linux shell: `head -c 16 /dev/urandom | md5sum`
* To have the status reported back to Phabricator, add the "Post to Phabricator" post-build step
#### Trigger build on commit
* You can trigger a build from Phabricator using Harbormaster and Herald applications, either on commit or on differential
##### Harbormaster (run build)
* Create a Build Plan for your repository, https://c4science.ch/harbormaster/plan/
* You'll have to create a custom policy so yourself can view and edit, as well as the @jenkins user
{F46021}
NOTE: This is a hack and potentially gives access to other users to your build plan. Only use Jenkins for public projects for now
* Add a new Build Step of type "Make HTTP Request"
{F16204}
* Add the following URL, `https://jenkins.c4science.ch/buildByToken/buildWithParameters?PHID=${target.phid}&token=<your token>&job=<your job name>`
* Select "GET" on the //HTTP Method// field
* Select "Wait For Message" on the //When Complete// field.
* Save your step
{F16207}
##### Herald (Notification and action)
* Go to Herald, https://c4science.ch/herald/
* Create a new Rule on Commits of type Object
* Put the reference of your object, the repository ID (rXX) or callsign (rCALLSIGN)
* Give it a name and select the condition as "Repository is any of <your repository>"
* Then select the "Run build plans" action with your newly created Build plan
* Save the rule
* Now when a commit is pushed on your repository, a build will be generated on your Jenkins job and the result will be reported back to phabricator.
* You can access the jenkins job build in the harbormaster build, under artifacts, as shown on this image
{F16202, size=full}
c4science · Help