Phriction Projects Wikis Bioimaging And Optics Platform Image Processing Version Control : Git, GitHub Actions CI (continuous integration) and Scijava Maven History Version 13 vs 33
Version 13 vs 33
Version 13 vs 33
Content Changes
Content Changes
= About GIT =
== On https://c4science.ch/ ==
Now there is a new GIT Management tool on EPFL, https://c4science.ch/
Oli created the [[https://c4science.ch/project/view/567/ | BioImaging And Optics Platform Group]] there and has added all the projects currently under version control.
To add a project to **c4science** we need to
# Create a new repository following the conventions below.
# Change the permissions so that only the members of [[https://c4science.ch/project/view/569/ | ImageJ Tools]] can make changes
# Activate the repository
# Use the link to push to it from the local repo
{F164844, layout=right, float}
=== Conventions ===
Please follow these conventions when creating a repository containing code
|Type |Name |Short Name|
|------|-------|-------------|
| ActionBars | {name} ActionBar | ijab-... |
| Plugins | {name} Plugin | ijp-... |
| Macros | {name} Macro | ijm-... |
| Scripts | {name } Script | ijs-... |
| XTensions | {name} XT | xt-... |
| Python Projects | {name} Python | py-...|
- All things ImageJ are to be placed under the project [[https://c4science.ch/project/view/569/ | ImageJ Tools]]
- Imaris Related Projects Should be under [[https://c4science.ch/project/view/569/ | Imaris Tools]]
- Other tools, projects (Cell Profiler, Ilastik, ...) you should create a new project.
=== Creating a new repository
# From the main page, go to **[[https://c4science.ch/diffusion/edit/form/default/| Create A New Repository]]**
# Choose **GIT**
# Please fill in the fields as suggested by the **Conventions** Section above
# Don't forget to add the repository to the right project!
The new repository **is not yet active**. You need to first go to **Policies** and change them as follows:
- **Visible To**: All Users
- **Editable By**: Members of Project// (ImageJ Tools / Imaris Tools...)//
- **Can Push**: Members of Project// (ImageJ Tools / Imaris Tools...)//
Save and go back to **Basics** and **Activate the Repository** On the right hand menu
There is also a page detailing [[c4science/simplerepo| how to create a simple repository]]
=== Migrating a git.epfl.ch repository ===
There's a [[c4science/migration | short documentation]] on how to migrate a project initially on https://git.epfl.ch
= GitHub PTBIOP Account =
https://github.com/PTBIOP
Oli created an account for the BIOP on GitHub. That way our public work is more visible and more professional
Both Romain and Oli are administrators and can add more people and create repositories.
= C4Science VCS Password =
In order to push data to C4Science, you need to define a **Version Control System** Password (VCS)
1. Click top right on your icon > select `Settings` .
2. On the left bar, select `VCS password`. Enter a password.
3. Now you can enter the correct password when asked when you push a repo.
= Working with multiple remotes for a given repo: GitHub & C4Science =
This means you need to be identified with two different accounts. Git is somehow clever because it chooses the right account based on the URL of the remote repository.
When you add a remote repository, specify your login. For instance
```
git remote add origin https://chiarutt@c4science.ch/source/bacteriaworkflow.git/
```
where `chiarutt` is your **c4science login**
You will be promted for your **c4science VCS password**. You NEED to specify the password in c4science beforehand
NOTE: Don't forget to add your user name before `c4science`
If the password was wrongly entered, you can remove it from the OS password manager. In windows 10, it means : execute `Credential Manager` (search with the windows key), then go to Windows Credentials, find the credentials and delete it. Git will re-ask for authentication, and this time you can enter the correct password.
= Scijava Maven, Travis and reproducible repository releases =
Related :
* https://imagej.net/Travis
* https://imagej.net/Project_management#Continuous_integration
In order to get reproducible release of repositories, it is possible to use continuous integration and public repositories to share these reproducible repositories. This is useful in order to give easily other developers to depend on your repository. Even if the principles are general the procedure described below works for the fiji scijava ecosystem.
This is a complicated procedure initially because you need your repository to abide by stiff rules. Also, at the moment not anybody have the right to trigger the continuous integration for every repository. In practice, either Jan Eglinger or Nicolas Chiaruttini have this right and a bit of time to do it. So for the initial step you will have to ask. But before contacting them, a few things need to be done:
* clone https://github.com/scijava/scijava-scripts and - it will ease the execution of these scripts - put the scripts repo at the same hierarchy as all the other repos you want to travisify :
{F16193593}
* clean your repository.
** All dependencies should be publicly available, either through scijava maven (https://maven.scijava.org/ - mainly imagej stuff) or through maven central (https://search.maven.org/). If that's not the case, I don't know what can be done for the moment...
** The dependency versions should : not be a SNAPSHOT dependency (because it can be overriden it's not reproducible) + their versions should be declared in the properties part of the pom.xml file of the repository.
DO NOT WRITE:
COUNTEREXAMPLE
<dependency>
<groupId>org.reflections</groupId>
<artifactId>reflections</artifactId>
<version>0.9.11</version>
</dependency>
WRITE:
<properties>
...
<reflections.version>0.9.11</reflections.version>
</properties>
<dependency>
<groupId>org.reflections</groupId>
<artifactId>reflections</artifactId>
<version>${reflections.version}</version>
</dependency>
This allows the versions to be managed / checked and overriden if necessary by scijava maven plugins.
** Take care with javadoc. To test if you did not make any mistake with javadoc, type `mvn javadoc:javadoc`in the intellij terminal (this can take a while). Then correct all the errors (this can take a while also). Warning can be ignored ( thus you can type `mvn javadoc:javadoc | grep error` to avoid displaying warnings, but that works only with git bash, not in the windows terminal`). Usually you have issues with special characters (`<`, `>`, `@`, etc...). You need to replace them with their html representation `&whatever`, see https://dev.w3.org/html5/html-author/charref)
* fix your pom file with proper settings for continuous integration. A correct pom file is the one of `https://github.com/BIOP/ijp-kheops`. So have a look at it, and fill the sections `<ciManagement>` `<issueManagement>` and `<scm>` appropriately.
* Once this is done, you need to run `travisify.sh` in simulation mode to see if everything looks right:
** in an explorer window, go to the folder where the pom file of your repo is located. Right-click and execute `git bash here`. Then type `../scijava-scripts/travisify.sh`. If this doesn't work, push to GitHub and ask Nico to take over from now on.
Reminder for windows : add permission to build.sh : https://github.com/scijava/scijava-scripts/issues/25
* what Nico or Jan will do :
** run `../scijava-scripts/travisify.sh -f` to run travisify for real
** push the repo - which will contain newly generated commits for CI
* then Travis will run the build and the badge on the github repo will turn green or red depending on the outcome. You can check the status of the build by going at the url located in the ciManagement section of the pom file.
* You are almost done. Now that the repo is travisified, each time you want to push a release, you need to:
** in an explorer window, go to the folder where the pom file of your repo is located. Right-click and execute `git bash here`. Then type `../scijava-scripts/release-version.sh`. You are asked which version that you want to release. Usually, you are working on a version like v1.2.3-SNAPSHOT, and the script will release the version v1.2.3 and will automatically commit modification in order to update your repo to v1.2.4-SNAPSHOT, where you can continue your development. If this doesn't work, push to GitHub and ask Nico for help.
Current BIOP repo currently using CI and present in the scijava maven repository:
- https://github.com/bigdataviewer/bigdataviewer-playground
- https://github.com/BIOP/bigdataviewer-bioformats
- https://github.com/BIOP/bigdataviewer-biop-tools
- https://github.com/BIOP/ijp-kheops
- https://github.com/NicoKiaru/bigdataviewer-selector
- LaRome TODO
- TODO : Easy-XT, but how to deal with dependencies ??
= About GIT =
== On https://c4science.ch/ ==
Now there is a new GIT Management tool on EPFL, https://c4science.ch/
Oli created the [[https://c4science.ch/project/view/567/ | BioImaging And Optics Platform Group]] there and has added all the projects currently under version control.
To add a project to **c4science** we need to
# Create a new repository following the conventions below.
# Change the permissions so that only the members of [[https://c4science.ch/project/view/569/ | ImageJ Tools]] can make changes
# Activate the repository
# Use the link to push to it from the local repo
{F164844, layout=right, float}
=== Conventions ===
Please follow these conventions when creating a repository containing code
|Type |Name |Short Name|
|------|-------|-------------|
| ActionBars | {name} ActionBar | ijab-... |
| Plugins | {name} Plugin | ijp-... |
| Macros | {name} Macro | ijm-... |
| Scripts | {name } Script | ijs-... |
| XTensions | {name} XT | xt-... |
| Python Projects | {name} Python | py-...|
- All things ImageJ are to be placed under the project [[https://c4science.ch/project/view/569/ | ImageJ Tools]]
- Imaris Related Projects Should be under [[https://c4science.ch/project/view/569/ | Imaris Tools]]
- Other tools, projects (Cell Profiler, Ilastik, ...) you should create a new project.
=== Creating a new repository
# From the main page, go to **[[https://c4science.ch/diffusion/edit/form/default/| Create A New Repository]]**
# Choose **GIT**
# Please fill in the fields as suggested by the **Conventions** Section above
# Don't forget to add the repository to the right project!
The new repository **is not yet active**. You need to first go to **Policies** and change them as follows:
- **Visible To**: All Users
- **Editable By**: Members of Project// (ImageJ Tools / Imaris Tools...)//
- **Can Push**: Members of Project// (ImageJ Tools / Imaris Tools...)//
Save and go back to **Basics** and **Activate the Repository** On the right hand menu
There is also a page detailing [[c4science/simplerepo| how to create a simple repository]]
=== Migrating a git.epfl.ch repository ===
There's a [[c4science/migration | short documentation]] on how to migrate a project initially on https://git.epfl.ch
= GitHub PTBIOP Account =
https://github.com/PTBIOP
Oli created an account for the BIOP on GitHub. That way our public work is more visible and more professional
Both Romain and Oli are administrators and can add more people and create repositories.
= C4Science VCS Password =
In order to push data to C4Science, you need to define a **Version Control System** Password (VCS)
1. Click top right on your icon > select `Settings` .
2. On the left bar, select `VCS password`. Enter a password.
3. Now you can enter the correct password when asked when you push a repo.
= Working with multiple remotes for a given repo: GitHub & C4Science =
This means you need to be identified with two different accounts. Git is somehow clever because it chooses the right account based on the URL of the remote repository.
When you add a remote repository, specify your login. For instance
```
git remote add origin https://chiarutt@c4science.ch/source/bacteriaworkflow.git/
```
where `chiarutt` is your **c4science login**
You will be promted for your **c4science VCS password**. You NEED to specify the password in c4science beforehand
NOTE: Don't forget to add your user name before `c4science`
If the password was wrongly entered, you can remove it from the OS password manager. In windows 10, it means : execute `Credential Manager` (search with the windows key), then go to Windows Credentials, find the credentials and delete it. Git will re-ask for authentication, and this time you can enter the correct password.
= Scijava Maven, Travis and reproducible repository releases =
Related :
* https://imagej.net/Travis
* https://imagej.net/Project_management#Continuous_integration
In order to get reproducible release of repositories, it is possible to use continuous integration and public repositories to share these reproducible repositories. This is useful in order to give easily other developers to depend on your repository. Even if the principles are general the procedure described below works for the fiji scijava ecosystem.
This is a complicated procedure initially because you need your repository to abide by stiff rules. Also, at the moment not anybody have the right to trigger the continuous integration for every repository. In practice, either Jan Eglinger or Nicolas Chiaruttini have this right and a bit of time to do it. So for the initial step you will have to ask. But before contacting them, a few things need to be done:
* clone https://github.com/scijava/scijava-scripts and - it will ease the execution of these scripts - put the scripts repo at the same hierarchy as all the other repos you want to travisify :
{F16193593}
* clean your repository.
** All dependencies should be publicly available, either through scijava maven (https://maven.scijava.org/ - mainly imagej stuff) or through maven central (https://search.maven.org/). If that's not the case, I don't know what can be done for the moment...
** The dependency versions should : not be a SNAPSHOT dependency (because it can be overriden it's not reproducible) + their versions should be declared in the properties part of the pom.xml file of the repository.
DO NOT WRITE:
COUNTEREXAMPLE
<dependency>
<groupId>org.reflections</groupId>
<artifactId>reflections</artifactId>
<version>0.9.11</version>
</dependency>
WRITE:
<properties>
...
<reflections.version>0.9.11</reflections.version>
</properties>
<dependency>
<groupId>org.reflections</groupId>
<artifactId>reflections</artifactId>
<version>${reflections.version}</version>
</dependency>
This allows the versions to be managed / checked and overriden if necessary by scijava maven plugins.
** Take care with javadoc. To test if you did not make any mistake with javadoc, type `mvn javadoc:javadoc`in the intellij terminal (this can take a while). Then correct all the errors (this can take a while also). Warning can be ignored ( thus you can type `mvn javadoc:javadoc | grep error` to avoid displaying warnings, but that works only with git bash, not in the windows terminal`). Usually you have issues with special characters (`<`, `>`, `@`, etc...). You need to replace them with their html representation `&whatever`, see https://dev.w3.org/html5/html-author/charref)
* fix your pom file with proper settings for continuous integration. A correct pom file is the one of `https://github.com/BIOP/ijp-kheops`. So have a look at it, and fill the sections `<ciManagement>` `<issueManagement>` and `<scm>` appropriately.
* Once this is done, you need to run `travisify.sh` in simulation mode to see if everything looks right:
** in an explorer window, go to the folder where the pom file of your repo is located. Right-click and execute `git bash here`. Then type `../scijava-scripts/travisify.sh`. If this doesn't work, push to GitHub and ask Nico to take over from now on.
Reminder for windows : add permission to build.sh : https://github.com/scijava/scijava-scripts/issues/25
* what Nico or Jan will do :
** run `../scijava-scripts/travisify.sh -f` to run travisify for real
** push the repo - which will contain newly generated commits for CI
* then Travis will run the build and the badge on the github repo will turn green or red depending on the outcome. You can check the status of the build by going at the url located in the ciManagement section of the pom file.
* You are almost done. Now that the repo is travisified, each time you want to push a release, you need to:
** in an explorer window, go to the folder where the pom file of your repo is located. Right-click and execute `git bash here`. Then type `../scijava-scripts/release-version.sh`. You are asked which version that you want to release. Usually, you are working on a version like v1.2.3-SNAPSHOT, and the script will release the version v1.2.3 and will automatically commit modification in order to update your repo to v1.2.4-SNAPSHOT, where you can continue your development. If this doesn't work, push to GitHub and ask Nico for help.
Current BIOP repo currently using CI and present in the scijava maven repository:
- https://github.com/bigdataviewer/bigdataviewer-playground
- https://github.com/BIOP/bigdataviewer-bioformats
- https://github.com/BIOP/bigdataviewer-biop-tools
- https://github.com/BIOP/ijp-kheops
- https://github.com/NicoKiaru/bigdataviewer-selector
- LaRome TODO
- TODO : Easy-XT, but how to deal with dependencies ??
c4science · Help