Skip to main content

Migration Guide

Introduction​

This is a migration guide for projects that have been created in the past and do not use the project-expanders yet. New projects should use the ns-initializer script to be generated. The migration is described in two steps which can be implemented as two separate pull requests:

  1. Modify the project setup to be what the project-expanders expect. A lot of files (the whole software project) will be moved.
  2. Add the project-expanders to the project. Not everything is described in this guide. This is a major technical change on your projects. It does not keep into account any specifics of your project. Handle with care!

Definitions​

  • bold: directories
  • software-project-name(kebab-case): kebab-cased version of the software project name
  • @angular: label indicating that this is for projects using the angular frontend.
// replace the softwareProjectName
  software-project-name(kebab-case)

Code block indication that the softwareProjectName should be replaced with the software-project-name. In the code we mention artifact groups like nsapp or nsapp/software-project-name(kebab-case). Just like the softwareProjectName this is dependent on your project and is extracted from the Docker image name Image names and versions are determined as follows: <service.artifact.group>/<service.artifact.technicalName>:<service.artifact.version or project.version or latest> This means that if no version is specified the project version is used. This has one exception. If an image is used without a version in a isLocal (development) environment, then latest is used. This should become clear once the model has been defined.

Preparation​

caution

Before you start, make sure that any compose file used on Jenkins does NOT contain ANY volume mounts that are bind mounts. Mounting a directory that does not exist will cause Docker to create it - breaking Jenkins. If you absolutely need to add bind mounts to your Jenkins compose file, they must contain the create_host_path: false and read_only: true options.

  • Make sure nss is up to date!
  • Make sure you are on a clean branch from develop!!
    • In the unlikely case you don’t use git. Take a backup.

Section 1: Moving files

The following files can be moved into the project/software-project-name(kebab-case) directory:

  • angular/ @angular
  • application/
  • components/
  • conf/
  • docker/ -> docker/backend-image/ @angular
  • docker/ -> docker/software-project-name(kebab-case)-image/
  • scripts/
  • settings/
  • Jenkinsfile
  • pom.xml
  • README.md
  • any other softwareProject specific directories or files Do not move the following files:
  • CHANGELOG.md
  • renovate.json
  • .editorconfig
  • .gitignore

Since the Jenkinsfile has been moved into a subfolder within the project, Jenkins will not find it. Create another Jenkinsfile on the root level of the project with the following content:

node {
  checkout scm
// Replace the softwareProjectName
  load 'project/software-project-name(kebab-case)/Jenkinsfile'
}

Your project should now look something like this:

project/
├── software-project-name(kebab-case)/
│   └── angular/
│   └── applications/
│   └── components/
│   └── conf/
│   └── docker/
│       └── software-project-name(kebab-case)-image/
│           └── sql/
│               └── BASE/
│                    └── V1.0.0__initial_provision.sql `adapt to your base!`
│               └── MIGRATIONS/
│                     └── 1.0.0/
│                           └── V2024.01.01.0900__a_previous_database_migration.sql
│                     └── PREV/ `to keep historic migration scripts, usually after re-basing`
│                           └── .gitkeep
│                     └── NEXT/
│                           └── V2025.01.01.0900__a_database_migration.sql
│                           └── .gitkeep
│   └── scripts/
│   └── settings/
│   └── Jenkinsfile
│   └── pom.xml
│   └── README.md
├── CHANGELOG.md
├── Jenkinsfile
├── README.md
├── renovate.json
├── .gitignore
.
caution

Make sure the jenkinsfile is adjusted to compensate for the Docker folders moving! If your project used the pom.info in the Jenkinsfile, to make sure your build is successful, make sure you have a pom.xml on the level jenkins is reading the pom info from and that it matches the information of the software project.

info

It is a recommended to add a back button in the README in the software project with [back](../../README.md) The sql folder structure is obligatory if you want to use the project.artifact.moveSqlScriptsOnRelease option. Make a pull request and make sure all builds are successful. After this merge everyone will have to reimport the softwareProject in their μRadiant.

Section 2: Apply the project expanders

Step 1: Add the basic setup​

Create the following directories and files in your project:

  • .nsproject/ new
    • model/ new
      • projectSetup.xml new
    • expansionSettings.xml new
  • scripts/
    • expand_project_setup.nss new
    • harvest_project_setup.nss new
  • project/
    • docker/
      • software-project-name/
        • in this folder move the prepare.sh -> **scripts/**prepare.sh moved
    • .cleanExclude new
  • .gitignore new
  • Jenkinsfile updated
  • pom.xml new
  • README.md new

File contents​

<project xmlns="https://schemas.normalizedsystems.org/xsd/project/2/1/0">
  <name>projectSetup</name>
  <!--  Replace the projectVersion -->
  <version>projectVersion</version>
</project>

Step 2: Removing files​

The following files can be removed (if present):

  • project/
    • software-project-name(kebab-case)/
      • conf/
        • nsmr.xml
      • docker/
        • software-project-name-image/
          • Dockerfile
      • scripts/
        • *.nss
      • package.json
info

Any .nss script that is not generated by the project setup should be self-contained. No more file imports.

Step 3: Creating the project setup model​

Open the project setup in the μRadiant. Do this by importing the expansionSettings.xml in the .nsproject folder of your project. Create the following elements with the µRadiant

info
  • Editing this is only available in μRadiant v2 (or higher) with the org-normalizedsystems-northstar-project-plugin (mrp-project-plugin) installed
  • When importing the project from expansionSettings.xml in μRadiant v2 on linux, hidden files (ex: .nsproject/) are not shown by default, make sure you tick the show hidden option.
  • μRadiant v2 currently doesn’t support the open folder functionality for the project setup (as this doesn’t use the conf/ folder)

Model​

Here we will highlight the different elements in the model to clarify their purpose. This is mainly related to Docker and Jenkins artifacts here but other systems like Tekton could be included in the future.

  • projectSetup::artifact
    • These are the artifacts used or produced by the projectSetup::softwareProject.
  • projectSetup::environment
    • These describe useful environments for development, Jenkins pipelines or production deployments.
  • projectSetup::service
    • These are the services that are used in the environments, they can be external or use artifacts created by the projectSetup::softwareProject.
  • projectSetup::softwareProject
    • This is the softwareProject creating the artifacts that are used to build the services
    • projectSetup::pipeline
      • describes the build pipline for the project
      • projectSetup::stage
        • describes the stages in the build pipeline for the project.

Artifacts​

caution

Neither the artifact name nor the technicalName should not contain any / ! If there is a group for the Docker images they should be added as the group field.

Below we will use the nsapp group as an example.

Project with an Angular frontend​

  • frontendImage
    • technicalName: frontend
    • group: nsapp/software-project-name(kebab-case)
    • type: angular-frontend-image
  • backendImage
    • technicalName: backend
    • group: nsapp/software-project-name(kebab-case)
    • type: jee-application-image
  • gatewayImage
    • technicalName: gateway
    • group: nsapp/software-project-name(kebab-case)
    • type: gateway-image
  • databaseImage
    • technicalName: postgres
    • version: 17.2 (use your current postgres version)
    • type: custom

Project without an Angular frontend​

  • software-project-name-image
    • technicalName: software-project-name(kebab-case)
    • group: nsapp
    • type: jee-application-image
  • databaseImage
    • technicalName: postgres
    • version: 17.2 (use your current postgres version)
    • type: custom

Environments​

  • default
    • options:
      • project.environment.isLocal

SoftwareProjects​

Project with an Angular frontend​

  • softwareProjectName
    • type: jee-application
    • artifacts:
      • projectSetup::frontend-image
      • projectSetup::backend-image
      • projectSetup::gateway-image
    • pipelines
      • build-pipeline
        • stages
          • validate
            • type: validate
          • expand
            • type: expand
          • build
            • type: build
          • deploy
            • type: deploy
          • report
            • type: report
          • analyze
            • type: analyze
        • (possible) options:
          • project.pipeline.slackChannel: team-jenkins-slack-channel
          • project.pipeline.changelogSlackChannel: team-release-slack-channel
          • project.pipeline.onlyReportChanges
          • project.pipeline.noReleasePromotion
          • project.pipeline.relaxBranchVersionCheck
    • options:
      • project.softwareProject.frontend: angular-app-name @angular

Project without an Angular frontend​

  • softwareProjectName
    • type: jee-application
    • artifacts:
      • softwareProjectNameImage
    • pipelines
      • build-pipeline
        • stages
          • validate
            • type: validate
          • expand
            • type: expand
          • build
            • type: build
          • deploy
            • type: deploy
          • report
            • type: report
          • analyze
            • type: analyze
        • (possible) options:
          • project.pipeline.slackChannel: team-jenkins-slack-channel
          • project.pipeline.changelogSlackChannel: team-release-slack-channel
          • project.pipeline.onlyReportChanges
          • project.pipeline.noReleasePromotion
          • project.pipeline.relaxBranchVersionCheck
    • options:
      • project.softwareProject.frontend: angular-app-name @angular

Services​

Project with an Angular frontend​

  • frontend
    • artifact: projectSetup::frontendImage
    • environments
      • projectSetup::default
    • type: angular-frontend
    • softwareProject: projectSetup::softwareProject
  • backend
    • artifact: project::backendImage
    • environments
      • projectSetup::default
    • type: jee-application
    • softwareProject: projectSetup::softwareProject
  • gateway
    • artifact: projectSetup::gatewayImage
    • environments
      • projectSetup::default
    • type: gateway
    • softwareProject: projectSetup::softwareProject
  • database
    • artifact: projectSetup::databaseImage
    • environments
      • projectSetup::default
    • type: database

Project without an Angular frontend​

  • software-project-name
    • artifact: projectSetup::softwareProjectName
    • environments
      • projectSetup::default
    • type: jee-application
    • softwareProject: projectSetup::softwareProject
  • database
    • artifact: projectSetup::databaseImage
    • environments
      • projectSetup::default
    • type: database

Step 4: Run!​

You should now be ready to use the project-expanders! To do so, do the following in the μRadiant

  1. In the project setup, go to build => script and run the scripts/expand_project_setup.nss script
  • This should provide you with all the files necessary to build and deploy the project.
caution

Here you transfer all the necessary data from your compose files into the environment/override compose files

  • Ports
  • Environment variables
  1. Now build your application using μRadiant or the nsScripts that have been expanded to your project setup.
  2. And finally, run start_default_services.nss. The local docker containers should be started and you should have an environment deployed on our machine!
  3. To undeploy, run stop_default_services.nss
info

Any files can still be created next to the expanded files.

Step 5: Opportunities for improvement​

Please do provide us with any and all feedback you have! As this is a substantial change, there are a lot of details not mentioned in this guide.