Configuring the Moderne agent

In order to securely communicate with the Moderne SaaS, you will need to set up an on-premise agent in your environment. To assist you with that process and provide you with information about the agent, this guide will:

High-level agent information

What does the agent do?

The Moderne on-premise agent:

  • Encrypts and ships LST and recipe artifacts from your artifact repository (e.g., Artifactory) to the Moderne SaaS

  • Provides the symmetric key that Moderne needs to decrypt your artifacts

  • Forwards requests from the Moderne SaaS to your SCM(s) (e.g., GitHub)

  • Forwards requests from the Moderne SaaS to the organization service (if configured)

Agent setup instructions

Step 1: Contact Moderne to obtain access

Please talk to your sales representative or contact us in order to obtain access to the agent. We will work with you to determine what access details are appropriate for your platform.

Step 2: Generate your symmetric key

The Moderne agent requires customers to create a hex-encoded 256-bit AES encryption key. This key will be used to encrypt LST and recipe artifacts before they are sent to your SaaS tenant. To generate a key, please run the following openssl command:

openssl enc -aes-256-cbc -k secret -P

This will return a salt, key, and iv. Please copy the key and save it for use in step 4 as the symmetricKey.

Step 3: Determine how you will run the agent

Moderne offers two ways of running the agent:

  1. An OCI image that can be run using any OCI runtime (e.g., Docker, Podman)

  2. A Spring Boot executable JAR that can be run with Java

Regardless of which one you pick, you'll want a minimum system spec of 2 CPU cores, 8 GB of memory, and at least 10 GB of persistent or local storage.

If you deploy to Kubernetes or any other containerized environment like AWS ECS, you'll want to use the OCI image to run the agent.

If you deploy to a PaaS environment such Cloud Foundry, you'll want to use the JAR to run the agent.

The table below provides the core command for running the agent. However, in order for the agent to function correctly, additional variables will need to be added based on your environment (such as what SCM(s) your company uses, what artifact repositories you have configured, and whether or not you've configured an Organizations service). We'll walk through each of those in the following steps.

How to run the agent:

  1. Log in to the Moderne registry:

docker login -u moderne-tenant -p <password provided by Moderne> moderne.azurecr.io
  1. Pull down the latest agent:

docker pull moderne.azurecr.io/moderne-dev/moderne/moderne-agent:latest
  1. Run the docker run command in combination with environment variables that you'll add in the subsequent steps. The final command will look similar to:

# Please note that if you create environment variables for secrets, you still need to let Docker
# know that these variables exist by including it via: `-e ENV_VAR_NAME`.
export MODERNE_AGENT_CRYPTO_SYMMETRICKEY=...
export MODERNE_AGENT_TOKEN=...

docker run \
# Example environment variables. These will be explained in step 4.
-e MODERNE_AGENT_APIGATEWAYRSOCKETURI=https://api.tenant.moderne.io/rsocket \
-e MODERNE_AGENT_CRYPTO_SYMMETRICKEY \
-e MODERNE_AGENT_NICKNAME=prod-1 \
-e MODERNE_AGENT_TOKEN \
# ... Additional environment variables
-p 8080:8080
moderne.azurecr.io/moderne-dev/moderne/moderne-agent:latest

Step 4: Configure the agent with the core variables/arguments

All agents must be configured with the variables listed as required below:

Variables:

  • MODERNE_AGENT_APIGATEWAYRSOCKETURI(Required) The URI used to connect to the Moderne API, provided by Moderne.

  • MODERNE_AGENT_CRYPTO_SYMMETRICKEY(Required) A 256-bit AES encryption key, hex encoded. Used to encrypt your artifacts.

  • MODERNE_AGENT_NICKNAME(Required) A name used to identify your agent in the SaaS agent dashboard UI.

  • MODERNE_AGENT_TOKEN(Required) The Moderne SaaS agent connection token, provided by Moderne.

  • MODERNE_AGENT_DOWNLOADPARALLELISM(Optional) How many threads are used to download LSTs. Defaults to 2 threads.

  • MODERNE_AGENT_ARTIFACTINDEXINTERVALSECONDS(Optional) How frequently LSTs will be indexed. Defaults to 120 seconds.

  • MODERNE_AGENT_DEFAULTCOMMITOPTIONS_{index}(Optional) Use to restrict which commit options are available on a tenant level (if the organizations service doesn't return any). Acceptable values: Direct, Branch, Fork, PullRequest, ForkAndPullRequest. By default all options are available.

Example:

# Please note that if you create environment variables for secrets, you still need to let Docker
# know that these variables exist by including it via: `-e ENV_VAR_NAME`.
export MODERNE_AGENT_CRYPTO_SYMMETRICKEY=...
export MODERNE_AGENT_TOKEN=...

docker run \
-e MODERNE_AGENT_APIGATEWAYRSOCKETURI=https://api.tenant.moderne.io/rsocket \
-e MODERNE_AGENT_CRYPTO_SYMMETRICKEY \
-e MODERNE_AGENT_NICKNAME=prod-1 \
-e MODERNE_AGENT_TOKEN \
-e MODERNE_AGENT_DEFAULTCOMMITOPTIONS_0=PullRequest \
-e MODERNE_AGENT_DEFAULTCOMMITOPTIONS_1=ForkAndPullRequest \
# ... Additional variables
-p 8080:8080
moderne.azurecr.io/moderne-dev/moderne/moderne-agent:latest

Step 5: Configure the agent to work with your SCM(s)

Connecting the agent to your SCM enables Moderne to display recipe results in the UI and commit changes from recipes back to your SCM (in the form of PRs, forks, commits, etc).

For every SCM that you want to connect to Moderne, please follow the instructions in the following guides. These guides will explain how to configure an SCM to talk to the Moderne agent and they will provide you with a list of variables to add to the agent run command. You can configure one agent with multiple SCMs.

SCM configuration:

Below is an example of what an agent run command might look like at the end of this step.

# Please note that if you create environment variables for secrets, you still need to let Docker
# know that these variables exist by including it via: `-e ENV_VAR_NAME`.
export MODERNE_AGENT_CRYPTO_SYMMETRICKEY=...
export MODERNE_AGENT_TOKEN=...
export MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTID=...
export MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTSECRET=...

docker run \
-e MODERNE_AGENT_APIGATEWAYRSOCKETURI=https://api.tenant.moderne.io/rsocket \
-e MODERNE_AGENT_CRYPTO_SYMMETRICKEY \
-e MODERNE_AGENT_NICKNAME=prod-1 \
-e MODERNE_AGENT_TOKEN \
-e MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTID \
-e MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTSECRET \
-e MODERNE_AGENT_GITHUB_0_URL=https://myorg.github.com \
-e MODERNE_AGENT_GITHUB_0_ALLOWABLE_ORGANIZATIONS_0=moderne \
-e MODERNE_AGENT_GITHUB_0_ALLOWABLE_ORGANIZATIONS_1=openrewrite \
-e MODERNE_AGENT_GITHUB_0_OAUTH_INCLUDEPRIVATEREPOS=true \
# ... Additional variables to come
-p 8080:8080
moderne.azurecr.io/moderne-dev/moderne/moderne-agent:latest

Step 6: Configure the agent to connect to your artifact repositories

The Moderne agent needs to connect to your artifact repositories for two reasons:

  1. To obtain your LST artifacts so that recipes can be run on your code.

  2. To obtain your recipe artifacts (if any exist). These recipe artifacts contain custom recipes, defined by your team, that perform transformations against your LST artifacts.

Your company might have many artifact repositories, potentially in different products, that you wish to connect the Moderne agent to. Each of these artifact repositories could contain LST artifacts, recipe artifacts, or a combination of both. The setup instructions differ based on what product you use to store your artifact repositories and what artifacts you wish to send to Moderne.

The Moderne agent can only talk to Maven formatted artifact repositories. There are a variety of open-source and commercial products that exist that can serve artifacts in this format (such as Artifactory and Sonatype Nexus). A single instance of one of these products may contain multiple Maven repositories.

Moderne offers two options for connecting to your artifact repository: a generic Maven connection that can connect to any Maven formatted repository regardless of vendor and an Artifactory-specific connection that is optimized to serve LST artifacts more quickly.

If you do not plan on using Artifactory to store LST or recipe artifacts, please follow the Maven repository configuration instructions and then jump to Step 7.

If you do plan on using Artifactory to store artifacts, you have two options:

  1. Use the Artifactory LST configuration instructions to set up a connection that serves LST artifacts to Moderne. Then, if you plan on creating custom recipes, you would follow the Artifactory recipe configuration instructions to set up a connection in Artifactory to serve recipe artifacts. (recommended)

  2. Use the Maven repository configuration instructions to set up a connection that serves both LST artifacts and recipe artifacts to Moderne. This is not recommended as LST artifacts will have a considerable delay between being published and showing up in Moderne. However, if for some reason you can not use AQL queries, this approach is necessary.

The below table shows the key differences between the two types of configuration:

Maven repository configuration

Artifactory repository configuration

Is not tied to a particular vendor.

Can only be used with Artifactory.

Serves BOTH recipe artifacts and LST artifacts.

Serves ONLY LST artifacts. Requires Maven configuration to serve recipe artifacts.

Recipe artifacts are immediately available for deployment to Moderne upon publishing to the Maven formatted repository.

Can not serve recipe artifacts without Maven configuration.

LST artifacts may be served if an index in the Maven Indexer format is regularly published to the repository. There will be a considerable delay between when an LST is published to the Maven repository and when it shows up in Moderne. This delay is approximately the delay between updates to the index – which is controlled by a batch process that your artifact repository executes on a schedule.

LST artifacts will show up in near-real time (within a minute or two) in the Moderne platform when they are published to Artifactory. This is because Artifactory configuration uses Artifactory Query Language (AQL) to identify recently published artifacts. AQL queries Artifactory's internal relational database for information about artifacts rather than using an index produced in a batch process.

Please ensure you've followed either the Maven or Artifactory instructions before continuing.

Below is an example of what an agent run command might look like at the end of this step.

# Please note that if you create environment variables for secrets, you still need to let Docker
# know that these variables exist by including it via: `-e ENV_VAR_NAME`.
export MODERNE_AGENT_CRYPTO_SYMMETRICKEY=...
export MODERNE_AGENT_TOKEN=...
export MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTID=...
export MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTSECRET=...
export MODERNE_AGENT_ARTIFACTORY_0_USERNAME=...
export MODERNE_AGENT_ARTIFACTORY_0_PASSWORD=...
export MODERNE_AGENT_MAVEN_0_USERNAME=...
export MODERNE_AGENT_MAVEN_0_PASSWORD=...

docker run \
-e MODERNE_AGENT_APIGATEWAYRSOCKETURI=https://api.tenant.moderne.io/rsocket \
-e MODERNE_AGENT_CRYPTO_SYMMETRICKEY \
-e MODERNE_AGENT_NICKNAME=prod-1 \
-e MODERNE_AGENT_TOKEN \
-e MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTID \
-e MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTSECRET \
-e MODERNE_AGENT_GITHUB_0_URL=https://myorg.github.com \
-e MODERNE_AGENT_GITHUB_0_ALLOWABLE_ORGANIZATIONS_0=moderne \
-e MODERNE_AGENT_GITHUB_0_ALLOWABLE_ORGANIZATIONS_1=openrewrite \
-e MODERNE_AGENT_GITHUB_0_OAUTH_INCLUDEPRIVATEREPOS=true \
-e MODERNE_AGENT_ARTIFACTORY_0_URL=https://myartifactory.example.com/artifactory/ \
-e MODERNE_AGENT_ARTIFACTORY_0_USERNAME \
-e MODERNE_AGENT_ARTIFACTORY_0_PASSWORD \
-e MODERNE_AGENT_ARTIFACTORY_0_ASTQUERYFILTERS_0='"name":{"$match":"*-ast.jar"}' \
-e MODERNE_AGENT_ARTIFACTORY_0_ASTQUERYFILTERS_1='"repo":{"$eq":"example-maven"}' \
-e MODERNE_AGENT_MAVEN_0_URL=https://myartifactory.example.com/artifactory/libs-releases-local \
-e MODERNE_AGENT_MAVEN_0_LOCALREPOSITORY=~/.moderne-maven \
-e MODERNE_AGENT_MAVEN_0_USERNAME \
-e MODERNE_AGENT_MAVEN_0_PASSWORD \
# ... Additional variables to come
-p 8080:8080
moderne.azurecr.io/moderne-dev/moderne/moderne-agent:latest

Step 7: (Optionally) Configure the Organizations service

Many organizations desire the ability to control the organizational structure of their repositories within the Moderne platform in a dynamic way. To facilitate this need, Moderne provides an optional integration with an Organizations service that is hosted inside of your environment.

If you want to set up this service, please follow these instructions. Then, once it has been set up, please configure the agent by following these instructions.

Below is an example of what an agent run command might look like at the end of this step if you set up the Organizations service.

# Please note that if you create environment variables for secrets, you still need to let Docker
# know that these variables exist by including it via: `-e ENV_VAR_NAME`.
export MODERNE_AGENT_CRYPTO_SYMMETRICKEY=...
export MODERNE_AGENT_TOKEN=...
export MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTID=...
export MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTSECRET=...
export MODERNE_AGENT_ARTIFACTORY_0_USERNAME=...
export MODERNE_AGENT_ARTIFACTORY_0_PASSWORD=...
export MODERNE_AGENT_MAVEN_0_USERNAME=...
export MODERNE_AGENT_MAVEN_0_PASSWORD=...

docker run \
-e MODERNE_AGENT_APIGATEWAYRSOCKETURI=https://api.tenant.moderne.io/rsocket \
-e MODERNE_AGENT_CRYPTO_SYMMETRICKEY \
-e MODERNE_AGENT_NICKNAME=prod-1 \
-e MODERNE_AGENT_TOKEN \
-e MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTID \
-e MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTSECRET \
-e MODERNE_AGENT_GITHUB_0_URL=https://myorg.github.com \
-e MODERNE_AGENT_GITHUB_0_ALLOWABLE_ORGANIZATIONS_0=moderne \
-e MODERNE_AGENT_GITHUB_0_ALLOWABLE_ORGANIZATIONS_1=openrewrite \
-e MODERNE_AGENT_GITHUB_0_OAUTH_INCLUDEPRIVATEREPOS=true \
-e MODERNE_AGENT_ARTIFACTORY_0_URL=https://myartifactory.example.com/artifactory/ \
-e MODERNE_AGENT_ARTIFACTORY_0_USERNAME \
-e MODERNE_AGENT_ARTIFACTORY_0_PASSWORD \
-e MODERNE_AGENT_ARTIFACTORY_0_ASTQUERYFILTERS_0='"name":{"$match":"*-ast.jar"}' \
-e MODERNE_AGENT_ARTIFACTORY_0_ASTQUERYFILTERS_1='"repo":{"$eq":"example-maven"}' \
-e MODERNE_AGENT_MAVEN_0_URL=https://myartifactory.example.com/artifactory/libs-releases-local \
-e MODERNE_AGENT_MAVEN_0_LOCALREPOSITORY=~/.moderne-maven \
-e MODERNE_AGENT_MAVEN_0_USERNAME \
-e MODERNE_AGENT_MAVEN_0_PASSWORD \
-e MODERNE_AGENT_ORGANIZATION_URL=http://localhost:8091 \
-e MODERNE_AGENT_ORGANIZATION_UPDATE_INTERVAL_SECONDS=600 \
-p 8080:8080
moderne.azurecr.io/moderne-dev/moderne/moderne-agent:latest

Step 8: (Optionally) Use strict recipe sources.

Some organizations want recipe artifacts to only come from locations configured in the Moderne agent. If you want to configure that, please follow the strict recipe sources instructions.

Below is an example of what an agent run command might look like at the end of this step if you configured the agent to use only configured recipe sources.

# Please note that if you create environment variables for secrets, you still need to let Docker
# know that these variables exist by including it via: `-e ENV_VAR_NAME`.
export MODERNE_AGENT_CRYPTO_SYMMETRICKEY=...
export MODERNE_AGENT_TOKEN=...
export MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTID=...
export MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTSECRET=...
export MODERNE_AGENT_ARTIFACTORY_0_USERNAME=...
export MODERNE_AGENT_ARTIFACTORY_0_PASSWORD=...
export MODERNE_AGENT_MAVEN_0_USERNAME=...
export MODERNE_AGENT_MAVEN_0_PASSWORD=...

docker run \
-e MODERNE_AGENT_APIGATEWAYRSOCKETURI=https://api.tenant.moderne.io/rsocket \
-e MODERNE_AGENT_CRYPTO_SYMMETRICKEY \
-e MODERNE_AGENT_NICKNAME=prod-1 \
-e MODERNE_AGENT_TOKEN \
-e MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTID \
-e MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTSECRET \
-e MODERNE_AGENT_GITHUB_0_URL=https://myorg.github.com \
-e MODERNE_AGENT_GITHUB_0_ALLOWABLE_ORGANIZATIONS_0=moderne \
-e MODERNE_AGENT_GITHUB_0_ALLOWABLE_ORGANIZATIONS_1=openrewrite \
-e MODERNE_AGENT_GITHUB_0_OAUTH_INCLUDEPRIVATEREPOS=true \
-e MODERNE_AGENT_ARTIFACTORY_0_URL=https://myartifactory.example.com/artifactory/ \
-e MODERNE_AGENT_ARTIFACTORY_0_USERNAME \
-e MODERNE_AGENT_ARTIFACTORY_0_PASSWORD \
-e MODERNE_AGENT_ARTIFACTORY_0_ASTQUERYFILTERS_0='"name":{"$match":"*-ast.jar"}' \
-e MODERNE_AGENT_ARTIFACTORY_0_ASTQUERYFILTERS_1='"repo":{"$eq":"example-maven"}' \
-e MODERNE_AGENT_MAVEN_0_URL=https://myartifactory.example.com/artifactory/libs-releases-local \
-e MODERNE_AGENT_MAVEN_0_LOCALREPOSITORY=~/.moderne-maven \
-e MODERNE_AGENT_MAVEN_0_USERNAME \
-e MODERNE_AGENT_MAVEN_0_PASSWORD \
-e MODERNE_AGENT_ORGANIZATION_URL=http://localhost:8091 \
-e MODERNE_AGENT_ORGANIZATION_UPDATE_INTERVAL_SECONDS=600 \
-e MODERNE_AGENT_RECIPE_USEONLYCONFIGURED=true \
-p 8080:8080
moderne.azurecr.io/moderne-dev/moderne/moderne-agent:latest

Step 9: (Optionally) Provide SSL client keystore

If you have configured any services that require client SSL certificates (such as Maven or Artifactory), you will need to provide a KeyStore with these certificates. Please follow these instructions to configure the KeyStore.

Step 10: Run the agent

At this point, you should have configured everything needed to run the Moderne agent. If you run into issues running the command, please don't hesitate to reach out.

Below is a table that has instructions for how to run the agent in combination with some examples of the variables/arguments provided in the previous steps:

  1. Log in to the Moderne registry:

docker login -u moderne-tenant -p <password provided by Moderne> moderne.azurecr.io
  1. Pull down the latest agent:

docker pull moderne.azurecr.io/moderne-dev/moderne/moderne-agent:latest
  1. Run the docker run command in combination with all of the environment variables you've added in the previous steps:

# Please note that if you create environment variables for secrets, you still need to let Docker
# know that these variables exist by including it via: `-e ENV_VAR_NAME`.
export MODERNE_AGENT_CRYPTO_SYMMETRICKEY=...
export MODERNE_AGENT_TOKEN=...
export MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTID=...
export MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTSECRET=...
export MODERNE_AGENT_ARTIFACTORY_0_USERNAME=...
export MODERNE_AGENT_ARTIFACTORY_0_PASSWORD=...
export MODERNE_AGENT_MAVEN_0_USERNAME=...
export MODERNE_AGENT_MAVEN_0_PASSWORD=...

docker run \
-e MODERNE_AGENT_APIGATEWAYRSOCKETURI=https://api.tenant.moderne.io/rsocket \
-e MODERNE_AGENT_CRYPTO_SYMMETRICKEY \
-e MODERNE_AGENT_NICKNAME=prod-1 \
-e MODERNE_AGENT_TOKEN \
-e MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTID \
-e MODERNE_AGENT_GITHUB_0_OAUTH_CLIENTSECRET \
-e MODERNE_AGENT_GITHUB_0_URL=https://myorg.github.com \
-e MODERNE_AGENT_GITHUB_0_ALLOWABLE_ORGANIZATIONS_0=moderne \
-e MODERNE_AGENT_GITHUB_0_ALLOWABLE_ORGANIZATIONS_1=openrewrite \
-e MODERNE_AGENT_GITHUB_0_OAUTH_INCLUDEPRIVATEREPOS=true \
-e MODERNE_AGENT_ARTIFACTORY_0_URL=https://myartifactory.example.com/artifactory/ \
-e MODERNE_AGENT_ARTIFACTORY_0_USERNAME \
-e MODERNE_AGENT_ARTIFACTORY_0_PASSWORD \
-e MODERNE_AGENT_ARTIFACTORY_0_ASTQUERYFILTERS_0='"name":{"$match":"*-ast.jar"}' \
-e MODERNE_AGENT_ARTIFACTORY_0_ASTQUERYFILTERS_1='"repo":{"$eq":"example-maven"}' \
-e MODERNE_AGENT_MAVEN_0_URL=https://myartifactory.example.com/artifactory/libs-releases-local \
-e MODERNE_AGENT_MAVEN_0_LOCALREPOSITORY=~/.moderne-maven \
-e MODERNE_AGENT_MAVEN_0_USERNAME \
-e MODERNE_AGENT_MAVEN_0_PASSWORD \
-e MODERNE_AGENT_ORGANIZATION_URL=http://localhost:8091 \
-e MODERNE_AGENT_ORGANIZATION_UPDATE_INTERVAL_SECONDS=600 \
-p 8080:8080
moderne.azurecr.io/moderne-dev/moderne/moderne-agent:latest

Updating your agent

If you want to update the Moderne agent over time, please follow the instructions in the table below:

If you're running the commands provided in this guide, you should see that the last line of every agent run command is moderne.azurecr.io/moderne-dev/moderne/moderne-agent:latest.

If that's true, then you can simply restart the agent and it should pick up the latest version. If you've decided to pin the version to something else instead of latest, please see our releases page for the versions.

Last updated