Source operations

On Platform.sh, you can run automated code updates through a feature called source operations. Defined in your app configuration, source operations let you specify commands that can commit changes to your project’s repository when called.

For example, you can set up a source operation to automatically update your application dependencies, update a site from an upstream repository, or revert to the last commit pushed to your Git repository.

To run your source operations, you can use the Platform.sh CLI or the Console. If you want to run your source operations and update your code automatically, you can also define cron jobs.

How source operations work

When you trigger a source operation, the following happens in order:

1. The current environment HEAD commit is checked out in Git. It doesn’t have any remotes or tags defined in the project. It only has the current environment branch.

2. Sequentially, for each app that has an operation bearing the name of the triggered source operation in its configuration, the operation command is run in the app container. The container isn’t part of the environment’s runtime cluster and doesn’t require that the environment is running.

   The environment has all of the variables normally available during the build phase. These may be optionally overridden by the variables specified when the operation is run.

3. If any new commits were created, they’re pushed to the repository and the normal build process is triggered.

   If multiple apps in a single project both result in a new commit, there are two distinct commits in the Git history but only a single new build process.

Define a source operation

A source operation requires two things:

• A name that must be unique within the application. The name is the key of the block defined under source.operations in your app configuration.

• A command that defines what’s run when the operation is triggered.

The syntax is similar to the following:

source:
  operations:
    SOURCE_OPERATION_NAME:
      command: COMMAND

For example, to update a file from a remote location, you could define an operation like this:

source:
  operations:
    update-file:
      command: |
        set -e
        curl -O https://example.com/myfile.txt
        git add myfile.txt
        git commit -m "Update remote file"        

The name of the source operation in this case is update-file.

For more possibilities, see other source operation examples.

Run a source operation

platform source-operation:run SOURCE_OPERATION_NAME

After running a source operation, to apply the changes to your local development environment run the git pull command.

Note that you can cancel pending or running source operations.

Use variables in your source operations

You can add variables to the environment of the source operation.

Use the env: prefix to expose each of those variables as a Unix environment variable. In this way, they’re referenced by the source operation and interpreted the same way as any other variable set in your project.

For example, you might want to have a FILE variable available with the value example.txt to pass to a source operation similar to the following:

source:
  operations:
    update-file:
      command: |
        set -e
        curl -O https://example.com/$FILE
        git add $FILE
        git commit -m "Update remote file"        

Follow these steps to run the source operation:

platform source-operation:run update-file --variable env:FILE="example.txt"

Source integrations

If your project is using a source integration, any new commits resulting from a source operation are first pushed to your external Git repository. Then the source integration pushes those commits to Platform.sh and redeploys the environment.

When using a source integration, you can’t run source operations on environments created from pull or merge requests created on the external repository.

If you try running a source operation on a non-supported environment, you see the following error:

[ApiFeatureMissingException]
This project doesn't support source operations.

Automated source operations using a cron job

You can use a cron to automatically run your source operations.

Note that it’s best not to run source operations on your production environment, but rather on a dedicated environment where you can test changes.

Make sure you have the Platform.sh CLI installed and an API token so you can run a cron job in your app container.

1. Set your API token as a top-level environment variable:

platform variable:create --environment main --level environment --prefix 'env' --name PLATFORMSH_CLI_TOKEN --sensitive true --value 'YOUR_PLATFORMSH_CLI_TOKEN' --inheritable false --visible-build true --json false --enabled true --visible-runtime true

2. Add a build hook to your app configuration to install the CLI as part of the build process:

hooks:
  build: |
    set -e
    echo "Installing Platform.sh CLI"
    curl -fsSL https://raw.githubusercontent.com/platformsh/cli/main/installer.sh | bash

    echo "Testing Platform.sh CLI"
    platform    

3. Then, to configure a cron job to automatically run a source operation once a day, use a configuration similar to the following:

crons:
  update:
    # Run the code below every day at midnight.
    spec: '0 0 * * *'
    commands:
      start: |
        set -e
        platform sync -e development code data --no-wait --yes
        platform source-operation:run update-file --no-wait --yes        

The example above synchronizes the development environment with its parent and then runs the update-file source operation defined previously.

Source operation examples

Update your application dependencies

You can set up a source operation and a cron job to automate your dependency updates.

Update a site from an upstream repository or template

The following source operation syncronizes your branch with an upstream Git repository.

1. Add a project-level variable named env:UPSTREAM_REMOTE with the Git URL of the upstream repository. That makes that repository available as a Unix environment variable in all environments, including in the source operation’s environment.

   • Variable name: env:UPSTREAM_REMOTE
   • Variable example value: https://github.com/platformsh/platformsh-docs

2. In your app configuration, define a source operation to fetch from that upstream repository:

source:
  operations:
    upstream-update:
      command: |
        set -e
        git remote add upstream $UPSTREAM_REMOTE
        git fetch --all
        git merge upstream/main        

3. Now every time you run the upstream-update operation on a given branch, the branch fetches all changes from the upstream git repository and then merges the latest changes from the default branch in the upstream repository. If there’s a conflict merging from the upstream repository, the source operation fails and doesn’t update from the upstream repository.

Run the upstream-update operation on a preview environment rather than directly on Production.

Revert to the last commit

The following source operation reverts the last commit pushed to the Git repository. This can be useful if you didn’t properly test the changes of another operation and you need to quickly revert to the previous state.

source:
  operations:
    revert:
      command: |
        git reset --hard HEAD~        

Now every time you run the revert operation on a given branch, the operation reverts to the last commit pushed to that branch.

Update Drupal Core

The following source operation uses Composer to update Drupal Core:

source:
  operations:
    update-drupal-core:
      command: |
        set -e
        composer update drupal/core --with-dependencies
        git add composer.lock
        git commit -m "Automated Drupal Core update."        

--with-dependencies is used to also update Drupal Core dependencies. Read more on how to update Drupal Core via Composer on Drupal.org.

Now every time you run the update-drupal-core operation, it updates Drupal Core.

Download a Drupal extension

The following source operation downloads a Drupal extension. You can define the Drupal extension by setting an EXTENSION variable or overriding it when running the source operation.

source:
  operations:
    download-drupal-extension:
      command: |
        set -e
        composer require $EXTENSION
        git add composer.json
        git commit -am "Automated install of: $EXTENSION via Composer."        

Now every time you run the download-drupal-extension operation, it downloads the defined extension.

If it’s a new extension, after the source operation finishes, you need to enable the new extension via the Drupal management interface or using Drush.

Update Git submodules

The following source operation updates all Git submodules recursively:

source:
  operations:
    rebuild:
      command: |
        set -e
        git submodule update --init --recursive
        git submodule update --remote --checkout
        SHA=$(git submodule | awk -F' ' '{print $1}' | sed -s 's/+//g')
        echo -n "$SHA" > .sha
        git add uppler .sha
        git commit -m "Updating submodule to commit '$SHA'"        

Now every time you run the rebuild operation, it updates the Git submodules.