Updating-duet

Table of Contents

• 5.1 Updating the duet pipeline and D(h)MR workflow
• 5.2 Manual update biomodal CLI script from 1.0.x to 1.1.3

(back to main documentation)

5.1. Updating the duet pipeline and D(h)MR workflow

To check for available duet pipeline versions you can run:

biomodal list

This will list all available versions of the duet pipeline software. Please be aware that some may be older than your current local version.

To install a newer version of the duet pipeline software please run:

biomodal download <version>

Running this command will download and install the specific version of the duet pipeline software you have requested. If you have already downloaded the same version, this command will overwrite all the files for the exsting local version. Other existing local duet installations will be left untouched. The D(h)MR workflow will be updated with the duet pipeline software.

Please note that the command biomodal info will show available new versions of the CLI and duet pipeline software.

(back to main documentation) | (back to top)

5.2. Manual update biomodal CLI script from 1.0.x to 1.1.3

If you choose not to use the provided installation scripts to update the CLI and duet pipeline software, you can manually update to CLI 1.1.3 and duet 1.4.1, or newer, by following the steps below. There is not any 3rd party software to install for this update, so there is not a requirement to install new versions of OS software packages or bootstrap any new enviroments. However, you have to download the new CLI installer to access the required files. Please see 1.1. Download and unzip the complete set of installation and CLI scripts.

Please note that you may have to have sudo rights to perform some of these steps. Please reach out to your system administrator if you are unsure.

• Copy the following new files from the CLI 1.1.3 download into the biomodal installation folder. The scripts are located in the ./cli folder where you unzipped the CLI download. Your installation folder was determined during your earlier install, essentially it is the same location chosen for the biomodal script. On HPC this may be a software module, so please make sure to liaise with your system administrator if you are unsure. Please note that some files will be overwritten. The relevant files you have to copy from the downloaded ./cli folder to the local installation folder are:

  • _biomodal_functions.sh
  • _biomodal_validate.sh
  • biomodal

• Update the two existing error.config templates files in the local installation folder:

  • failfast.error.config

    HPC or local version:

    process {
    errorStrategy = { sleep(Math.pow(2, task.attempt) as long); return task.exitStatus in [14] ? 'retry' : 'terminate' }
    }

    Cloud version:

    process {
    errorStrategy = { task.attempt.toString().isInteger() ? sleep(Math.pow(2, task.attempt) as long) : sleep(1) ; return task.exitStatus in [0,10,14,50001,50002] ? 'retry' : 'terminate' }
    maxRetries    = 1
    }

  • retry.error.config

    HPC or local version:

    process {
    errorStrategy = { task.attempt.toString().isInteger() ? sleep(Math.pow(2, task.attempt) as long) : sleep(1) ; return task.exitStatus in [14] ? 'retry' : 'ignore' }
    maxRetries    = 10
    }

    Cloud version:

    process {
    errorStrategy = { task.attempt.toString().isInteger() ? sleep(Math.pow(2, task.attempt) as long) : sleep(1) ; return task.exitStatus in [0,10,14,50001,50002] ? 'retry' : 'ignore' }
    maxRetries    = 10
    }

• Ensure you can execute the scripts by running the following command in the local installation folder:

   chmod +x biomodal _biomodal_functions.sh _biomodal_validate.sh

• Update duet and local artifact registry by running the following command:

   biomodal download 1.4.1

• Update containers, reference data and reconfirm your local settings by running the following command:

   biomodal init

• You should edit the existing local config.yaml file, found in the same local installation folder you copied the CLI script into. Please do not copy the newly downloaded config.yaml file, but instead edit the existing one. Add the following line to the config.yaml file:

   ref_pipeline_version: 1.0.1

• You have to add a new report overwrite flag to the CLI nextflow.config file in the same folder you copied the script into. This is necessary to avoid the error message “Report file already exists: <path>/diagnostics/nf_report.html -- enable the 'report.overwrite' option in your config file to overwrite existing files” when running the pipeline. Add the following lines to the nextflow.config file:

  report {
    overwrite = true
  }

You should now be able to run a duet pipeline test using the latest versions of the CLI and pipeline.

   biomodal test

(back to main documentation) | (back to top) | (Next)