Run Information

When you launch a pipeline, you are navigated to the Runs tab which contains all executed workflows, with your submitted run at the top of the list.

Each new or resumed job is given a random name, such as "grave_williams". Each row corresponds to a specific job. As a job executes, it can transition through the following states:

• submitted: Pending execution
• running: Running
• succeeded: Completed successfully
• failed: Successfully executed, where at least one task failed with a terminate error strategy
• cancelled: Stopped forceably during execution
• unknown: Indeterminate status

Click to show animation

1. View the run details for nf-core/rnaseq

The pipeline launched in the previous step will take some time to begin running. In the meantime, we can take a look at a previous successful run to observe the run details.

Select a previous run to view the Run details page.

2. Run details page

As the pipeline begins to run, you will see the run details populating with the following tabs:

• Command-line: The Nextflow command invocation that would be used to run the pipeline. This contains details about the version (through the -r flag), and profile, if specified (through the -profile flag).

• Parameters: The exact set of parameters used in the execution. This is helpful for reproducing results of a previous run.

• Resolved Nextflow configuration: The full Nextflow configuration settings used for the run. This includes parameters, but also settings specific to task execution (such as memory, CPUs, and output directory).

• Execution Log: A summarized Nextflow log providing information about the pipeline and the status of the run.

• Datasets: Link to datasets, if any were used in the run.

• Reports: View outputs of your pipeline directly in the Platform.

Click to show animation

3. View reports

Most Nextflow pipelines will generate reports or output files which are useful to inspect at the end of the pipeline execution.

Reports can contain quality control (QC) metrics that are important to assess the integrity of the results.

For example, for the nf-core/rnaseq pipeline, you can view the MultiQC report generated. MultiQC is a helpful reporting tool to generate aggregate statistics and summaries from bioinformatics tools.

Notice the path to the file will still point to a location on the user's infrastructure, but we can view the contents and download the file without having to go to the Cloud or a remote filesystem.

Specifying outputs in reports

To customize and instruct Platform where to find reports generated by the pipeline, a YAML file tower.yml that contains the locations of the generated reports must be included in the pipeline repository.

In the nf-core/rnaseq pipeline, the MULTIQC process step generates a MultiQC report file in HTML format.

reports:
  multiqc_report.html:
    display: "MultiQC HTML report"

4. View general information

The run details page includes general information about who executed the run and when, the Git hash and tag used, and additional details about the compute environment and Nextflow version used.

Click to show animation

The 'General' panel displays top-level information about a pipeline run:

• Unique workflow run ID
• Workflow run name
• Timestamp of pipeline start
• Project revision and Git commit ID
• Nextflow session ID
• Username of the launcher
• Work directory path

5. View details for a Task

Scroll down the page and you will see:

• The progress of individual pipeline Processes
• Aggregated stats for the run (total walltime, CPU hours)
• Workflow metrics (CPU efficiency, memory efficiency)
• A Task details table for every task in the workflow

The task details table can provide further information on every step in the pipeline, including task statuses and metrics.

6. Task details window

Select a task in the task table to open the Task details dialog. The dialog has three tabs: About, Execution log and Data Explorer.

About

The About tab provides the following information:

1. Name: Process name and tag

2. Command: Task script, defined in the pipeline process

3. Status: Exit code, task status, number of attempts

4. Work directory: Directory where the task was executed

5. Environment: Environment variables that were supplied to the task

6. Execution time: Metrics for task submission, start, and completion time

7. Resources requested: Metrics for the resources requested by the task

8. Resources used: Metrics for the resources used by the task

Click to show animation

Execution log

The Execution log tab provides a real-time log of the selected task's execution. Task execution and other logs (such as stdout and stderr) are available for download from here, if still available in your compute environment.

7. Task work directory in Data Explorer

If a task fails, a good place to begin troubleshooting is the task's work directory.

Nextflow hash-addresses each task of the pipeline and creates unique directories based on these hashes. Instead of navigating through a bucket on the Cloud console or filesystem to find the contents of this directory, we can make use of the 'Data Explorer' tab in the Task window.

The Data Explorer tab allows you to view the log files and output files generated for each task in it's working directory, directly within Platform.

You can view, download, and retrieve the link for these intermediate files stored in the Cloud from the Explorer tab, making troubleshooting much simpler.

Click to show animation

8. Resume a Pipeline

Seqera Platform enables you to use Nextflow's resume functionality to resume a workflow run with the same parameters, using the cached results of previously completed tasks and only executing failed and pending tasks.

Click to show animation

To resume a failed or cancelled run:

• Select the three dots next to the run
• Select Resume from the options menu
• Edit the parameters before launch, if needed
• If you have the appropriate permissions, you may edit the compute environment if needed