Mixed language pipeline

This example shows a simple Nextflow pipeline consisting of two processes written in different languages.

#!/usr/bin/env nextflow

/*
 * Pipeline parameters
 */

// Range
params.range = 100

/*
 * A trivial Perl script that produces a list of number pairs
 */
process perlTask {

    input:
    val x

    output:
    stdout

    shell:
    '''
    #!/usr/bin/env perl
    use strict;
    use warnings;

    my $count;
    my $range = !{x};
    for ($count = 0; $count < 10; $count++) {
        print rand($range) . ', ' . rand($range) . "\n";
    }
    '''
}

/*
 * A Python script which parses the output of the previous script
 */
process pyTask {

    input:
    stdin

    output:
    stdout

    script:
    """
    #!/usr/bin/env python
    import sys

    x = 0
    y = 0
    lines = 0
    for line in sys.stdin:
        items = line.strip().split(",")
        x += float(items[0])
        y += float(items[1])
        lines += 1

    print("avg: %s - %s" % ( x/lines, y/lines ))
    """
}

workflow {

    // Creates channel using the Channel.of() channel factory
    range_ch = Channel.of(params.range)

    // A Perl script that produces a list of number pairs
    perlTask(range_ch)

    // A Python script which parses the output of the previous script
    pyTask(perlTask.out)

    // View pyTask output
    pyTask.out.view()
}

Synopsis

This example shows a simple Nextflow pipeline consisting of two processes written in different languages. The perlTask process starts with a Perl shebang declaration and executes a Perl script that produces pairs of numbers. Since Perl uses the $ character for variables, the special shell block is used instead of the normal script block to distinguish the Perl variables from Nextflow variables. Similarly, the pyTask process starts with a Python shebang declaration. It takes the output from the Perl script and executes a Python script that averages the number pairs. The output from the pyTask process is then printed to screen.

Try it

To try this pipeline:

  1. Follow the Nextflow installation guide to install Nextflow (if not already available).

  2. Copy the script above and save it as mixed-languages.nf.

  3. Launch the pipeline:

    nextflow run mixed-languages.nf

NOTE: To run this example with versions of Nextflow older than 22.04.0, you must include the -dsl2 flag with nextflow run.