PERL Threading – Boss/Worker Design Pattern

  • Author:xnite
  • Date:2015/06/04

When threading in PERL it is often useful to have a master process checking up on threaded processes to make sure they are doing their job correctly, or to gather data from the threaded process. At any rate the design pattern that I often find useful is called Boss & Worker. In this pattern we have one ( or more ) threads that act as a boss, and this (or these) thread(s) will spawn child threads which carry out a set task.First, as with any threaded program, we will need to pull in libraries such as threads, and strict:

#!/usr/bin/perl
use strict;
use threads;

As usual, we will want to define our worker and boss sub-routines. For this tutorial I have written a script that uses worker threads to find prime numbers. Believe it or not, there are entire super computers dedicated to this task, but we will be working on a MUCH smaller scale. 😛

First, let’s take a look at the worker thread, this is the simple part where we just have a subroutine check for prime numbers between a given set of integers.

When the worker thread finds a prime number, it will print it out to the screen. Once the thread has processed the given numeric range it will exit, effectively killing the thread.

sub worker
{
        my ($i, $end) = @_;
        while( $i <= $end )
        {
                if( prime( $i ) == 1 )
                {
                        print $i."\n";
                }
                $i++;
        }
        threads->exit();
}

Our boss thread will be responsible for creating blocks of numbers for the worker threads to run through. This process is also straight forward, but there is a little more too it than just generating a list of numbers.

The first thing that we need to bare in mind is that your computer can’t just handle an unlimited amount of threads. We’ll need to limit the number of threads running to something sane, like 250.

The second thing we need to think about is how the boss will know when to stop. In my example we simply have a while loop that stops execution once our $i variable reaches a maximum integer.

Wrapped inside the while loop is an if statement which makes sure we aren’t running too many threads already, and if so will not execute a new thread or increment $i. Effectively, our boss thread will wait for old threads to close before creating new threads.

sub boss
{
      # Generate blocks of numbers and have workers check for primes.
      my $i = 0;
      my @workers = ();
      while ( $i < 9223372036854775000 )
      {
             my @threads = threads->list(threads::running);
             if( $#threads < 250 ) # 250 maximum threads
             {
                   push( @workers, threads->create( \&worker, $i, $i+1000 ) ); # Have worker process a block of 1000 numbers.
                   foreach ( @workers )
                   {
                         if( $_->is_joinable() ) {
                               $_->join();
                         }
                   }
                   $i=$i+1001; # incriment count.
             }
       }
      # Wait until all threads finish.
       my @threads = threads->list(threads::running);
       while($#threads > 0)
       {
             my @threads = threads->list(threads::running);
       }
}

Now let’s see the whole script put together (the prime subroutine is borrowed from a post on PERL monks.):

#!/usr/bin/perl
use strict;
use threads;
 
sub prime
{
        my $number = shift;
        my $d = 2;
        my $sqrt = sqrt $number;
        while( 1 )
        {
                if( $number%$d == 0 )
                {
                        return 0;
                }
                if ($d < $sqrt)
                {
                        $d++;
                } else {
                        return 1;
                }
        }
}
 
sub boss
{
        # Generate blocks of numbers and have workers check for primes.
        my $i = 0;
        my @workers = ();
        while ( $i < 9223372036854775000 )
        {
                my @threads = threads->list(threads::running);
                if( $#threads < 250 ) # 250 maximum threads
                {
                        push( @workers, threads->create( \&worker, $i, $i+1000 ) ); # Have worker process a block of 1000 numbers.
                        foreach ( @workers )
                        {
                                if( $_->is_joinable() ) {
                                        $_->join();
                                }
                        }
                        $i=$i+1001; # incriment count.
                }
        }
       
        # Wait until all threads finish.
        my @threads = threads->list(threads::running);
        while($#threads > 0)
        {
                my @threads = threads->list(threads::running);
        }
}
 
sub worker
{
        my ($i, $end) = @_;
        while( $i <= $end )
        {
                if( prime( $i ) == 1 )
                {
                        print $i."\n";
                }
                $i++;
        }
        threads->exit();
}
 
boss(); # Start boss process.

Is there something wrong in this tutorial, or something that you would like to add? Leave a comment below. 🙂

Robert Whitney
I'm a geek, gamer and breaker of things. I'm a programmer by day and an apache attack helicopter by night. Some would call me their spirit animal.
Opinions expressed here, even 💩 ones, are my own and do not represent those of my employer or associates.
Referral Links

Using my referral links is the best way to help me pay for my projects and development servers and get something out of it for yourself.

Copyright©2011 - 2018, Robert Whitney; All rights reserved.