[PLUG] perl advice (forking, lists problems)

Ben Prew bprew at logiccloud.com
Wed Oct 23 19:50:17 UTC 2002


Hey Russ,

Here are some changes I would make...

<comments below>

gilmanhunt at attbi.com wrote:

>Good morning! 
>
>I'm working on this perl script. I'm not very good (yet) at Perl, but 
>I'm getting there. Most of my learning has been at the hands of 
>the Perl Cookbook (Bighorn Ram) and Programming Perl 
>(Camel).  I've got this perl script that is replacing another script.
>
>What I need to do is process a file of 30000 lines; and normally 
>to do it takes a very long time (The process takes a long time, 
>not the perl part). I can break it down into 6 5000 line files and it 
>goes much faster.  Currently I'm running 6 copies of the script 
>concurrently each with its own input file:
>$ script.1 &
>$ script.2 &
>$ script.3 & (etc.)
>
>That's clumsy and it won't work well for my next step without a 
>rewrite (which will require processing on the main data file) 
>And since I'm rewriting it anyway ... 
>
>Here's what I have now:
>----------
>#!/usr/bin/perl
>
#turn on warning

#!/usr/bin/perl -w

# I always use strict...

use strict;

>
># sub: handleme
># input: index for the $lists array.
># output: whatever the function does.
># use: processes the @lists[$reference] array
># with these items.
>sub handleme
>{
>
# I'm not quite sure what this function is doing... and, you won't be 
able to access @lists inside this sub...
# you can shift here... then you're not required to make a list for a 
single value
my $reference = shift;
# or
my $reference = $_[0];

>  my ($reference) = @_;
>
# but, if you could, @lists is a list of lists... so
my $list_ref = @lists[$reference];
print "$reference has " . @$list_ref  . " elements \n";

>  print "$reference has " . @lists[$reference]. " elements\n";
>  sleep 2;
>}
>
>
># set up our array of array references.
># these first two arrays are sub-arrays for 
># the main "lists" array.
>
#now that you're using strict, you'll have to my all these variables.. ala

my @list1;
my @list2;
my @lists = (\@list1, \@list2);

>@list1=();
>@list2=();
>@lists=(\@list1,\@list2);
>
># splittest.data contains 1\n2\n etc through 20
># live functionality has 30000 lines.
>
>open BIGDOG, "< splittest.data";
>while (<BIGDOG>)
>{
>  chomp;
>  push(@bigarray, $_);
>}
>close BIGDOG;
>
my $file = "splittest.data";
open (BIGDOG, $file) or
    die("Err opening $file: $!\n");

#slurps the file into the array
@bigarray = <BIGDOG>;

close (BIGDOG);
#chomps all elements in @bigarray
chomp @bigarray;

#If you're only splitting into 2 lists, I would recommend something like 
this

for(my $i; $i < ($#bigarray / 2); $i++) {
    push @list1, shift(@bigarray);
}
#then
@list2 = @bigarray;

#and I'm sure there's an even better way to do that, but I can't think 
of the way off-hand.

>while (scalar (@bigarray) > 0)
>{ 
>  push @list1, shift(@bigarray);
>  push @list2, shift(@bigarray);
>}
> print "data acquired\n";
>
># now list1 and list2 contain alternating lines of the 
># original file. The alternating doesn't matter.
>
>
># this is the main processing area.
># i is the index of @lists that we're working with
># and it's also the number of children. 
>
#this will have to by my'ed as well.. also, perl defaults the value to 
0, so you don't need to specify it.
my $i ;

>$i=0;
>FORK:
>{
>  if ($pid= fork)
>
# should be (note the 2 equals signs)
if( $pid == fork)

> 
>  { 
>    # parent here
>    # this is currently testing against 1; 
>    # in development, we want 6 iterations not 2
>    # it's the last index of the array
>    # and it's the maximum children.
>    while ($i<1)
>    {
>    print " $i - father stuff\n";
>    $i++;
>    redo FORK;
>    }
>  }
>  elsif (defined $pid)
>  {
>    print "\tchild process begin\n";
>    handleme $i;
>    print "\tchild process end\n";
>  }
>  elsif ($! =~ /No more process/)
>  {
>  sleep 5;
>  redo FORK;
>  }
>  else
>  {
>   die "ack! $! \n";
>  }
>}
>
>-----------
>
>However, in the function, I can't access the arrays; with a 
>"foreach $item (@array) {print "$item\n"}, I get "ARRAY(code)".
>If I ask "what am I doing wrong?" I'm sure you'll say "oh geez, 
>where do we begin" so perhaps ... "How do I reference the 
>arrays in @lists, so I can see the items of the lists in lists ?"
>
>Your help is appreciated :)
>
>-Russ
>
>_______________________________________________
>PLUG mailing list
>PLUG at lists.pdxlinux.org
>http://lists.pdxlinux.org/mailman/listinfo/plug
>
>  
>








More information about the PLUG mailing list