[PLUG] perl advice (forking, lists problems)
Ben Prew
bprew at logiccloud.com
Wed Oct 23 19:50:17 UTC 2002
Hey Russ,
Here are some changes I would make...
<comments below>
gilmanhunt at attbi.com wrote:
>Good morning!
>
>I'm working on this perl script. I'm not very good (yet) at Perl, but
>I'm getting there. Most of my learning has been at the hands of
>the Perl Cookbook (Bighorn Ram) and Programming Perl
>(Camel). I've got this perl script that is replacing another script.
>
>What I need to do is process a file of 30000 lines; and normally
>to do it takes a very long time (The process takes a long time,
>not the perl part). I can break it down into 6 5000 line files and it
>goes much faster. Currently I'm running 6 copies of the script
>concurrently each with its own input file:
>$ script.1 &
>$ script.2 &
>$ script.3 & (etc.)
>
>That's clumsy and it won't work well for my next step without a
>rewrite (which will require processing on the main data file)
>And since I'm rewriting it anyway ...
>
>Here's what I have now:
>----------
>#!/usr/bin/perl
>
#turn on warning
#!/usr/bin/perl -w
# I always use strict...
use strict;
>
># sub: handleme
># input: index for the $lists array.
># output: whatever the function does.
># use: processes the @lists[$reference] array
># with these items.
>sub handleme
>{
>
# I'm not quite sure what this function is doing... and, you won't be
able to access @lists inside this sub...
# you can shift here... then you're not required to make a list for a
single value
my $reference = shift;
# or
my $reference = $_[0];
> my ($reference) = @_;
>
# but, if you could, @lists is a list of lists... so
my $list_ref = @lists[$reference];
print "$reference has " . @$list_ref . " elements \n";
> print "$reference has " . @lists[$reference]. " elements\n";
> sleep 2;
>}
>
>
># set up our array of array references.
># these first two arrays are sub-arrays for
># the main "lists" array.
>
#now that you're using strict, you'll have to my all these variables.. ala
my @list1;
my @list2;
my @lists = (\@list1, \@list2);
>@list1=();
>@list2=();
>@lists=(\@list1,\@list2);
>
># splittest.data contains 1\n2\n etc through 20
># live functionality has 30000 lines.
>
>open BIGDOG, "< splittest.data";
>while (<BIGDOG>)
>{
> chomp;
> push(@bigarray, $_);
>}
>close BIGDOG;
>
my $file = "splittest.data";
open (BIGDOG, $file) or
die("Err opening $file: $!\n");
#slurps the file into the array
@bigarray = <BIGDOG>;
close (BIGDOG);
#chomps all elements in @bigarray
chomp @bigarray;
#If you're only splitting into 2 lists, I would recommend something like
this
for(my $i; $i < ($#bigarray / 2); $i++) {
push @list1, shift(@bigarray);
}
#then
@list2 = @bigarray;
#and I'm sure there's an even better way to do that, but I can't think
of the way off-hand.
>while (scalar (@bigarray) > 0)
>{
> push @list1, shift(@bigarray);
> push @list2, shift(@bigarray);
>}
> print "data acquired\n";
>
># now list1 and list2 contain alternating lines of the
># original file. The alternating doesn't matter.
>
>
># this is the main processing area.
># i is the index of @lists that we're working with
># and it's also the number of children.
>
#this will have to by my'ed as well.. also, perl defaults the value to
0, so you don't need to specify it.
my $i ;
>$i=0;
>FORK:
>{
> if ($pid= fork)
>
# should be (note the 2 equals signs)
if( $pid == fork)
>
> {
> # parent here
> # this is currently testing against 1;
> # in development, we want 6 iterations not 2
> # it's the last index of the array
> # and it's the maximum children.
> while ($i<1)
> {
> print " $i - father stuff\n";
> $i++;
> redo FORK;
> }
> }
> elsif (defined $pid)
> {
> print "\tchild process begin\n";
> handleme $i;
> print "\tchild process end\n";
> }
> elsif ($! =~ /No more process/)
> {
> sleep 5;
> redo FORK;
> }
> else
> {
> die "ack! $! \n";
> }
>}
>
>-----------
>
>However, in the function, I can't access the arrays; with a
>"foreach $item (@array) {print "$item\n"}, I get "ARRAY(code)".
>If I ask "what am I doing wrong?" I'm sure you'll say "oh geez,
>where do we begin" so perhaps ... "How do I reference the
>arrays in @lists, so I can see the items of the lists in lists ?"
>
>Your help is appreciated :)
>
>-Russ
>
>_______________________________________________
>PLUG mailing list
>PLUG at lists.pdxlinux.org
>http://lists.pdxlinux.org/mailman/listinfo/plug
>
>
>
More information about the PLUG
mailing list