[p4perl] P4.pm performance concerns

[Luke Petre]

I have a script that potentially can sync a very large number of files
and so I'd like to display progress in a friendly manner to the user.

Basically I take a list of directories I need to sync, do a sync preview
to figure out how many files there are, sync those files and update a
percentage along the way.

But I'm worried that calling $client->Sync for each file will be
prohibitably expensive, vs. calling Sync once and passing an array of
all the files in.

My code (pardon the formatting):

sub SyncDirectories
{
  my ($directories, $slow) = @_;

  my $start_time = time;

  # open a connection to perforce
  my $p4 = new P4;
  $p4->Connect() or die( "\n * Failed to connect to Perforce\n" );

  # preview sync
  my @files_to_sync = $p4->Run( "sync", "-n", @$directories );

  # get a list of files to sync
  @files_to_sync = map { /(.*\#\d+) -/; $1; } @files_to_sync;

  if ( $slow )
  {
    # sync each file, updating the percentage along the way
    print( "\n o Sync Progress: " );

    my $percentage = 0;
    print( sprintf( "%3d%%", $percentage ) );

    my $count = 0;
    foreach my $file ( @files_to_sync )
    {
      $p4->Run( "sync", $file );
      
      my $new_percentage = int ( ($count++ / @files_to_sync) * 100 );
      if ( $new_percentage > $percentage )
      {
        $percentage = $new_percentage;

        print( "\010\010\010\010" );
        print( sprintf( "%3d%%", $percentage ) );
      }
    }
    
    print( "\010\010\010\010" );
    print( sprintf( "100%%\n", $percentage ) );
  }
  elsif ( @files_to_sync > 0 )
  {
    # sync files wholesale
    $p4->Run( "sync", @files_to_sync );
  }

  $p4->Disconnect();

  my $total_time = time - $start_time;
  my $num_files = @files_to_sync;
  print( "\n o SyncDirectories got $num_files files, took  $total_time
seconds\n" );
}

I ran this function in a loop on the same set of files, syncing those
files to revision 0 in between calls to SyncDirectories.  The set of
files is about 500KB, and I have compress turned on in my clientspec. 

with slow = 0:
 o SyncDirectories got 97 files, took 0 seconds
 o SyncDirectories got 97 files, took 1 seconds
 o SyncDirectories got 97 files, took 1 seconds
 o SyncDirectories got 97 files, took 0 seconds
 o SyncDirectories got 97 files, took 0 seconds

with slow = 1:
 o SyncDirectories got 97 files, took 22 seconds
 o SyncDirectories got 97 files, took 21 seconds
 o SyncDirectories got 97 files, took 21 seconds
 o SyncDirectories got 97 files, took 23 seconds
 o SyncDirectories got 97 files, took 22 seconds


With a larger set of files (253MB):

with slow = 0:
 o SyncDirectories got 630 files, took 46 seconds
 o SyncDirectories got 630 files, took 47 seconds
 o SyncDirectories got 630 files, took 46 seconds
 o SyncDirectories got 630 files, took 47 seconds
 o SyncDirectories got 630 files, took 45 seconds

with slow = 1:
 o SyncDirectories got 630 files, took 163 seconds
 o SyncDirectories got 630 files, took 162 seconds
 o SyncDirectories got 630 files, took 157 seconds
 o SyncDirectories got 630 files, took 162 seconds
 o SyncDirectories got 630 files, took 161 seconds

At this point I'm considering calling "p4 -x <tmpfile> sync" twice, and
parsing the output.  Only because I expect that I'll get both the
performance and the nice progress display that I want.

I'm completely open to suggestions on how better to use the module.  

If I were to request a new feature I guess I would like to be able to do
something like:
  $p4->SetOutput( \&HandleOutput );
  $p4->Run( "sync", @files_to_sync );

And handle each line of the output to update the percentage.  Thoughts?

Luke