[p4] P4P Network Performance

Frank Compagner frank.compagner at guerrilla-games.com
Mon Feb 4 16:02:37 PST 2008


We've been using local Perforce proxies to relieve the sync load on
the server as well. I've been doing lots of tests to improve the p4p
throughput, and here's some things I've found so far:
- CPU load on the proxy is strangely high. This is still true while
  syncing files that aren't compressed on the server, or over the
  network (as confirmed with a packet sniffer)
- We have a virus scanner running on the (Windows) proxies and
  disabling this seems to give a slight performance increase, but p4p
  will still consume almost 100% of a single CPU core for a single
  (large) sync.
- Usually the real bottleneck of a sync is client-side file fragmentation
  (all our clients are on NTFS), but even if we completely eliminate
  this factor I can't seem to get p4p throughput above ~20 MByte/sec.
- There is considerable per-file overhead. Not sure exactly where
  (could be the proxy, the client or both), but when syncing normal
  source files (say ~20 KB each) I get considerably less than
  10 MB/sec. Not really a problem for me, as I'm mostly interested in
  much larger files than that, but still relevant.

The last two remarks are equally valid when testing a direct server
connection without a proxy, BTW.

What I find slightly odd in your numbers is that you manage to get 100
MB/sec using ftp; I've never been able to get anything over 35 MB/sec
when writing to a single sata drive. You seem to use pretty fast
client hard disk setups, but even then 100 MB/sec seems rather high.
Maybe the system disk cache makes this possible, but then you have to
make sure it influences both measurements equally.

I would love to understand more about (proxy) sync performance, but
that will take more experimentation.

----------------------------------------------------------------
Frank Compagner                                  Guerrilla Games

CC> Hello all,

CC> Recently our main P4D server has been under increasing IO load. As a
CC> temporary measure I've setup a local P4P server, during testing I
CC> noticed we can't seem to get more than 10MB/sec throughput from the
CC> server, even for files that are in the P4P cache. I've tried various
CC> hardware configurations to rule out it wasn't the initial server's
CC> hardware. Out of the three servers I tested, each peak at around
CC> 10MB/sec.  If I transfer files from the P4P cache on these servers via
CC> FTP or CIFS, then I get nearly 100MB/sec to the same client machines.

CC> To test I fetch a file to force it into the cache, I've verified it is
CC> there by going through the file system and checked. Then I make sure it
CC> is not in the client's workspace, then fetch the file. Every time, its
CC> 10MB/sec. if I remove the file, and sync that file again I get the same
CC> speed. I've verified via -Zproxyverbose that it is coming from the
CC> proxy's cache.

CC> If anyone could offer advice on speeding it up I'd appreciate it
CC> greatly.

CC> BTW, here are the specs on the various machines I tested with

CC> Client #1:
CC>         4GB of memory
CC>         2 DualCore AMD @ 2.8Ghz
CC>         2 x 250GB sata drives in a RAID 0 array
CC>         Running Windows Vista.
CC>         Networking is a single gigabit link to the network.

CC> Client #2
CC>         2 Xeons @ 2.8GHz 
CC>         4GB of memory
CC>         2 x 146GB scsi drives in a RAID 1 array
CC>         Running Ubuntu 6.06LTS
CC>         Networking is a single gigabit link to the network.

CC> Server #1:
CC>         2 Xeons @ 2.8GHz 
CC>         4GB of memory
CC>         2 x 146GB scsi drives in a RAID 1 array
CC>         Running Windows 2003 R2 x64
CC>         Networking is a single gigabit link to the network.

CC> Server #2:
CC>         2 Xeons @ 2.8GHz 
CC>         4GB of memory
CC>         2 x 146GB scsi drives in a RAID 0 stripe array
CC>         Running Windows 2003 R2 x64
CC>         Networking is a single gigabit link to the network.

CC> Server #3:
CC>         2 x AMD Opteron 250 @ 2.3GHz
CC>         2GB of memory
CC>         4 x 250 sat drives in a RAID 0 strip array.
CC>         Running Ubuntu 6.06LTS
CC>         Networking is 3 x 1gbit links for user facing traffic
CC>                 one gbit link for talking to main P4D server.

CC> All these were connected to our core network switch, which is a Cisco
CC> Catalyst 4510R. 



CC> Thanks for any advice that can be offered.

CC> Chad

CC> _______________________________________________
CC> perforce-user mailing list  -  perforce-user at perforce.com
CC> http://maillist.perforce.com/mailman/listinfo/perforce-user



More information about the perforce-user mailing list