[p4] Performance of remote perforce depots and proxies

Tue Nov 6 17:08:07 PST 2007

Hi Stanton,

Good to know for our next swapover - we are all on 2007.2.

Latest update - our ISP found and resolved a problem with our upload
speed.  However, without the proxy caching files locally, we've decided
to mirror the data remotely and automate a daily sync / merge between
the two sites.  Our remote colleagues need to locally share files
through the day and the hit of the proxy resyncing the remote file after
another user submits makes it's use impractical right now. 

Thanks for all the info,
Cj

-----Original Message-----
From: Stanton Stevens [mailto:sstevens at adobe.com] 
Sent: Wednesday, 7 November 2007 5:54 AM
To: Craig James; perforce-user at perforce.com
Subject: RE: [p4] Performance of remote perforce depots and proxies

One more note for this thread:

We found when testing new releases that some mismatches of p4p and p4d
releases are particularly bad. For instance, a 2004.2 p4p with a 2006.1
p4d combination took 100x longer to return output to commands like "p4
groups". So it is good to be careful to use the same rev of p4p as p4d. 

Stanton 

-----Original Message-----
From: perforce-user-bounces at perforce.com
[mailto:perforce-user-bounces at perforce.com] On Behalf Of steve at vance.com
Sent: Thursday, November 01, 2007 9:26 PM
To: craig.james at thq.com; perforce-user at perforce.com
Subject: Re: [p4] Performance of remote perforce depots and proxies

Yes, you are correct on the behavior of the proxy. In general, all
requests pass through the proxy. Commands that request files are broken
up to first request the metadata. The proxy determines which revisions
it has and then requests the content of revisions it doesn't have from
the server. This amounts to two trips from the proxy to the server if it
doesn't have the revision cached. At that point, it returns the results
to the client.

All of the proxy-server traffic occurs over the WAN and is compressed
unless you explicitly turn that off. The client-proxy traffic happens
over the LAN and should never be a bottleneck except in a local outage.
Unless a user bypasses the proxy, you should never have direct
client-server communication, so that does not need to be optimized.

You are correct about the improvement. And I'm guessing that a 250 MB
file is binary, not a ktext type that would need refreshing.

Steve

Original Message:
-----------------
From: Craig James Craig.James at thq.com
Date: 	Fri, 2 Nov 2007 14:08:49 +1000
To: steve at vance.com, perforce-user at perforce.com
Subject: RE: [p4] Performance of remote perforce depots and proxies

Thanks Steve,

If I read you right... Even though it doesn't cache the file locally,
submits still pass through the proxy and it copies the data back to the
remote server (in compressed transfers).

Makes sense then it wouldn't have much affect - Tim really only
mentioned it as another thing.

Even with the client remaining locked in 2007.3, still a definite
improvement having that file available as soon as it is submitted -
instead of waiting for the proxy to re-cache it (250MB files, ouch)

I am wanting to talk more with our ISP, we're on a pretty expensive plan
and my understanding is that 250ms ping to the US is pretty average
(even being down under).  I'll keep in mind your further comments
though.

Thanks,
Cj

-----Original Message-----
From: steve at vance.com [mailto:steve at vance.com]
Sent: Friday, 2 November 2007 1:56 PM
To: Craig James; perforce-user at perforce.com
Subject: RE: [p4] Performance of remote perforce depots and proxies

No problem, Craig. Tim is good and would be awake now. I just happen to
be near your time zone right now.

Don't forget to also work with your ISP to tune the packet parameters
there. I wouldn't be as concerned with your client parameters except to
the extent that they match the parameters on your server. Your main
issue will be the traffic between the proxy and the remote server.

As for the compress flags, I don't see how that would help much. The
communication between the proxy and the server is compressed by default.
I
assume you haven't turned that off. Since your communication with the
proxy is over a local network and compression of metadata rarely is
worth the CPU, I don't see the benefit, but Tim usually has good reasons
for his recommendations, so I may be missing something.

The upcoming of caching local submits still won't release the client
before the submit is finished. It will just prevent having to make
another request for the file before it is brought back. It's not clear
to me how it will behave if you have RCS keywords turned on, since that
requires a client file refresh anyway. The submittor will probably incur
the penalty.

The last release of the year is usually squeezed in right around Dec 31.
:)
But since they have it in beta right now, I would hope for sooner.

Steve

Original Message:
-----------------
From: Craig James Craig.James at thq.com
Date: 	Fri, 2 Nov 2007 13:24:22 +1000
To: steve at vance.com, perforce-user at perforce.com
Subject: RE: [p4] Performance of remote perforce depots and proxies

Thanks for the comments Steve,

Perforce local support also came back to be very quickly (thanks Tim)
and had similar advice to offer.

I'm going to play with "DefaultSendWindow" in the registry
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Afd\Parameters\ on
server and client - based on advice here:
http://www-didc.lbl.gov/TCP-tuning/

	Buffer size = 2 * bandwidth * delay 

Which for us is:

	2 * 10Mbit * 0.250 / 8 = 625000 bytes

There was also the suggestion we turn on the compress flag in each
clientspec (off by default).

Tim also noted that the current proxy does not cache local submits to
remote servers but that this is coming in 2007.3 (currently in beta).

We aren't going to swap to this yet (2007.3 final release possibly early
next year), so will likely have submits coordinated on a single machine
that submits all changes overnight.

Cj

--------------------------------------------------------------------
mail2web.com - Enhanced email for the mobile individual based on
Microsoft(r)
Exchange - http://link.mail2web.com/Personal/EnhancedEmail

--------------------------------------------------------------------
mail2web.com - Enhanced email for the mobile individual based on
Microsoft(r) Exchange - http://link.mail2web.com/Personal/EnhancedEmail

_______________________________________________
perforce-user mailing list  -  perforce-user at perforce.com
http://maillist.perforce.com/mailman/listinfo/perforce-user