[p4] Performance of remote perforce depots and proxies

Patrick M. Slattery patrickmslattery at mac.com
Wed Nov 7 23:40:25 PST 2007 

You might want to check out the 2007.3 update of P4P that's currently  
in beta, it looks like it solves this double hit problem.

http://www.perforce.com/perforce/doc.073/user/p4pnotes.txt
New features in 2007.3
         #129045 ** ***
	    Proxy will now cache files as they are submitted
	    through the proxy.  A new cache validation system
	    based on MD5 signatures is being used.  If a new
	    proxy is working with an old server the old timestamp
	    validation is used.  Recommend running "p4 verify" to
	    assure that the p4d server has MD5 signatures computed
	    as the proxy will not validate it's cache for revisions
	    which the p4d server does not have an MD5 signature.
	    (Bug #10866)

On Nov 6, 2007, at 8:08 PM, Craig James wrote:

> Hi Stanton,
>
> Good to know for our next swapover - we are all on 2007.2.
>
> Latest update - our ISP found and resolved a problem with our upload
> speed.  However, without the proxy caching files locally, we've  
> decided
> to mirror the data remotely and automate a daily sync / merge between
> the two sites.  Our remote colleagues need to locally share files
> through the day and the hit of the proxy resyncing the remote file  
> after
> another user submits makes it's use impractical right now.
>
> Thanks for all the info,
> Cj
>
> -----Original Message-----
> From: Stanton Stevens [mailto:sstevens at adobe.com]
> Sent: Wednesday, 7 November 2007 5:54 AM
> To: Craig James; perforce-user at perforce.com
> Subject: RE: [p4] Performance of remote perforce depots and proxies
>
> One more note for this thread:
>
> We found when testing new releases that some mismatches of p4p and p4d
> releases are particularly bad. For instance, a 2004.2 p4p with a  
> 2006.1
> p4d combination took 100x longer to return output to commands like "p4
> groups". So it is good to be careful to use the same rev of p4p as  
> p4d.
>
> Stanton
>
> -----Original Message-----
> From: perforce-user-bounces at perforce.com
> [mailto:perforce-user-bounces at perforce.com] On Behalf Of  
> steve at vance.com
> Sent: Thursday, November 01, 2007 9:26 PM
> To: craig.james at thq.com; perforce-user at perforce.com
> Subject: Re: [p4] Performance of remote perforce depots and proxies
>
> Yes, you are correct on the behavior of the proxy. In general, all
> requests pass through the proxy. Commands that request files are  
> broken
> up to first request the metadata. The proxy determines which revisions
> it has and then requests the content of revisions it doesn't have from
> the server. This amounts to two trips from the proxy to the server  
> if it
> doesn't have the revision cached. At that point, it returns the  
> results
> to the client.
>
> All of the proxy-server traffic occurs over the WAN and is compressed
> unless you explicitly turn that off. The client-proxy traffic happens
> over the LAN and should never be a bottleneck except in a local  
> outage.
> Unless a user bypasses the proxy, you should never have direct
> client-server communication, so that does not need to be optimized.
>
> You are correct about the improvement. And I'm guessing that a 250 MB
> file is binary, not a ktext type that would need refreshing.
>
> Steve
>
> Original Message:
> -----------------
> From: Craig James Craig.James at thq.com
> Date: 	Fri, 2 Nov 2007 14:08:49 +1000
> To: steve at vance.com, perforce-user at perforce.com
> Subject: RE: [p4] Performance of remote perforce depots and proxies
>
>
> Thanks Steve,
>
> If I read you right... Even though it doesn't cache the file locally,
> submits still pass through the proxy and it copies the data back to  
> the
> remote server (in compressed transfers).
>
> Makes sense then it wouldn't have much affect - Tim really only
> mentioned it as another thing.
>
> Even with the client remaining locked in 2007.3, still a definite
> improvement having that file available as soon as it is submitted -
> instead of waiting for the proxy to re-cache it (250MB files, ouch)
>
> I am wanting to talk more with our ISP, we're on a pretty expensive  
> plan
> and my understanding is that 250ms ping to the US is pretty average
> (even being down under).  I'll keep in mind your further comments
> though.
>
> Thanks,
> Cj
>
> -----Original Message-----
> From: steve at vance.com [mailto:steve at vance.com]
> Sent: Friday, 2 November 2007 1:56 PM
> To: Craig James; perforce-user at perforce.com
> Subject: RE: [p4] Performance of remote perforce depots and proxies
>
> No problem, Craig. Tim is good and would be awake now. I just  
> happen to
> be near your time zone right now.
>
> Don't forget to also work with your ISP to tune the packet parameters
> there. I wouldn't be as concerned with your client parameters  
> except to
> the extent that they match the parameters on your server. Your main
> issue will be the traffic between the proxy and the remote server.
>
> As for the compress flags, I don't see how that would help much. The
> communication between the proxy and the server is compressed by  
> default.
> I
> assume you haven't turned that off. Since your communication with the
> proxy is over a local network and compression of metadata rarely is
> worth the CPU, I don't see the benefit, but Tim usually has good  
> reasons
> for his recommendations, so I may be missing something.
>
> The upcoming of caching local submits still won't release the client
> before the submit is finished. It will just prevent having to make
> another request for the file before it is brought back. It's not clear
> to me how it will behave if you have RCS keywords turned on, since  
> that
> requires a client file refresh anyway. The submittor will probably  
> incur
> the penalty.
>
> The last release of the year is usually squeezed in right around  
> Dec 31.
> :)
> But since they have it in beta right now, I would hope for sooner.
>
> Steve
>
> Original Message:
> -----------------
> From: Craig James Craig.James at thq.com
> Date: 	Fri, 2 Nov 2007 13:24:22 +1000
> To: steve at vance.com, perforce-user at perforce.com
> Subject: RE: [p4] Performance of remote perforce depots and proxies
>
>
> Thanks for the comments Steve,
>
> Perforce local support also came back to be very quickly (thanks Tim)
> and had similar advice to offer.
>
> I'm going to play with "DefaultSendWindow" in the registry
> HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Afd\Parameters 
> \ on
> server and client - based on advice here:
> http://www-didc.lbl.gov/TCP-tuning/
>
> 	Buffer size = 2 * bandwidth * delay
>
> Which for us is:
>
> 	2 * 10Mbit * 0.250 / 8 = 625000 bytes
>
> There was also the suggestion we turn on the compress flag in each
> clientspec (off by default).
>
>
> Tim also noted that the current proxy does not cache local submits to
> remote servers but that this is coming in 2007.3 (currently in beta).
>
> We aren't going to swap to this yet (2007.3 final release possibly  
> early
> next year), so will likely have submits coordinated on a single  
> machine
> that submits all changes overnight.
>
>
> Cj
>
>
>
> --------------------------------------------------------------------
> mail2web.com - Enhanced email for the mobile individual based on
> Microsoft(r)
> Exchange - http://link.mail2web.com/Personal/EnhancedEmail
>
>
>
> --------------------------------------------------------------------
> mail2web.com - Enhanced email for the mobile individual based on
> Microsoft(r) Exchange - http://link.mail2web.com/Personal/ 
> EnhancedEmail
>
>
> _______________________________________________
> perforce-user mailing list  -  perforce-user at perforce.com
> http://maillist.perforce.com/mailman/listinfo/perforce-user
>
> _______________________________________________
> perforce-user mailing list  -  perforce-user at perforce.com
> http://maillist.perforce.com/mailman/listinfo/perforce-user

More information about the perforce-user mailing list