[p4] Slow Downs - More RAM?

Wed Jan 10 01:43:19 PST 2007

Hi there,

Our site was hitting the 2GB memory limit on Windows.  The cause of this
was actually a memory leak in the server version we were using at the
time.  It was simple to see by plotting the p4d process memory over time
(there is only one process on Windows).  

As a temporary measure we restarted the server every few days.  The bug
was also eventually fixed:

Bugs fixed since 2005.2/88704 (beta)

	#101271 **
	    Memory leak fixed.  A server running with spec depot enabled
or
	    additional local depots would leak memory.  Typically this
would
	    be 20 bytes for each additional depot multiplied by the
number
	    of file arguments.  (Bug 21127).

After the fix, the server ran for about 50 days non-stop (still watching
the memory use at this time).  But we still decided to move to Linux
where the process-per-request model mops up any leaks that could occur.
Since then we have had uptime in excess of 100 days with much better
performance overall, although we still see occasional lockups where a
large branch, integrate or submit can take locks a block all other
requests for upto 10 minutes.

Personally I think the problem is that Perforce only does locking at a
table level.  If it were to use row level locking, I think it could get
a lot more concurrency, although it would probably be at the cost of a
more complex server.

Cheers,

Mike

-----Original Message-----
From: perforce-user-bounces at perforce.com
[mailto:perforce-user-bounces at perforce.com] On Behalf Of Robert Cowham
Sent: 08 January 2007 10:28
To: 'Terry Metler'; perforce-user at perforce.com
Subject: Re: [p4] Slow Downs - More RAM?

I know of at least a couple of larger sites running on Windows that have
had
performance problems, and indeed stability problems.

Perforce now say the following:

http://www.perforce.com/perforce/technotes/note077.html

Windows Version Limitations

The Windows 32-bit platform (including Windows 2000 and 32 bit versions
of
Windows 2003 and XP) has a 2GB per-process memory utilization limit. On
Windows, the Perforce Server runs as a single process, servicing each
client
request as a thread within that process. For sites with a very large
transaction volume, this 2GB limitation can inhibit performance and
large
operations. The Windows 64-bit platform does not have this memory
limitation. The use of a 64-bit version of Windows is preferable for
sites
that might encounter the 2GB per process limitation of the 32-bit
version.  

One site cured their issues by a move to Linux, and another site is
moving
to Unix but has not yet fixed on platform. That said, database locks can
always cause backups whatever the platform, so you get approaches like
Google presented at the European conference to avoid those.

Robert

> -----Original Message-----
> From: perforce-user-bounces at perforce.com 
> [mailto:perforce-user-bounces at perforce.com] On Behalf Of Terry Metler
> Sent: 08 January 2007 02:38
> To: perforce-user at perforce.com
> Subject: [p4] Slow Downs - More RAM?
> 
> I need opinions about hardware and how "big" I need to go...
> 
>  
> 
> We've been running Perforce now for about 7 months.  Up to 
> November everything was running very smoothly but since then 
> we have many slow downs a day.  There are no indications on 
> the system except that I have noticed just before everything 
> goes back to normal that the cores all peg to 100% 
> utilization for a few seconds.  This has slowly gotten worse 
> to now it is happening two or three times a day.
> 
>  
> 
> After talking with Perforce support they indicated that it 
> looks like a case that the server is running out of 
> resources.  There are points when there are over 700 
> concurrent transactions.  Support mentioned that it looks 
> like a large process is backing everything up and eventually 
> it does finish and everything else runs really fast that was 
> backed up behind that one process.  
> 
>  
> 
> I've rebuilt the databases, defragged the drives, removed the 
> audit file and moved the database files to a separate drive 
> to try and get some additional performance.  None of these 
> have made a significant improvement.
> 
>  
> 
> Currently...
> 
> Users: 160
> 
> Files: 460 000 (not including all revisions)
> 
> Server
> 
>             CPU: 2 x 3.0Ghz Xeon
> 
>             RAM: 4GB
> 
>             OS: Windows Server 2003 Standard
> 
> New server I'm looking at...
> 
> CPU: 2 x 2.6Ghz Opteron
> 
> RAM: 32GB (Can be upgraded to 64GB)
> 
> OS: Windows Server 2003 Enterprise
> 
> I am hoping to hear from some people that have been using 
> Perforce for a lot longer than we have.  I don't want to 
> purchase a bigger server and not get the performance I'm 
> being told I would get with "a lot more" RAM (the current one 
> is almost maxed out physically and is maxed with the Standard 
> OS).  All recommendations are more than welcomed as I want to 
> have a better feeling that it is the hardware that is causing 
> the slow downs before shelling out for a new server.
> 
> Thanks in advance for your recommendations and feedback!!
> 
> Terry
_______________________________________________
perforce-user mailing list  -  perforce-user at perforce.com
http://maillist.perforce.com/mailman/listinfo/perforce-user