[p4] Perforce server lockup problem

Bruce McPeek bruce.mcpeek at vidiator.com
Tue Mar 8 09:03:49 PST 2005


Erik,
 
To me this, this sounds more like a hardware issue under load. I am
especially suspicious of the SATA RAID 5.
 
Could you describe the hardware upgrades you mentioned? How is your SATA
RAID 5 configured? Hardware RAID or software RAID? How many drives of
what size? Even better which models. How are you doing your SATA? On
motherboard or add-on card? Are your SATA drivers native windows? Third
party?
 
I need to look at how SATA interfaces with the rest of a system's I/O
again but I'm wondering if this may be your bottleneck.
 
I agree with the other posters about the anti-virus. If it is installed,
how is it configured with respect to what is scanned?
 
I just noticed you are at Valve Software. What are the typical sizes of
the files you are working with? Large binaries for games?
 
 
Bruce
 

  _____  

From: Erik Johnson [mailto:erik at valvesoftware.com] 
Sent: Monday, March 07, 2005 11:22 AM
To: perforce-user at perforce.com
Subject: [p4] Perforce server lockup problem


I'll try and give as much data on our setup, along with the problem
we've been having. I haven't gotten any really crisp leads from Perforce
support on this problem. Maybe someone else has already solved it.
 
We're running our Perforce server on Windows 2003 server on a machine
with 2GB RAM, a RAID 5 disk subsystem with 7200RPM SATA drives, and a
single 3.GHz HT processor. We're running server version
P4D/NTX86/2004.2/73359 (2004/12/27).
 
Our general workflow (and the one that tends to generate the problem) is
that we have a main branch that few people directly work on, with
personal branches off of it that individual developers integrate into.
There are roughly 20 developers integrating roughly 5,000 lines of code
a day. Unfortunately, we changed a couple of variables at once when the
problem started happening (upgraded server software, server hardware),
so I can't reasonably point to a specific root cause.
 
When a developer is merging from their personal branch into our main
codeline, we're seeing total database lockup, and constant reads on the
server (viewed via Windows perfmon). This means that all other users on
the system cannot sync, integrate, checkout, checkin, etc. The condition
generally takes around 15 minutes to clear, and then things all go back
to normal. CPU and RAM usage on the system is all nominal, and while the
reads on the disk subsystem are constant, they are not within 1/3 of the
observed peak read throughput.
 
Has anyone else seen behavior like this? It appears that the database is
being read table by table for some reason.
 
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.perforce.com/pipermail/perforce-user/attachments/20050308/d1b62367/attachment-0007.html>


More information about the perforce-user mailing list