[p4] Managing diskspace usage of Perforce Proxy

[Frank Compagner]

Well, the proxy machines are currently just older dev machines that
were about to be taken apart, but one of our admins figured he could use
them for a perforce proxy experiment. With the experiment succesful,
we're now looking at putting in more of them, with bigger harddisks,
so they should fill up more slowly. Still, we're now looking to
increase the number of proxy's to around 10, and with the depot size
now growing at about 25GB a week, we'd need about 1TB of diskspace per
proxy to prevent frequent manual cleanups. Diskspace might be cheap,
but it's not entirely free yet.

But the main point is that a proxy server should be almost completely
maintenace free. In the current situation, if the admin forgets to
check the proxy regularly (happened once or twice), the proxy fills up,
and perforce stops working for the users on that proxy. This is just
stupid, and I want something more low maintenance.

----------------------------------------------------------------
Frank Compagner                                  Guerrilla Games

MJ> At the risk of sounding like an idiot/simpleton, I suggest you get more
MJ> storage capacity. Disk space is *cheap*. You can't even buy a PC these
MJ> days with 200G or less in it. Where did your IT department get these
MJ> proxy servers? :)

MJ> The small cost of expanding disk capacity will be better than having
MJ> developers that are annoyed because Perforce works great on Monday, but
MJ> slowly degrades as the week wears on.


MJ> -Matt


MJ> -----Original Message-----
MJ> From: perforce-user-bounces at perforce.com
MJ> [mailto:perforce-user-bounces at perforce.com] On Behalf Of Jeff Grills
MJ> Sent: Wednesday, May 23, 2007 6:29 AM
MJ> To: 'Frank Compagner'; 'Perforce Users'
MJ> Subject: Re: [p4] Managing diskspace usage of Perforce Proxy


MJ> Are you running on top of a filesystem which records access time for
MJ> files?
MJ> If so, you could run a cron job which removes files that haven't been
MJ> accessed in N days.  If you're on a Unix style OS for your proxies,
MJ> "find
MJ> -atime" will be useful.

MJ> j


MJ> -----Original Message-----
MJ> From: perforce-user-bounces at perforce.com
MJ> [mailto:perforce-user-bounces at perforce.com] On Behalf Of Frank Compagner
MJ> Sent: Monday, May 21, 2007 5:04 PM
MJ> To: Perforce Users
MJ> Subject: [p4] Managing diskspace usage of Perforce Proxy


MJ> Hi,

MJ> we've got a problem with diskspace usage of our proxy servers, and I'd
MJ> like
MJ> to know if anybody else has had to deal with this before, and if so,
MJ> what
MJ> they did about it. Let me first explain the problem:

MJ> We have a central Perforce server, serving some 150 users and about 20
MJ> build
MJ> machines. Depot size is about 2TB. To increase performance and server
MJ> stability, we recently deployed 4 Perforce Proxy's throughout the
MJ> building,
MJ> 3 serving some 50 users and the final one for the buildmachines. This
MJ> has
MJ> worked well, performance and stability have increased noticeably.

MJ> However, the proxy servers have a disk capacity of about 200GB, which
MJ> fills
MJ> up in about a week. Sofar our admins are manually deleting the entire
MJ> contents of the proxy server at the end of each week, after which they
MJ> start
MJ> to fill up again. This is too messy and high maintenance, and I'm
MJ> looking
MJ> for a better alternative. Here's some possible solutions I've come up
MJ> with:

MJ> - As the bulk of the data is in large binary compressed files, where
MJ> most
MJ> people only want the head revision, I could run a script that will walk
MJ> through the proxy cache every night, and from every ,d directory delete
MJ> all
MJ> files but the most recent one.

MJ> - Not all branches/projects are equal; some are very heavily used by
MJ> many
MJ> people, others are only used by a few. I could make a shortlist of
MJ> "privileged" parts of the depot, and if diskspace becomes low, zap
MJ> everything in the proxy cache that comes from outside those parts.

MJ> - If, after doing the above, diskspace is still low, go through the
MJ> entire
MJ> proxy cache and randomly delete (say) 50% of the files.

MJ> It won't be hard to put all of this in a python script and run it every
MJ> night, but it all feels rather ad-hoc, so I was wondering if anybody had
MJ> some better ideas.

MJ> Let me know what you think,
MJ> Frank.
MJ> ----------------------------------------------------------------
MJ> Frank Compagner                                  Guerrilla Games

MJ> _______________________________________________
MJ> perforce-user mailing list  -  perforce-user at perforce.com
MJ> http://maillist.perforce.com/mailman/listinfo/perforce-user

MJ> _______________________________________________
MJ> perforce-user mailing list  -  perforce-user at perforce.com
MJ> http://maillist.perforce.com/mailman/listinfo/perforce-user