[p4] What actually happens during checkpoint

Jamie.Echlin at barclayscapital.com Jamie.Echlin at barclayscapital.com
Thu Mar 5 08:50:25 PST 2009


Thanks Geoff, that's useful. We have 32G ram which we know is
inadequate, and have got more on the way.

OK, so the implication here is that after doing a restore, actually
doing the checkpoint should be much quicker.

> some such thing.  My server has 1800 users and in practice we 
> rebalance our database every four months or so.

I've decided anyway to schedule maintenance windows quarterly, for
database rebuilds. For various reasons perforce has not had much admin
love lately, and we have rebuilt it for perhaps 9 months or a year. We
have 2300 users, and over half a million changelists.

To Jeff Smith - it's Linux RHEL4, no AV.

Cheers, jamie


> -----Original Message-----
> From: Mantel, Geoffrey [mailto:geoffrey.mantel at sap.com] 
> Sent: 05 March 2009 16:20
> To: Echlin, Jamie: IT (LDN); perforce-user at perforce.com
> Subject: RE: [p4] What actually happens during checkpoint
> 
> Checkpoint duration will degrade over time.  The Perforce 
> database tables are stored in a B-tree format.  Over time, 
> this tree will become unbalanced, as records are inserted and 
> deleted from the tables.
> 
> The checkpoint procedure has to walk the tree and reconstruct 
> a compact version of the database.  Since the tree becomes 
> unbalanced, the checkpoint has to do more and more walking.  
> This is discussed at the bottom of this link:
> 
> http://www.perforce.com/perforce/doc.current/manuals/p4sag/07_
> perftune.h
> tml
> 
> "Rebalancing the trees is normally useful only if the 
> database files have become more than about 10 times the size 
> of the checkpoint.  Given the length of time required for the 
> trees to become unbalanced during normal Perforce use, we 
> expect that the majority of sites will never need to restore 
> the database from a checkpoint (that is, rebalance the
> trees) for performance reasons."
> 
> The first sentence is true.  The second sentence, in my 
> opinion, is false for larger sites.  The sentence should be 
> re-written in terms of the number of check-ins per day, or 
> some such thing.  My server has 1800 users and in practice we 
> rebalance our database every four months or so.
> I know some sites which rebalance on a weekly basis.  And, by 
> the way, "rebalancing the trees" is a fancy way of saying 
> "restoring your DB from a checkpoint".
> 
> Another piece of unsolicited advice -- the more RAM, the 
> better.  Your RAM should be at least the size of your 
> db.have.  You will find that your server performance will 
> jump dramatically once db.have can be entirely cached in RAM. 
>  32GB seems too small for a 104GB database.
> 
> Cheers,
> 
>  -- Geoff
> 
> 
> -----Original Message-----
> From: perforce-user-bounces at perforce.com
> [mailto:perforce-user-bounces at perforce.com] On Behalf Of 
> Jamie.Echlin at barclayscapital.com
> Sent: Thursday, March 05, 2009 7:23 AM
> To: perforce-user at perforce.com
> Subject: Re: [p4] What actually happens during checkpoint
> 
> I should have mentioned we're using ext3, and it appears 
> mounting it with data=writeback would help, and XFS help even 
> more. But even so, I'm currently interested in why this 
> process seems to have degraded so much over the last 6 months 
> (checkpoint used to take about an hour), and for personal 
> curiousity I'm interested in the mechanics of a checkpoint.
> 
> Cheers, jamie 
> 
> > -----Original Message-----
> > From: perforce-user-bounces at perforce.com
> > [mailto:perforce-user-bounces at perforce.com] On Behalf Of Echlin, 
> > Jamie: IT (LDN)
> > Sent: 05 March 2009 14:26
> > To: perforce-user at perforce.com
> > Subject: [p4] What actually happens during checkpoint
> > 
> > Can anyone help me understand what happens during a checkpoint?
> > Background is, the checkpoint procedure has got incredibly 
> slow, about 
> > 9 hours to produce a 4.5 G compressed checkpoint from a 
> 104G database.
> > 
> > We have some problems with our SAN. The LUNS that make up the data 
> > volume are split over two different array controllers, one 
> of them in 
> > a different data centre. However, this shouldn't cause problems in 
> > itself, it's just not "best practice", or so I've been told.
> > 
> > My theory about the performance then was that the checkpoint would 
> > write the transactions in the same order that they were originally 
> > done, causing reads over all the different tables, somehow 
> > exacerbating a disk problem. Also we don't have enough main memory, 
> > only 32G. However, reading more, checkpoints are not the same as 
> > journals, the transactions are not in the same order. But a 
> checkpoint 
> > is also not just a compressed copy of all the tables, if 
> that was true 
> > there was no way it could so long. So what gives? ;-)
> > 
> > A restore of a checkpoint takes 10 hours, so not much difference 
> > between reads and writes. Just to rule out decompressing being the 
> > bottleneck, it's only using a couple per cent of CPU.
> > 
> > Cheers, jamie
> > 
> > 
> > _______________________________________________
> > 
> > This e-mail may contain information that is confidential, 
> privileged 
> > or otherwise protected from disclosure. If you are not an intended 
> > recipient of this e-mail, do not duplicate or redistribute 
> it by any 
> > means. Please delete it and any attachments and notify the 
> sender that 
> > you have received it in error. Unless specifically indicated, this 
> > e-mail is not an offer to buy or sell or a solicitation to 
> buy or sell 
> > any securities, investment products or other financial product or 
> > service, an official confirmation of any transaction, or an 
> official 
> > statement of Barclays. Any views or opinions presented are solely 
> > those of the author and do not necessarily represent those of 
> > Barclays. This e-mail is subject to terms available at the 
> following 
> > link:
> > www.barcap.com/emaildisclaimer. By messaging with Barclays 
> you consent 
> > to the foregoing.  Barclays Capital is the investment 
> banking division 
> > of Barclays Bank PLC, a company registered in England 
> (number 1026167) 
> > with its registered offi!
> >  ce at 1 Churchill Place, London, E14 5HP.  This email may 
> relate to 
> > or be sent from other members of the Barclays Group.
> > _______________________________________________
> > 
> > _______________________________________________
> > perforce-user mailing list  -  perforce-user at perforce.com 
> > http://maillist.perforce.com/mailman/listinfo/perforce-user
> > 
> _______________________________________________
> 
> This e-mail may contain information that is confidential, 
> privileged or otherwise protected from disclosure. If you are 
> not an intended recipient of this e-mail, do not duplicate or 
> redistribute it by any means. Please delete it and any 
> attachments and notify the sender that you have received it 
> in error. Unless specifically indicated, this e-mail is not 
> an offer to buy or sell or a solicitation to buy or sell any 
> securities, investment products or other financial product or 
> service, an official confirmation of any transaction, or an 
> official statement of Barclays. Any views or opinions 
> presented are solely those of the author and do not 
> necessarily represent those of Barclays. This e-mail is 
> subject to terms available at the following link:
> www.barcap.com/emaildisclaimer. By messaging with Barclays 
> you consent to the foregoing.  Barclays Capital is the 
> investment banking division of Barclays Bank PLC, a company 
> registered in England (number 1026167) with its registered offi!
>  ce at 1 Churchill Place, London, E14 5HP.  This email may 
> relate to or be sent from other members of the Barclays Group.
> _______________________________________________
> 
> _______________________________________________
> perforce-user mailing list  -  perforce-user at perforce.com 
> http://maillist.perforce.com/mailman/listinfo/perforce-user
> 
_______________________________________________

This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing.  Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP.  This email may relate to or be sent from other members of the Barclays Group.
_______________________________________________




More information about the perforce-user mailing list