[p4] What actually happens during checkpoint

Smith, Jeff jsmith at medplus.com
Thu Mar 5 10:25:41 PST 2009


If there is anyway that you can do a local test to take the SAN out of
the equation it would be helpful.  Set up a second server on that box
with the depot on local disk and see if that takes as long.  Of course,
you'd have to have that much disk somewhere.  Alternately, you could do
some write tests between local disk and the SAN to see if other writes
to the SAN are slow.

Jeff

-----Original Message-----
From: perforce-user-bounces at perforce.com
[mailto:perforce-user-bounces at perforce.com] On Behalf Of
Jamie.Echlin at barclayscapital.com
Sent: Thursday, March 05, 2009 11:50 AM
To: geoffrey.mantel at sap.com; perforce-user at perforce.com
Subject: Re: [p4] What actually happens during checkpoint

Thanks Geoff, that's useful. We have 32G ram which we know is
inadequate, and have got more on the way.

OK, so the implication here is that after doing a restore, actually
doing the checkpoint should be much quicker.

> some such thing.  My server has 1800 users and in practice we 
> rebalance our database every four months or so.

I've decided anyway to schedule maintenance windows quarterly, for
database rebuilds. For various reasons perforce has not had much admin
love lately, and we have rebuilt it for perhaps 9 months or a year. We
have 2300 users, and over half a million changelists.

To Jeff Smith - it's Linux RHEL4, no AV.

Cheers, jamie


> -----Original Message-----
> From: Mantel, Geoffrey [mailto:geoffrey.mantel at sap.com]
> Sent: 05 March 2009 16:20
> To: Echlin, Jamie: IT (LDN); perforce-user at perforce.com
> Subject: RE: [p4] What actually happens during checkpoint
> 
> Checkpoint duration will degrade over time.  The Perforce database 
> tables are stored in a B-tree format.  Over time, this tree will 
> become unbalanced, as records are inserted and deleted from the 
> tables.
> 
> The checkpoint procedure has to walk the tree and reconstruct a 
> compact version of the database.  Since the tree becomes unbalanced, 
> the checkpoint has to do more and more walking.
> This is discussed at the bottom of this link:
> 
> http://www.perforce.com/perforce/doc.current/manuals/p4sag/07_
> perftune.h
> tml
> 
> "Rebalancing the trees is normally useful only if the database files 
> have become more than about 10 times the size of the checkpoint.  
> Given the length of time required for the trees to become unbalanced 
> during normal Perforce use, we expect that the majority of sites will 
> never need to restore the database from a checkpoint (that is, 
> rebalance the
> trees) for performance reasons."
> 
> The first sentence is true.  The second sentence, in my opinion, is 
> false for larger sites.  The sentence should be re-written in terms of

> the number of check-ins per day, or some such thing.  My server has 
> 1800 users and in practice we rebalance our database every four months

> or so.
> I know some sites which rebalance on a weekly basis.  And, by the way,

> "rebalancing the trees" is a fancy way of saying "restoring your DB 
> from a checkpoint".
> 
> Another piece of unsolicited advice -- the more RAM, the better.  Your

> RAM should be at least the size of your db.have.  You will find that 
> your server performance will jump dramatically once db.have can be 
> entirely cached in RAM.
>  32GB seems too small for a 104GB database.
> 
> Cheers,
> 
>  -- Geoff
> 
> 
> -----Original Message-----
> From: perforce-user-bounces at perforce.com
> [mailto:perforce-user-bounces at perforce.com] On Behalf Of 
> Jamie.Echlin at barclayscapital.com
> Sent: Thursday, March 05, 2009 7:23 AM
> To: perforce-user at perforce.com
> Subject: Re: [p4] What actually happens during checkpoint
> 
> I should have mentioned we're using ext3, and it appears mounting it 
> with data=writeback would help, and XFS help even more. But even so, 
> I'm currently interested in why this process seems to have degraded so

> much over the last 6 months (checkpoint used to take about an hour), 
> and for personal curiousity I'm interested in the mechanics of a 
> checkpoint.
> 
> Cheers, jamie
> 
> > -----Original Message-----
> > From: perforce-user-bounces at perforce.com
> > [mailto:perforce-user-bounces at perforce.com] On Behalf Of Echlin,
> > Jamie: IT (LDN)
> > Sent: 05 March 2009 14:26
> > To: perforce-user at perforce.com
> > Subject: [p4] What actually happens during checkpoint
> > 
> > Can anyone help me understand what happens during a checkpoint?
> > Background is, the checkpoint procedure has got incredibly
> slow, about
> > 9 hours to produce a 4.5 G compressed checkpoint from a
> 104G database.
> > 
> > We have some problems with our SAN. The LUNS that make up the data 
> > volume are split over two different array controllers, one
> of them in
> > a different data centre. However, this shouldn't cause problems in 
> > itself, it's just not "best practice", or so I've been told.
> > 
> > My theory about the performance then was that the checkpoint would 
> > write the transactions in the same order that they were originally 
> > done, causing reads over all the different tables, somehow 
> > exacerbating a disk problem. Also we don't have enough main memory, 
> > only 32G. However, reading more, checkpoints are not the same as 
> > journals, the transactions are not in the same order. But a
> checkpoint
> > is also not just a compressed copy of all the tables, if
> that was true
> > there was no way it could so long. So what gives? ;-)
> > 
> > A restore of a checkpoint takes 10 hours, so not much difference 
> > between reads and writes. Just to rule out decompressing being the 
> > bottleneck, it's only using a couple per cent of CPU.
> > 
> > Cheers, jamie
> > 
> > 
> > _______________________________________________
> > 
> > This e-mail may contain information that is confidential,
> privileged
> > or otherwise protected from disclosure. If you are not an intended 
> > recipient of this e-mail, do not duplicate or redistribute
> it by any
> > means. Please delete it and any attachments and notify the
> sender that
> > you have received it in error. Unless specifically indicated, this 
> > e-mail is not an offer to buy or sell or a solicitation to
> buy or sell
> > any securities, investment products or other financial product or 
> > service, an official confirmation of any transaction, or an
> official
> > statement of Barclays. Any views or opinions presented are solely 
> > those of the author and do not necessarily represent those of 
> > Barclays. This e-mail is subject to terms available at the
> following
> > link:
> > www.barcap.com/emaildisclaimer. By messaging with Barclays
> you consent
> > to the foregoing.  Barclays Capital is the investment
> banking division
> > of Barclays Bank PLC, a company registered in England
> (number 1026167)
> > with its registered offi!
> >  ce at 1 Churchill Place, London, E14 5HP.  This email may
> relate to
> > or be sent from other members of the Barclays Group.
> > _______________________________________________
> > 
> > _______________________________________________
> > perforce-user mailing list  -  perforce-user at perforce.com 
> > http://maillist.perforce.com/mailman/listinfo/perforce-user
> > 
> _______________________________________________
> 
> This e-mail may contain information that is confidential, privileged 
> or otherwise protected from disclosure. If you are not an intended 
> recipient of this e-mail, do not duplicate or redistribute it by any 
> means. Please delete it and any attachments and notify the sender that

> you have received it in error. Unless specifically indicated, this 
> e-mail is not an offer to buy or sell or a solicitation to buy or sell

> any securities, investment products or other financial product or 
> service, an official confirmation of any transaction, or an official 
> statement of Barclays. Any views or opinions presented are solely 
> those of the author and do not necessarily represent those of 
> Barclays. This e-mail is subject to terms available at the following 
> link:
> www.barcap.com/emaildisclaimer. By messaging with Barclays you consent

> to the foregoing.  Barclays Capital is the investment banking division

> of Barclays Bank PLC, a company registered in England (number 1026167)

> with its registered offi!
>  ce at 1 Churchill Place, London, E14 5HP.  This email may relate to 
> or be sent from other members of the Barclays Group.
> _______________________________________________
> 
> _______________________________________________
> perforce-user mailing list  -  perforce-user at perforce.com 
> http://maillist.perforce.com/mailman/listinfo/perforce-user
> 
_______________________________________________

This e-mail may contain information that is confidential, privileged or
otherwise protected from disclosure. If you are not an intended
recipient of this e-mail, do not duplicate or redistribute it by any
means. Please delete it and any attachments and notify the sender that
you have received it in error. Unless specifically indicated, this
e-mail is not an offer to buy or sell or a solicitation to buy or sell
any securities, investment products or other financial product or
service, an official confirmation of any transaction, or an official
statement of Barclays. Any views or opinions presented are solely those
of the author and do not necessarily represent those of Barclays. This
e-mail is subject to terms available at the following link:
www.barcap.com/emaildisclaimer. By messaging with Barclays you consent
to the foregoing.  Barclays Capital is the investment banking division
of Barclays Bank PLC, a company registered in England (number 1026167)
with its registered offi!
 ce at 1 Churchill Place, London, E14 5HP.  This email may relate to or
be sent from other members of the Barclays Group.
_______________________________________________

_______________________________________________
perforce-user mailing list  -  perforce-user at perforce.com
http://maillist.perforce.com/mailman/listinfo/perforce-user










Confidentiality Notice: The information contained in this electronic transmission is confidential and may be legally privileged. It is intended only for the addressee(s) named above. If you are not an intended recipient, be aware that any disclosure, copying, distribution or use of the information contained in this transmission is prohibited and may be unlawful. If you have received this transmission in error, please notify us by telephone (513) 229-5500 or by email (postmaster at MedPlus.com). After replying, please erase it from your computer system.






More information about the perforce-user mailing list