[p4] What actually happens during checkpoint

Stephen Vance steve at vance.com
Thu Mar 5 15:10:00 PST 2009


Checkpoints are simple. They lock all the tables, dump all the data a 
table at a time to the checkpoint file in a textual representation with 
additional command and integrity information, and unlock the tables.

Checkpoint time problems are rarely memory. First, make sure you are 
measuring the time it takes to actually perform the checkpoint (i.e. the 
time no one else can do anything on the server) and not the time of the 
checkpoint command itself. My understanding is that in order for the 
checkpoint to obtain all the locks it needs, it has to wait for existing 
locks to clear. This could take some time on a busy server.

Other than that, your constraints are, most likely in order of 
probability, a) data volume, b) disk I/O performance, b.1) disk I/O 
slowed down by virus scanning while writing, and c) CPU used for 
compression if you're compressing your checkpoints with Peforce.

(a) can be addressed by trimming excess metadata, particularly obsolete 
clients and non-automatic labels. Integration history can also 
contribute, but isn't usually something you want to trim. Restoring from 
checkpoint can also reduce the effective data volume. Diagnosing (b) is 
an art in itself, although the specific sub-problem (b.1) is easy low 
hanging fruit. Especially for large checkpoints, I've seen significant 
speedups in the effective downtime for (c) by writing the checkpoint 
without compression and then compressing as a separate step.

Steve

Jamie.Echlin at barclayscapital.com wrote:
> Can anyone help me understand what happens during a checkpoint?
> Background is, the checkpoint procedure has got incredibly slow, about 9
> hours to produce a 4.5 G compressed checkpoint from a 104G database.
>
> We have some problems with our SAN. The LUNS that make up the data
> volume are split over two different array controllers, one of them in a
> different data centre. However, this shouldn't cause problems in itself,
> it's just not "best practice", or so I've been told.
>
> My theory about the performance then was that the checkpoint would write
> the transactions in the same order that they were originally done,
> causing reads over all the different tables, somehow exacerbating a disk
> problem. Also we don't have enough main memory, only 32G. However,
> reading more, checkpoints are not the same as journals, the transactions
> are not in the same order. But a checkpoint is also not just a
> compressed copy of all the tables, if that was true there was no way it
> could so long. So what gives? ;-)
>
> A restore of a checkpoint takes 10 hours, so not much difference between
> reads and writes. Just to rule out decompressing being the bottleneck,
> it's only using a couple per cent of CPU.
>
> Cheers, jamie
>
>
> _______________________________________________
>
> This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing.  Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered offi!
>  ce at 1 Churchill Place, London, E14 5HP.  This email may relate to or be sent from other members of the Barclays Group.
> _______________________________________________
>
> _______________________________________________
> perforce-user mailing list  -  perforce-user at perforce.com
> http://maillist.perforce.com/mailman/listinfo/perforce-user
>
>   

-- 
Stephen Vance
www.vance.com





More information about the perforce-user mailing list