[p4] What actually happens during checkpoint

Smith, Jeff jsmith at medplus.com
Thu Mar 5 08:12:42 PST 2009


You don't indicate what kind of server you on which you are doing the
checkpoint.  If it is a Windows box is it possible that a new virus
scanner has been installed and it is scanning the checkpoint as it is
being written to disk?

Jeff 

-----Original Message-----
From: perforce-user-bounces at perforce.com
[mailto:perforce-user-bounces at perforce.com] On Behalf Of Brad Holt
Sent: Thursday, March 05, 2009 10:58 AM
To: Jamie.Echlin at barclayscapital.com; perforce-user at perforce.com
Subject: Re: [p4] What actually happens during checkpoint

This should be an interesting thread.  I'm no expert, but I'm guessing
that  "... the checkpoint would write the transactions in the same order
that they were originally done..." would be off, if by that you mean it
uses the transaction log (the journals).  The checkpoint doesn't
need/use the transaction journals to create itself.  Similarly, the act
of taking a checkpoint is the handiest way to check the veracity of the
database, so it shows that it is mining the metadata directly.  This
also means that you're right that "...a checkpoint is also not just a
compressed copy of all the tables..." as that wouldn't check the db
integrity.

As a benchmark, one of my servers is about your size - 150GB metadata,
5.75GB compressed checkpoint takes about 2.5 hours on Windows2003x64,
metadata volume NTFS RAID 1/0 on its own channel.  I wouldn't reckon the
RAM is the problem.  When I checkpoint, my memory usage doesn't go
anywhere much (but it's well handy for the day to day grind).  If you've
got the room, maybe try taking an uncompressed checkpoint just to make
sure compression hasn't become the bottleneck.

Sorry for the non-answer, but if you get one offline or from p4, be sure
to share it.  I'd be the better for it.

-----Original Message-----
From: perforce-user-bounces at perforce.com
[mailto:perforce-user-bounces at perforce.com] On Behalf Of
Jamie.Echlin at barclayscapital.com
Sent: Thursday, March 05, 2009 7:23 AM
To: perforce-user at perforce.com
Subject: Re: [p4] What actually happens during checkpoint

I should have mentioned we're using ext3, and it appears mounting it
with data=writeback would help, and XFS help even more. But even so, I'm
currently interested in why this process seems to have degraded so much
over the last 6 months (checkpoint used to take about an hour), and for
personal curiousity I'm interested in the mechanics of a checkpoint.

Cheers, jamie 

> -----Original Message-----
> From: perforce-user-bounces at perforce.com
> [mailto:perforce-user-bounces at perforce.com] On Behalf Of Echlin, 
> Jamie: IT (LDN)
> Sent: 05 March 2009 14:26
> To: perforce-user at perforce.com
> Subject: [p4] What actually happens during checkpoint
> 
> Can anyone help me understand what happens during a checkpoint?
> Background is, the checkpoint procedure has got incredibly slow, about

> 9 hours to produce a 4.5 G compressed checkpoint from a 104G database.
> 
> We have some problems with our SAN. The LUNS that make up the data 
> volume are split over two different array controllers, one of them in 
> a different data centre. However, this shouldn't cause problems in 
> itself, it's just not "best practice", or so I've been told.
> 
> My theory about the performance then was that the checkpoint would 
> write the transactions in the same order that they were originally 
> done, causing reads over all the different tables, somehow 
> exacerbating a disk problem. Also we don't have enough main memory, 
> only 32G. However, reading more, checkpoints are not the same as 
> journals, the transactions are not in the same order. But a checkpoint

> is also not just a compressed copy of all the tables, if that was true

> there was no way it could so long. So what gives? ;-)
> 
> A restore of a checkpoint takes 10 hours, so not much difference 
> between reads and writes. Just to rule out decompressing being the 
> bottleneck, it's only using a couple per cent of CPU.
> 
> Cheers, jamie
> 
> 
> _______________________________________________
> 
> This e-mail may contain information that is confidential, privileged 
> or otherwise protected from disclosure. If you are not an intended 
> recipient of this e-mail, do not duplicate or redistribute it by any 
> means. Please delete it and any attachments and notify the sender that

> you have received it in error. Unless specifically indicated, this 
> e-mail is not an offer to buy or sell or a solicitation to buy or sell

> any securities, investment products or other financial product or 
> service, an official confirmation of any transaction, or an official 
> statement of Barclays. Any views or opinions presented are solely 
> those of the author and do not necessarily represent those of 
> Barclays. This e-mail is subject to terms available at the following 
> link:
> www.barcap.com/emaildisclaimer. By messaging with Barclays you consent

> to the foregoing.  Barclays Capital is the investment banking division

> of Barclays Bank PLC, a company registered in England (number 1026167)

> with its registered offi!
>  ce at 1 Churchill Place, London, E14 5HP.  This email may relate to 
> or be sent from other members of the Barclays Group.
> _______________________________________________
> 
> _______________________________________________
> perforce-user mailing list  -  perforce-user at perforce.com 
> http://maillist.perforce.com/mailman/listinfo/perforce-user
> 
_______________________________________________

This e-mail may contain information that is confidential, privileged or
otherwise protected from disclosure. If you are not an intended
recipient of this e-mail, do not duplicate or redistribute it by any
means. Please delete it and any attachments and notify the sender that
you have received it in error. Unless specifically indicated, this
e-mail is not an offer to buy or sell or a solicitation to buy or sell
any securities, investment products or other financial product or
service, an official confirmation of any transaction, or an official
statement of Barclays. Any views or opinions presented are solely those
of the author and do not necessarily represent those of Barclays. This
e-mail is subject to terms available at the following link:
www.barcap.com/emaildisclaimer. By messaging with Barclays you consent
to the foregoing.  Barclays Capital is the investment banking division
of Barclays Bank PLC, a company registered in England (number 1026167)
with its registered offi!
 ce at 1 Churchill Place, London, E14 5HP.  This email may relate to or
be sent from other members of the Barclays Group.
_______________________________________________

_______________________________________________
perforce-user mailing list  -  perforce-user at perforce.com
http://maillist.perforce.com/mailman/listinfo/perforce-user

_______________________________________________
perforce-user mailing list  -  perforce-user at perforce.com
http://maillist.perforce.com/mailman/listinfo/perforce-user










Confidentiality Notice: The information contained in this electronic transmission is confidential and may be legally privileged. It is intended only for the addressee(s) named above. If you are not an intended recipient, be aware that any disclosure, copying, distribution or use of the information contained in this transmission is prohibited and may be unlawful. If you have received this transmission in error, please notify us by telephone (513) 229-5500 or by email (postmaster at MedPlus.com). After replying, please erase it from your computer system.






More information about the perforce-user mailing list