[p4] Is there a way to write-lock the database while my backup runs?

[Jeff A. Bowles]

How about restating the problem this way: you need to have
a quiescent (if I've spelled that right) revision tree when you
do filesystem backups.

The db.* files are trivial; just back up the checkpoints and
journals and you pretty much have them, in a portable and
recreateable form. The issue is the depot files.

Why not ... if it's "paramount" importance, just put a line in
the protect table (at the end) when your backup starts, taking write
permission (p4 help undoc has a specific entry for this sort of thing)
away from everything.  The epilogue of the backup script
would remove that entry.

I admit, the tiny entries for 'spec depot' files would still be
in flux, but that's very little for anyone to lose sleep over.

If you really need it, play with protections. Might be a
simpler nut to crack.

	-Jeff Bowles
	Perforce Consulting Partner

(ps. "p4 help undoc" is handy, but .... undocumented. Meaning
unsupported. So, take all this with a grain of salt.)

On Feb 21, 2006, at 3:38 PM, Craig A wrote:

>>> You should re-read...And make sure you understand...
>
> Thanks very much for your comments. I have read the doc and I do  
> understand
> how all the pieces work. I am not trying to backup db.* files. We do a
> weekly checkpoint + nightly journal truncate. Each night we backup the
> checkpoint/journal files along with the versioned files. We stop  
> the service
> while this happens to ensure consistency.
>
> What I am trying to accomplish is (nightly) backing up the  
> versioned files
> in an absolutely consistent state with the checkpoints+journal files -
> without stopping the service.
>
> This is from an older version of the P4 doc:
> http://www.perforce.com/perforce/doc.042/manuals/p4sag/02_backup.html
>
>
> "There are rare instances (for instance, users obliterating files  
> during
> backup, or submitting files on Windows during the file backup  
> portion of the
> process) in which your depot files may change during the interval  
> between
> the time the checkpoint was taken and the time at which the depot  
> files get
> backed up by the backup utility.
> Most sites are [not] affected by these issues; having the Perforce  
> server
> available on a 24/7 basis is generally a benefit worth this minor  
> risk,
> especially if backups are being performed at times of low system  
> activity.
> If, however, the reliability of every backup is of paramount  
> importance,
> consider stopping the Perforce server before checkpointing, and  
> restarting
> it after the backup process has completed. Doing so will eliminate  
> all risk
> of the system state changing during the backup process. "
>
> And the current doc says:
>
> "While your versioned files can be newer than the data stored in your
> checkpoint, it is in your best interest to keep this difference to a
> minimum"
>
> and
> "As you might expect, the shorter this time gap, the better."
>
> At my site "reliability of every backup is of paramount  
> importance". We have
> a global operation with heavy usage 24/7. We have no "times of low  
> system
> activity". It's not clear from the doc what problems might arise  
> from this
> situation. If there was no problem, then why the two statements above?
> Therefore I conclude there is some possibility of problems and  
> hence my
> search for some way to lock out writes. I'm not sure why Perforce  
> removed
> the first quoted note from the doc.
>
> Robert Cowham mentioned 'p4d -jj'. This does not solve the problem of
> versioned files being consistent with checkpoint/journal files. You  
> have to
> ensure there are no versioned files being written to for as long as  
> it takes
> your backup to complete.
>
> As Jeff Grills mentioned, there are lots of other database updates  
> besides
> submitting depotfiles. I would be interested in a lock that only  
> affected
> operations that would cause a change in the versioned files.
>
> Also, we don't have the option of using a 'Snap server' to do the
> ultra-quick snapshots of versioned files.
>
> Craig
>
> -----Original Message-----
> From: kgraham at uasc-id.com [mailto:kgraham at uasc-id.com]
> Sent: Tuesday, February 21, 2006 2:39 PM
> To: bfquiz at yahoo.com; perforce-user at perforce.com
> Subject: RE: [p4] Is there a way to write-lock the database while  
> my backup
> runs?
>
> The database checkpoint procedure does not require you to shut down  
> the
> service.
>
> You perform a checkpoint with p4d -jc, which truncates the journal and
> creates a checkpoint file.  The checkpoint file contains the entire  
> contents
> of the DB files.
>
> You do not need to, and may cause problems if you, back up the  
> actual DB
> files while the server is running.
>
> The checkpoint file, plus any journal file(s) created after that  
> checkpoint,
> is sufficient to recreate the DB up to the point where the journal  
> ends.
> This is why the journal (and the checkpoint) should be kept on a  
> different
> drive (and preferably controller) than the DB files.  In the event  
> the drive
> containing the DB fails, you use the last checkpoint and the  
> journal to
> reconstruct the DB.
>
> While this -jc is running, the server is fairly slow.  But  
> operations (both
> read and write) can still take place.
>
> You should re-read
> http://www.perforce.com/perforce/doc.052/manuals/p4sag/ 
> 02_backup.html#104333
> 6
>
> And make sure you understand depot files, database files,  
> checkpoints, and
> journals.  Do some testing on a non-production server and get a  
> feel for how
> all of this works.
>
> Keith Graham
>
>
>
>
> _______________________________________________
> perforce-user mailing list  -  perforce-user at perforce.com
> http://maillist.perforce.com/mailman/listinfo/perforce-user