[p4] Obliterate metrics

Tue Apr 17 12:46:44 PDT 2007

On 4/17/07, David Ferguson <daf at vmware.com> wrote:
> Ah ... MUCH better doing it within a client.  Thanks.
> I tried this first, but may have bitten off too much with the first couple of
> tries since they never seemed to complete.  As long as the hierarchy being
> removed is somewhere under 10K files, I'm reliably getting 2-4 revisions/sec
> being obsoleted (as opposed to 2-4 sec/revision)...

Yes, it appears like it is not doing anything for about maybe 70% of
the total time it will take.  There is really no way to estimate how
long its going to run from just the stuff it outputs.  The number of
lazy copies it has to undo I'm sure is a big factor.

As I mentioned about the form of the command, when I used the "wrong" form, my
oblit took 3 days!  It was over Christmas weekend, so it didn't
matter. I did start to
worry/wonder how long it was going to take!

dave

>
> In case anybody else cares ...
> Overnight -- obsoleting individual files in 49140 secs, I purged 28458
> revisions
> Today -smaller cases- obsoleteing individual revs of individual files 598
> revisions took 1200 seconds
>                                 Individual changesets of individual files 490
> revisions tool 1380 seconds
> Client -- Obsoleting hierarchy (//depot/autotest/bar/...) did 9600 revisions
> in 180 seconds
>
> -daf
>
>
> > -----Original Message-----
> > From: perforce-user-bounces at perforce.com
> > [mailto:perforce-user-bounces at perforce.com] On Behalf Of Dave Lewis
> > Sent: Tuesday, April 17, 2007 8:43 AM
> > To: Paul Goffin
> > Cc: perforce-user at perforce.com; Rick Macdonald
> > Subject: Re: [p4] Obliterate metrics
> >
> > the way you present the set of files also makes a difference.
> > we usually setup a client and map all the files to be
> > obliterated to it.
> > then
> >
> > p4 -c obliterate-client obliterate -y //obliterate-client/...
> >
> > In past versions, it has actually been sensitive to the form
> > of the command
> >
> > p4 -c obliterate-client obliterate -y ...
> >
> > ran *much* slower.   It should run the same.  obliterate is much, much
> > faster in new versions.
> >
> > As Paul says, perforce may copy files around to undo lazy
> > copies, and this can result in increased disk space usage.  I
> > have never tried to order the list of files to oblit because
> > the "one whak" method was so much more efficient.
> > (so I have never observed the possible increased efficency of
> > the other method!)  If I remember correctly, 50,000 files
> > would take about 40 minutes.
> >
> > having your db.* files freshly recovered to minimize their
> > size is also a good idea, since each oblit run must read
> > pretty much all the metadata.
> >
> > dave
> >
> >
> > On 4/17/07, Paul Goffin <paul.goffin at dsl.pipex.com> wrote:
> > > I was just outlining.
> > >
> > > But if you want better optimisation, an even better way is
> > to find the
> > > changelists connected with the path-to-be-removed by "p4 changes
> > > path-to-be-removed" and only attempt to obliterate those
> > changenumbers.
> > >
> > > Paul.
> > >
> > > -----Original Message-----
> > > From: Rick Macdonald [mailto:rickmacd at shaw.ca]
> > > Sent: 17 April 2007 08:48
> > > To: Paul Goffin
> > > Cc: perforce-user at perforce.com
> > > Subject: Re: [p4] Obliterate metrics
> > >
> > >
> > > What if your counter is up over a couple hundred thousand?
> > >
> > > Wouldn't you just stop at the counter that created
> > "path-to-be-removed"?
> > >
> > > Rick
> > >
> > > Paul Goffin wrote:
> > > > Obliterate is much faster if you can reduce the workload on the
> > > > server by not making it unwind branches.  In other words,
> > obliterate
> > > > the destination of a branch or rename location before the source.
> > > >
> > > > Sometimes it's easy to do this manually but a script that applies
> > > > "obliterate" to the section of the depot you want to
> > remove "change
> > > > by change" and works BACKWARDS in time can do the job
> > with no effort.
> > > >
> > > > i.e. write a script that does the following:
> > > >
> > > > Set counter to highest change number of interest.
> > > > Repeat
> > > >   p4 obliterate //path-to-be-removed/... at counter,counter
> > > >   decrement counter
> > > > Until counter = 0
> > > >
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: perforce-user-bounces at perforce.com
> > > > [mailto:perforce-user-bounces at perforce.com] On Behalf Of David
> > > > Ferguson
> > > > Sent: 17 April 2007 07:42
> > > > To: perforce-user at perforce.com
> > > > Subject: [p4] Obliterate metrics
> > > >
> > > >
> > > > Okay, we split our depot into two last summer.  I'm
> > finally getting
> > > > around to cleaning up all the old cruft that is now
> > obsolete (since
> > > > it was put into the other depot)... Two choices: obliterate
> > > > checkpoint surgery.
> > > >
> > > > Surgery works, but support really seems to frown on it.
> > > >
> > > > obliterate works ...  But wow it's slow.  Anybody got
> > metrics on how
> > > > slow?
> > > >
> > > >
> > > > _______________________________________________
> > > > perforce-user mailing list  -  perforce-user at perforce.com
> > > > http://maillist.perforce.com/mailman/listinfo/perforce-user
> > > >
> > > >
> > > _______________________________________________
> > > perforce-user mailing list  -  perforce-user at perforce.com
> > > http://maillist.perforce.com/mailman/listinfo/perforce-user
> > >
> > _______________________________________________
> > perforce-user mailing list  -  perforce-user at perforce.com
> > http://maillist.perforce.com/mailman/listinfo/perforce-user
> >
>