[p4] migrating 4Tb depot

Sat Oct 4 06:21:25 PDT 2008

Hi Robert,

Not sure on the transfer rates - I'm going to be running some tests next
week on 10-20Gb blocks of random data to see what I get.

I unfortunately can't connect the new array directly to the old due to
connectivity (SCSi vs FC) but I intend to send form one server to
another over our Gb network. Any bottleneck here will only come from the
read speed of the SATA-based MSA anyway so no great hardship.

When you say parallelizable do you mean that I can run simultaneous
verifies on different file branches in the depot? 

Regarding interference with normal usage - that's the reason I'm asking
this question. I intend to have the depot migrated and running on the
new server over a weekend so users are not on the system any time during
the process. I'm trying to ascertain how long this will require in case
I need to allow for some downtime at either end.

Cheers,

Ben

-----Original Message-----
From: Robert Cowham [mailto:robert at vizim.com] 
Sent: 04 October 2008 00:23
To: Benjamin Nias; perforce-user at perforce.com
Subject: RE: [p4] migrating 4Tb depot

Hi Ben

> I was hoping to ask three questions:
> 
> 1.    From the size of the depot can anyone estimate how long the
> pre-copy verify will take

This is a suck it and see type operation - will take a long time -
depends
on transfer rates between server and disk - server reads all versions
and
calculates MD5 checksum on them. How long does it take to get that
amount of
data off your disks?

Note that verify is parallelizable - up to a point at least - at some
level
you start hitting things like disk seek time and thrashing.

For example, for a largish customer with 6m+ revision files (but only
250Gb
total), parallelizing verifies took say 17 hours as opposed to 48 hours
if
just done sequentially (as "p4 verify -q //..." )

CPU speed is a factor, but usually not the limiting one.

> 2.    From the size of the depot can anyone estimate how long the
> post-copy verify will take (if diff from above)

Same factors apply :)

> 3.    Has anyone performed this procedure and have any advice or tips!

See above :)

Some thoughts:

If I were you I would try some verifies on smaller chunks of the
repository
and attempt to extrapolate from those. 

After the event you may want to only verify the last few weeks of data
initially - leave the rest until later.

If it's windows, then using robocopy to copy the files would seem
sensible,
and pretty reliable.

When doing the copy, consider your network topology between old
server/disks
and new server/disks - you don't want to be competing for network
bandwidth
with normal usage.

You can start the copy of the versioned files long before you need to
migrate metadata, and do incremental copies afterwards.

Assuming new machine is faster why not do copy and then run verify on
new
machine - it will not be under load. If any problems check if same files
cause problems on old machine.

Robert

________________________________________________________________________
This e-mail has been scanned for all viruses by Star. The
service is powered by MessageLabs. For more information on a proactive
anti-virus service working around the clock, around the globe, visit:
http://www.star.net.uk
________________________________________________________________________