[p4] How do YOU identify the code set for a release?

[Frank Compagner]

>> Add to that the temp files needed to make a build and it's over
>> 250GB. With these amounts of data it does become a bit unpractical
>> to keep several branches live on your local machine.

IW> Are you sure? My personal machine has about 2.5 TB hung on it and
IW> that cost less than $1000. (I work with a lot 90+ Megapixel photos
IW> at home.)

Sure, it is possible with today's harddisks, but it's still a bit
unwieldy to work with.

IW> If you have a lot of files that don't change, perhaps the solution
IW> is to reorganize the projects to isolate relatively stable blocks
IW> of data in their own projects. (You could do this client-side, as
IW> well, by using two+ clients, carefully adjusting the views to
IW> coexist properly. One client would represent stable files, while
IW> the other(s) would replace the branch-specific files.)

That would be a good idea, but with about 150 people working on the
project it is hard to find stable areas. We are trying to weed them
out, though.

>> It would be great if their was a reliable and cheap way of
>> changing branches in-place on the client, where only the files
>> that are actually different between the branches are downloaded.

IW> Only if it didn't impact speed or reliability for the more general
IW> situations.

IW> I would think, given this is a rather special case, it would be
IW> best dealt with by creating a custom client - using either an API
IW> or commands - that did the necessary manipulations to make it
IW> happen in a way the server could accept without modification.

Yes, sure, I'm not asking to any fundamental changes to the Perforce
branching mechanism. We already do a lot of custom tool development to
make Perforce a bit easier for our non-technical users, we're figuring
out how best to integrate branching into them. With a bit of luck we
only need some small changes to make switching branches a bit more
practical.

>> but if the local files that do not differ between the branches
>> remain untouched only a partial build will be necessary.

IW> There are build managers that can make decisions like that based
IW> on the file's content rather than a time stamp. That wouldn't
IW> solve the re-sync problem, but it would cut the subsequent build
IW> times.

Won't that make the up-to-date check prohibitively expensive? Then a
build where only a single file was changed would still need to check
every single file to know it was done.

----------------------------------------------------------------
Frank Compagner                                  Guerrilla Games

IW> -Wm


IW> -----Original Message-----
IW> From: frank at compagner.com [mailto:frank at compagner.com] 
IW> Sent: Friday, January 04, 2008 5:52 PM
IW> To: Ivey, William
IW> Cc: Perforce Users
IW> Subject: Re[2]: [p4] How do YOU identify the code set for a release?

>>> With Subversion, I can create the branch then switch my local
>>> working copy from trunk to the branch in just a few seconds, do
>>> the build, then switch back to the trunk.

IW>> It's even faster when you have both branches resident on the
IW>> machine all the time. In fact, you can build both at once,
IW>> tools permitting. (I usually have four or five branches of our
IW>> product synced at one time since I have to work on and build
IW>> all the currently supported versions. Changing the "identity"
IW>> of a single directory would be a nightmare.)

IW>> It's a one time expense in terms of time, and disk space is
IW>> cheap.

IW> Well, that isn't always true. The size of a full head revision of
IW> our main project is currently about 150GB. Add to that the temp
IW> files needed to make a build and it's over 250GB. With these amounts
IW> of data it does become a bit unpractical to keep several branches live
IW> on your local machine. And as several Gigabytes of data are checked
IW> into each branch every week keeping them in sync isn't exactly cheap
IW> either.

IW>> You can "fake it" to make what you're used to happen by playing
IW>> games with your client mapping or with symbolic links if you're
IW>> using a Unix system, but it's not as safe and convenient as
IW>> one directory per branch.

IW> It would be great if their was a reliable and cheap way of changing
IW> branches in-place on the client, where only the files that are
IW> actually different between the branches are downloaded. This would
IW> also help a lot in reducing build times after switching branches. A
IW> full build takes several hours, but if the local files that do not
IW> differ between the branches remain untouched only a partial build will
IW> be necessary.

IW> As it is, you can sort of fake it with a change in the client mapping
IW> followed by a sync -k, but you would still need to do at least a
IW> diff -se to get the list of changed files. As that means calculating
IW> an md5 hash (or something similar) of all files in the entire branch
IW> this can easily take hours. It would be great if the hash values could
IW> be cached locally on the client (they are now cached on the server,
IW> which helps a lot at that end). That way a simple check against the
IW> atime would be enough to see if the hash needs to be recalculated.
IW> Another idea would be to expose the hash function in the p4api (maybe
IW> through a FileSys::GetHash() function), that way we could make a
IW> custom tool to manage the hash cache ourselves. When combined with a
IW> filesystem hook to trap filewrites to the branch directory this might
IW> make switching branches on a project like this a much less painful
IW> operation.

IW> ----------------------------------------------------------------
IW> Frank Compagner                                  Guerrilla Games

IW> _______________________________________________
IW> perforce-user mailing list  -  perforce-user at perforce.com
IW> http://maillist.perforce.com/mailman/listinfo/perforce-user