[p4] Performance impact of "large" clients

Anders Kjærgaard Hansen ANH at etiglobal.net
Sun Aug 9 23:56:11 PDT 2009


Once again I agree with Jeff and Stephen.

I think the primary concern in this case is the negligence of best practice and the increased risk of "hidden" submits as described by Ines Heinz.

Also the fact that submitted changelists might scope more than one branch, or multiple products, makes it harder for other developers to figure out what happened if they have to revert a changelist, or track where changes stem from.

Secondary, we don't want a handful of users to cause small DOS-like symptoms arise on our servers.

Especially since the experience using P4CONFIG is so good. Whether you prefer working in a prompt, shell, Visual Studio or from the Perforce visual tools these settings are respected and make it almost unnecessary for our users to spend time thinking about clients.

With our tool suite, combined with P4, users just have to write "work myProduct", and all is set. If the later on write "work myOtherProduct", a new client is ready there. And whenever they switch which product they are working on, they are sure they only have changes for that product in their changelist. And all the common tools used to edit the files support the invisible change of clients.

So I would state that for 99% of our users, there is no need at all for them to concentrate on the concept client or workspace. It's just there ready to use and scoped as intended.

The only argument for wanting to scope several products in one client, should be that there are cross product changes the belongs together because of tight-coupled software, and thus could belong to one changelist. Besides the bad architecture if two products are so tight coupled, there should be no problems though of having these changes in two changelists, which are submitted "simultaneously" instead of one widely scoped client.

On the other hand I agree Todd that there is no need in securing just for that sake of securing. But where it makes sense, and in this case both for the process and performance, I find it the only way to make sure our users behave correctly.

/Anders


________________________________
From: Stephen Vance [mailto:steve at vance.com]
Sent: Saturday, August 08, 2009 10:39 PM
To: todd.benham at kodak.com
Cc: Anders Kjærgaard Hansen; perforce-user at perforce.com
Subject: Re: [p4] Performance impact of "large" clients

It all depends on what you're trying to achieve.

If you tackle it as a societal statement, you're absolutely right that it's purely a matter of opinion. That's why it should be driven by concrete goals and measurements.

The reality is that larger client specs have a performance impact for larger source bases and larger user communities. It's the realities of data scaling and data access contention. The larger your source, the greater the number of branches, and the greater the number of users, the more you have to manage your server performance.

Perforce scales much better if you constrain your users to the patterns of usage that support your process. You also require fewer people to support the process if you select a narrower band of usage patterns. For example, if you have 1000 users, do you want to pay for another Perforce admin just to handle the obliterate requests from the kinds of accidents that can and do happen with larger client specs? Or to answer the same questions repeatedly rather than just automate the restrictions? Do you want 1000 or more different variations on how to do things in your source control system or, even worse, your software promotion and release model?

It comes down to a choice. If your answers to the above are "yes" or "I don't mind" or "we're not big enough for that to be a worry" or something along those lines, then don't establish or enforce those kinds of policies.

On the other hand, with the use of P4CONFIG and workspace selections in the graphical tools, I've never seen why maintaining multiple smaller workspaces is inconvenient compared to a larger one. You have the extra client specs and a few extra syncs, but those things tend to go along with the locality of your workflow anyway.

Just my $.02,
Steve

todd.benham at kodak.com<mailto:todd.benham at kodak.com> wrote:

Ok, I need to express the other side of this argument.



I don't get it -- society seems need to be so "secure" and "protected"

these days. Why make life difficult? Are you really gaining anything?

Give the user permissions to the areas they might or probably need at a

relatively high level in the depots and then you are done (change

permissions as needed). They can use a tiny scope or a large scope in

their clients as they see fit. How many times have you had to stop what

you are doing because you get a "not in client" message from Perforce.

For scripts and tools and the like, I understand this completely. But

the focus seems to be on how to make sure users are controlled and

limited (which usually means low productivity and a lot of overhead for

setup). Sure you can start "beginners" out with narrow clients but I

don't see that many or certainly not all groups would need to write

triggers and enforcements to restrict things. Why discourage small

client if someone has a use for it? Or why make some jump through hoops

to have a large client. The examples given so far seem contrived. This

just seems like a waste of time to me for the vast majority of

developers. If you have a real need for this kind of thing, certainly go

ahead an do it.



Todd







-----Original Message-----

From: perforce-user-bounces at perforce.com<mailto:perforce-user-bounces at perforce.com>

[mailto:perforce-user-bounces at perforce.com] On Behalf Of

ANH at etiglobal.net<mailto:ANH at etiglobal.net>

Sent: Friday, August 07, 2009 1:51 AM

To: 'perforce-user at perforce.com<mailto:perforce-user at perforce.com>'

Subject: Re: [p4] Performance impact of "large" clients





I totally agree.



The ignored best practice, and risk of making errors, unintended

submits, is the biggest problem.



We do have a tool that helps our users create their clients, and that

tool create clients per branch. So, if a product has main, some devel

branches and some release branches, the client generated by our tool is

only for e.g. the main branch of that product.



And ~97% of our users use these clients as they are supposed to do.



Our challenge is the last few users, which are competent enough to

figure out how they alter the generated clients (or create new ones by

hand), but apparently not competent enough to understand the possible

pitfalls of using the large scoped clients spanning several branches.



We have tried the educational way.



And what I wanted from this thread was to gain some technical arguments

that might convince them that they are actually disturbing the other

developers by locking the database. Besides running the risk of

submitting/changing files unintended as Ines examples showed.



"Luckily" some of the worst of our users are database developers, and

the argument with database locking might actually be something they can

understand. :-) Thanks to Jeff for that explanation.



Thank you all for your replies. I will head back to those developers

trying to convince them why clients spanning the entire depot are BAD.



For the record, I think we might combine our educational effort with

some form-out, form-in triggers on client.



Especially a form-out trigger that changes default clients to span over

//depot/sandbox/main/... instead of //depot/...



Not quite sure yet if we create a form-in trigger, the disallows these

clients, or if we just react upon it if someone creates a client we

don't like.



Once again, thanks for the replies. I good first experience with the p4

mail lists. :-)



Best regards

/Anders







-----Original Message-----

From: perforce-user-bounces at perforce.com<mailto:perforce-user-bounces at perforce.com>

[mailto:perforce-user-bounces at perforce.com] On Behalf Of Ines Heinz

Sent: Thursday, August 06, 2009 5:47 PM

To: 'perforce-user at perforce.com<mailto:perforce-user at perforce.com>'

Cc: 'perforce-user at perforce.com<mailto:perforce-user at perforce.com>'

Subject: Re: [p4] Performance impact of "large" clients



When we have allowed the code developers to have

clients  that contain multiple branches, they

have not only done what Steve stated, but they

also have mistakenly submitted changes for work

that had not been tested and reviewed.  Yes, you

can put in triggers to help limit the pain, but

going for the simpler solution and having a

client per branch is far safer and easier.



===========================Here is the

scenario============================

I go into winedev and p4 edit the wine.atd file

because my wine label is the up and coming one --

with the really witty comments about nuttiness and hints of berry.



Then someone realizes that the wine label in

2009kegs has a misspelling of the word blackberry,

so I go and do a quick edit of that, get my edit

reviewed and submit.  All well and good with

separate clients, but with a wide-open client, I

accidentally submit all my opened files -- which

submits my witty comments (some of which had not

been spell-checked)  into the winedev

branch without the appropriate review.



=====================wide open

client=====================================

Client Root: /home/ines/perforce

Client View:

//depot/winedev/...

//ines_perforce_workspace_for_updating_docsn/winedev/...

//depot/2009kegs/...

//ines_perforce_workspace_for_updating_docs/2009kegs/...



         cd /home/ines/perforce/winedev

         p4 edit wine.atd

         cd /home/ines/perforce/2009kegs

         p4 edit wine.atd

         p4 submit -d "quick edit to fix typo in

blackberry"  #oops -- winedev/wine.atd is submitted!



=====================separate

clients=====================================

Client Root: /home/ines/perforce/winedev

Client View:

//depot/winedev/... //ines_perforce_workspace_for_updating_docs_dev/...

Client Root: /home/ines/perforce/2009kegs

Client View:

//depot/2009kegs/...

//ines_perforce_workspace_for_updating_docs_2009/...



         cd /home/ines/perforce/winedev

         p4 edit wine.atd

         cd /home/ines/perforce/2009kegs

         p4 edit wine.atd

         p4 submit -d "quick edit to fix typo in

blackberry"  #winedev/wine.atd is not submitted!





At 07:16 AM 8/6/2009, Stephen Vance wrote:



Large subsets for view resolution which

increases database lock time, memory requirements, and processor



requirements.



Large have tables increase your database sizes.

Have tables have a high churn rate increasing

database fragmentation and slowing down performance.

P4V and P4Win don't fstat all files on opening,

but they do fstat all files, changes, jobs, etc.

that are displayed and in scope. Larger clients

increase the scope, although not necessarily

more than "Entire Depot View" would.



I strongly discourage catch-all clients. It

leaves opportunities for several mistakes and

best practice violations. It allows changelists

to span branches which violates the principle of

a changelist being a single logical change to a

codeline. It allows for sloppy or mistakenly

targeted integrations. Those are just two of the issues.



Steve



Anders Kjærgaard Hansen wrote:



Hi



We are currently doing some housekeeping and

cleanup in our Perforce user database, and old unused clients.



We came to discuss what impact large clients

have on both the perforce server, but also our storage.



We have several users who was clients with views as //depot/...

//client/...



What are the impacts of these "global" clients?



Do p4, p4v, p4win e.g. fstat all files in a

client each time you open it in the application?



We are especially interested in hearing

opinions on whether we should allow these clients or not.



We have tools supporting creation and use of correct product scoped

clients.



I hope some of you have some valuable input



Sorry if I'm reposting this, my first mail didn't show up so I am

retrying.



Best regards

/Anders



_______________________________________________

perforce-user mailing list  -  perforce-user at perforce.com<mailto:perforce-user at perforce.com>

http://*maillist.perforce.com/mailman/listinfo/perforce-user







--

Stephen Vance

www.*vance.com<http://www.*vance.com>

_______________________________________________

perforce-user mailing list  -  perforce-user at perforce.com<mailto:perforce-user at perforce.com>

http://*maillist.perforce.com/mailman/listinfo/perforce-user







======================================

Aura Ines Heinz

WCI LC Liaison

Lawrence Livermore National Laboratory

Phone:  (925) 423-7900

Fax: (925)423-5209

B132N R1250

======================================

_______________________________________________

perforce-user mailing list  -  perforce-user at perforce.com<mailto:perforce-user at perforce.com>

http://maillist.perforce.com/mailman/listinfo/perforce-user



_______________________________________________

perforce-user mailing list  -  perforce-user at perforce.com<mailto:perforce-user at perforce.com>

http://maillist.perforce.com/mailman/listinfo/perforce-user



_______________________________________________

perforce-user mailing list  -  perforce-user at perforce.com<mailto:perforce-user at perforce.com>

http://maillist.perforce.com/mailman/listinfo/perforce-user







--

Stephen Vance

www.vance.com<http://www.vance.com>



More information about the perforce-user mailing list