[p4] Python scripting with Perforce

Jeff A. Bowles jab at pobox.com
Wed Oct 5 22:48:33 PDT 2005

It's worth more than an afternooon.

The "p4 -G" work usually gets used for the best of reasons: because the
data is preparsed and ready to use.   There are other options available,
but this is probably the most immediately portable one - if you have  
you have this. (I think that's right, within a reasonable set of  

Most of the time, I've seen:
     1) "A simple program that reads a single 'p4 -G something' and
          pretty-prints the output;"
     2) "An all-purpose reformatter for the data, that does basic simple
         grouping/analysis. You run 'p4 -G something |  
thisTool'."   (There
         was a presentation at the most recent Perforce conference by
         J.T. Goldstone on this.)
     3) You write a single routine that reads in the marshal, and then
         a layer or two above it. You can find examples on the net, and
         I have some in the "Perforce Public Depot" under the pathname,

You can make more elaborate versions of any of these strategies,
and some have.

     -Jeff Bowles

ps. As an aside, it was particularly cool when I realized that my
overnight.py script could be imported by a second application,
"continuousbuild.py", and that almost 90% of the code would be
directly imported from the already-functional overnight build code.
That reuse was especially easy because of the language choice.
(Another "pro" argument.)

On Oct 5, 2005, at 2:05 PM, Darin Petty wrote:

> Paul,
> I didn't see another answer.
> Marshal.load only reads the first record.
> I use a home-made function to read the whole collection:
> def RunMarshal(command):
>     # run marshal.load in a loop
>     results=[]
>     stream=os.popen(command,'rb')
>     try:
>         while 1:
>                 results.append(marshal.load(stream))
>     except EOFError:
>         pass
>     stream.close()
>     return results
> I hope that solves your problem with the counters.
> Darin
> Paul said:
> Hey all:
> I've just started playing with scripting some Perforce triggers and  
> such
> with Python. I went with Python because of Perforce's -G option, but
> after playing with it for an afternoon, I'm slightly disappointed.
> I'm curious if anyone knows:
> 1. Why the marshal module was used, as opposed to the pickle/shelve  
> modules?
> I'm having problems fetching data from a Perforce ("p4 counters")
> invocation. In further reading the Python documentation, I saw:
> [marshal] is not a general ``persistence'' module. For general
> persistence and transfer of Python objects through RPC calls, see the
> modules pickle and shelve. The marshal module exists mainly to support
> reading and writing the ``pseudo-compiled'' code for Python modules of
> .pyc files. Therefore, the Python maintainers reserve the right to
> modify the marshal format in backward incompatible ways should the  
> need
> arise. If you're serializing and de-serializing Python objects, use  
> the
> pickle module instead.
> I'm wondering if my problems (namely that my counters invocation only
> returns the first counter) are related to possible changes that may  
> have
> been made to the Python interpreter in 2.3.x that are incompatible  
> with
> the binary marshaled data Perforce is dumping.
> 2. Am I the only one that finds the structure of the marshaled output
> somewhat... confusing?
> The sample trigger on page 102 of 2005.1 Administrator's Guide gives a
> "p4 group -o" invocation with marshaled output; the output dumps  
> keys of
> UserNNN, where N is 0 to the total number of users you have. This
> requires matching against the keys (see if it matches '^User', as the
> sample does) to see if it's a key you want the value to. Why wasn't  
> this
> just put into an array of hashes called "Users"? Is this a  
> limitation of
> the marshal module, or?
> I'd be curious to see the reasoning used here; I'm building some
> infrastructure that I rallied behind using Python for, most notably
> because of this convenient marshaling capability, but my initial
> impression of this feature is that it's somewhat incomplete and  
> fragile
> in terms of specific Python interpreter version dependencies, and if
> that's actually a case, that makes this a no-go; I can't have
> infrastructure breaking when we do a x.y.z+1 Python interpreter  
> upgrade.
> Is this accurate or am I doing something wrong?
> Later,
> Paul
> --
> Build/Release Engineer
> preed at vmware.com
> (650) 475-5248
> Darin Petty
> Project Manager, Support & Tools
> Configuresoft, Inc.
> Security. Compliance. Control.
> www.configuresoft.com
> Office: 719.687-2676
> mailto:Darin.Petty at Configuresoft.com
> _______________________________________________
> perforce-user mailing list  -  perforce-user at perforce.com
> http://maillist.perforce.com/mailman/listinfo/perforce-user

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://maillist.perforce.com/pipermail/perforce-user/attachments/20051005/a2c86498/attachment-0006.html>

More information about the perforce-user mailing list