[os-infrastructure] RE: Piccolo to Subversion cheat sheet

Thu May 1 17:42:57 PDT 2008

Hi Alex,

Yes, that confirms what I read in the manual. It is possible to pull selected changes into what they call a “hotchpotch” client, which is good. As you noted, the bad news is that there doesn’t seem to be any way of making a permanent record (label) of that, except by using the “advanced tag” mechanism, which seems to require taking a branch.

It seems that this whole issue roots from the fact that piccolo maintains an exact record of the state of every client on the server (the client have list, the opened files, the reserved files, etc…); this feature allows piccolo to have the concept of a “label” recorded on the server. Subversion only maintains the state of the client on the client itself (in the various .svn directories), which probably makes it much more difficult to record the current state of the client on the server without creating a permanent footprint there (the branch). Piccolo is optimized for an intranet and can keep client information, but it won’t scale to many internet clients; Subversion is optimized for the internet and scales nicely, but it can’t easily record the client’s state on the server….I guess you can’t have everything.

Steve

From: Alex Hanshaw 
Sent: Thursday, May 01, 2008 7:44 PM
To: Stephen Ball
Cc: opensource-infrastructure at lists.ingres.com
Subject: RE: [os-infrastructure] RE: Piccolo to Subversion cheat sheet

Hi Steve

I've found that I can setup a client at revision 1 and then decide I want to pull in change 3 (because I don't want change 2) using the merge
function.

> svn status -q -u /home/ingres/svn/clients/svn3/ingres
       *        1   /home/ingres/svn/clients/svn3/ingres/dir1/dir1a/file1a1.c
       *        1   /home/ingres/svn/clients/svn3/ingres/dir1/dir1b/file1b1.c
       *        1   /home/ingres/svn/clients/svn3/ingres/dir1/dir1b/file1b3.c
       *        1   /home/ingres/svn/clients/svn3/ingres/dir2/dir2a/file2a1.odt
Status against revision:      4

Shows that there are three changes above the original import (chgs 2, 3, 4) and that I'm at revision 1 on all files
with the 4 that are listed being behind head rev.

> svn merge -c 3 file:///home/ingres/svn/repository/ingres /home/ingres/svn/clients/svn3/ingres
U    /home/ingres/svn/clients/svn3/ingres/dir1/dir1b/file1b3.c
U    /home/ingres/svn/clients/svn3/ingres/dir2/dir2a/file2a1.odt

Is used to pull change 3 from the repository to the client.

> svn status -q -u /home/ingres/svn/clients/svn3/ingres
       *        1   /home/ingres/svn/clients/svn3/ingres/dir1/dir1a/file1a1.c
       *        1   /home/ingres/svn/clients/svn3/ingres/dir1/dir1b/file1b1.c
M      *        1   /home/ingres/svn/clients/svn3/ingres/dir1/dir1b/file1b3.c
M      *        1   /home/ingres/svn/clients/svn3/ingres/dir2/dir2a/file2a1.odt
Status against revision:      4

Shows that the merge applies the changes as local modifications over the revision the build is at.
So the pull change now shows up as local modification that in-line the file with change 3.
I need to dig in further to see what happens when there are dependencies. I can confirm that without
dependencies the text and binary files have the expected content.

I'm very surprised that svn does not seem to have a label mechanism to record the content of a client
so that the same build can be established on a different client. The merge mechanism might put a spanner
in the works for any attempt to add our own scripts because the svn status does not show when change
level we have merge a given file to. It just shows that it is locally modified. An open label equivalent
would allow the build to be restored on another client.

Alex

On Wed, 2008-04-30 at 22:26 -0400, Durwin Wright wrote: 

We create Piccolo labels for each daily build (about 15 to 20 per month for all codelines).  There are two build areas for each codeline – one for the OpenROAD runtime and one for the OpenROAD eClient and mClient runtimes.  So I guess that would be between 30 and 40 labels per month.   

It is imperative that we continue to be able to selectively pull changes.  I agree with Steve, without this capability our products will never exit QA.

	Durwin Wright | Sr. Architect | Durwin.Wright at ingres.com | Ingres | 500 Arguello Street | Suite 200 | Redwood City | CA | 94063 | USA <http://maps.google.com/maps?q=500+arguello+street,+94063&ll=37.487297,-122.233200&spn=0.004602,0.012771&t=k&hl=en>   +1 650-587-5523 | fax: +1 650-587-5550 

________________________________

	From: opensource-infrastructure-bounces at lists.ingres.com [mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of Stephen Ball
	Sent: Wednesday, April 30, 2008 7:04 PM
	To: Alex Hanshaw
	Cc: OpenSourceInfrastructure
	Subject: [os-infrastructure] RE: Piccolo to Subversion cheat sheet

	Yes; that seems to be the implication from my reading too; after a few passes through the doc it would appear to me that the only way to achieve the equivalent of a “label” that is anything other than all head-revs is to take a branch. Given that we have taken to labeling quite frequently, and many of them are not head-rev labels, we could end up with a lot of branches, I hope this doesn’t cause an unmaintainable mess….or maybe there’s something I’m missing; I think some more reading is required.

	Steve

	From: Alex Hanshaw 
	Sent: Thursday, May 01, 2008 2:50 AM
	To: Stephen Ball
	Cc: OpenSourceInfrastructure
	Subject: RE: Piccolo to Subversion cheat sheet

	Hi Steve

	merge and tagging are for branches. They don't look like they provide the same notion of a label.
	I can confirm that a client can be at a given revision and an individual change can be pulled in and that it is possible to determine
	any dependencies. However, I don't currently see a way to do the equivalent of recording a label for that client so
	that another client can be pulled to the same level.

	Alex

	On Tue, 2008-04-29 at 03:16 -0400, Stephen Ball wrote: 

	It seems that it might be possible to selectively take changes  without necessarily branching. The implication is that a client may use a “merge” operation to selectively merge changes from the current branch into the client; see cherrypicking in:

	http://svnbook.red-bean.com/nightly/en/svn.branchmerge.advanced.html

	“merge” is intended for cross-branch merging, but the implication is that it will work within a current branch; much like the piccolo “integrate”. 

	Subversion also provides a way for a client to “update –r” to revert some files to specific revisions and crate a “hodge-podge” of different revisions on the client which can then be “tagged” (read piccolo label) for production builds. See “creating a complex tag” at:

	http://svnbook.red-bean.com/nightly/en/svn.branchmerge.tags.html

	The pre-requisite here is that we must be running Subversion 1.5 so that Subversion can keep track of merge operations.

	Steve

	From:Stephen Ball 
	Sent: Tuesday, April 29, 2008 3:09 PM
	To: Durwin Wright; Andrew Ross
	Cc: Joseph C. Kronk; Teresa L. King; Alex Hanshaw; David Tondreau; Bodo Bergmann
	Subject: RE: Piccolo to Subversion cheat sheet

	Yes, I would add that, in the final stages of delivery to QA, we only tend to pull the changes needed to exactly fix the outstanding QA issue list, and we will deliver an incremental build to QA (e.g. only re-build optimizedb to fix an issue) which means they only have to do limited re-testing; at that point we will avoid taking other changes that may fix “minor” sev3 or sev4 issues, especially if they cause a complete re-build of the product (e.g. changes in ADF). I think it is essential that we have the ability to do this otherwise we would end up with extended QA cycles that we may never come out of (bad fix breaks something else and so we go round again). I can see that we might manage this in Subversion with branching, but unless there’s a good way to keep the branches in control the whole thing could quickly turn into an unmaintainable mess.

	Ingres (and OpenROAD) is somewhat unique in that it is a *very* large product with complicated build and release procedures, but having said this, I’m figuring someone must have had similar issues with Subversion before and solved them somehow.

	One side note that might be worth mentioning is that Bill will tell you stories of the various code management systems that Oracle have used over the years, including some large, expensive, and functionally rich products, but they always ended up coming back to their own home-brewed code management system. The needs of a DBMS code-line are very exacting and Oracle learned after a few tries that their own custom built system is the only thing that cuts the mustard; of course, they have more money to spend on it than we do.

	Steve

	From:Durwin Wright 
	Sent: Tuesday, April 29, 2008 2:51 PM
	To: Stephen Ball; Andrew Ross
	Cc: Joseph C. Kronk; Teresa L. King; Alex Hanshaw; David Tondreau; Bodo Bergmann; Durwin Wright
	Subject: Piccolo to Subversion cheat sheet

	Good points Steve.  In the case of OpenROAD, during the Irish Revenue Commissioner Migration Project and in the Infor Adage Unicode Project, we needed to be able to take selected changes during the final phases of each project.  It was critical that we have the ability to skip changes. 

	I guess we could have setup separate branches for each project.  I wonder how cross integrations would work under those circumstances.

	Durwin Wright | Sr. Architect | Durwin.Wright at ingres.com | Ingres |500 Arguello Street | Suite 200 | Redwood City | CA | 94063 | USA <http://maps.google.com/maps?q=500+arguello+street,+94063&ll=37.487297,-122.233200&spn=0.004602,0.012771&t=k&hl=en>   +1 650-587-5523 | fax: +1 650-587-5550 

________________________________

		From:Stephen Ball 
		Sent: Monday, April 28, 2008 7:40 PM
		To: Andrew Ross; Durwin Wright
		Cc: Joseph C. Kronk; Teresa L. King; Alex Hanshaw; David Tondreau; Bodo Bergmann
		Subject: RE: Piccolo to Subversion cheat sheet

		Ultimately, I think the right solution to this problem is to store OpenROAD and Ingres in a directory structure in Subversion that makes it practical to check out either (or both), including overlapping files, without any mapping, and have it build O.K…..I don’t know what OpenROAD uses to build, but in Ingres I think this would be possible.

		BTW: I thought of another issue with Piccolo vs Subversion when talking to Bob this morning, which I know SE might have an issue with. In Piccolo, since it is possible to pull individual changes, we sometimes make a decision to create a build or a patch by pulling a later change without pulling a non-overlapping earlier change; e.g. we may pull change 1234 to get a fix, but not pull change 1233 because it contains a more controversial or irrelevant fix, in subversion, since the whole library *is* the revision, it’s not possible to do this. Also in piccolo, although it’s not possible to commit half a change, it *is* possible to pull only half the files for a change on to a client; in subversion it’s also not possible to do this; we use this less frequently because of the implication, but it has been done before. It seems we could solve the “laver” without the “earlier” problem by using branching; in piccolo this would be very messy, but maybe it’s handled better in Subversion….thoughts??? In any case, the “half a change” scenario is not possible in Subversion, anyone have any thoughts in this?

		Steve

		From: Andrew Ross 
		Sent: Tuesday, April 29, 2008 5:44 AM
		To: Durwin Wright
		Cc: Joseph C. Kronk; Teresa L. King; Stephen Ball; Alex Hanshaw; David Tondreau; Bodo Bergmann
		Subject: RE: Piccolo to Subversion cheat sheet

		Hi Durwin,

		While there are no map files in subversion, there is flexibility regarding which part of the tree you check out from. You can check out the root of the tree, or just a file and everything in between. You can also specify which version (revision in svn speak) you wish to pull. The destination path is what you specify on the local machine.This allows you similar flexibility you would be used to in piccolo with mapfiles and labels.

		I believe it would be feasible to use creating branching (see svn copy) to create a new structure for OpenROAD if that suits us better.

		If you're interested in storing the OpenROAD facilities next to the server and driver facilities, this is entirely possible and perhaps a good idea as it would allow checking out all dependencies by simply checking out from a point higher in the tree.

		The latter, baring any major reason not to that I'm not aware of, is my preference since it is simpler for people to get everything they want.

		Please let me know the follow up questions I'm sure this has triggered.

		Thank you,

		Andrew

________________________________

		From:Durwin Wright 
		Sent: April 28, 2008 3:20 PM
		To: Andrew Ross
		Cc: Joseph C. Kronk; Teresa L. King; Stephen Ball; Alex Hanshaw; Durwin Wright; David Tondreau; Bodo Bergmann
		Subject: RE: Piccolo to Subversion cheat sheet

		Hello Andrew,

		I have been watching in fascination the emails that have been flowing about the use of subversion.  In thinking about how we would use this in OpenROAD I have run into a stumbling block.  I would like to get your input on our problem.

		The Ingres Server consists of five major facilities: BACK, FRONT, COMMON, GL and CL.  OpenROAD also consists of five facilities: W4GL, FRONT, COMMON, GL and CL.  Four facilities are shared between Ingres and OpenROAD.  In Piccolo, we typically will map these four facilities to a version of Ingres that we want to build against.  Since there are no map files in subversion, how can we achieve something similar?  

		Alternatively, it seems to me that OpenROAD is simply another Ingres facility of similar to FRONT.  This would imply that OpenROAD should be built (optionally) built at the same time as Ingres.  This would be similar to how VDBA is built, i.e. only on certain platforms and underr certain circumstances.

		Do you have any ideas or thoughts?

________________________________

		From: Andrew Ross 
		Sent: Monday, April 28, 2008 11:50 AM
		To: Andrew Ross
		Subject: Piccolo to Subversion cheat sheet

		Hi Everyone,

		Following up on the discussion from the summit. I encourage you to try out subversion to see if there area any unexpected issues that might cause trouble for your workflow/role. For this reason, we've set up a test machine.

		Information on workflow is available here:

		http://bugs.ingres.com/

		and in particular here: http://bugs.ingres.com/wiki/DevelopmentBasics

		If you have any questions, please join us in the #ingres IRC channel. (information on getting there is on the bugs.ingres.com web site)

		So you have a handy copy, here's a quick:

		Cheat Sheet

		pulling the code

		p label <label>  > file; p get -l file, etc 

		becomes

		svn checkout <URL>

		(or svn co for short)

		e.g.

		svn co http://code.ingres.com/ingres/main ~/ingres/server

		This will pull the code and put it where ever you specify.

		* There are no map files in svn. You simply choose the correct path where you want to derive from. e.g. http://code.ingres.com/ingres/main for instance

		checking out the file

		There is no p open. All files are available to be edited from your editor of choice after svn co.

		* Since you don't need to communicate with the server, you can work offline after you pull the code. Some operations do require server access. We'll provide a list later for consideration.

		abandoning the file

		p release <file>

		becomes

		svn revert <file>

		checking what you've touched

		p opened

		becomes

		svn status

		updating your work area

		p need > file.txt

		p get -l file.txt

		becomes

		svn update

		reviewing differences

		p rcompare <file>

		becomes

		svn diff <file>

		* svn diff does a lot more than this such as compare the codeline between any two points in time  and can easily be redirected to produce patch files

		submitting

		p sc

		becomes

		svn commit --<username> -m "Fixed issue number #NN and #MM as well as prep for #OO"

		where NN, MM, and OO are ticket numbers at a working state in trac

		adding a file or directory to the repository

		svn add <file or dir>

		* if you add a directory, by default the contents of the directory are added recursively. you can disable this with -N or --non-recursive

		Andrew

Alex Hanshaw
Manager, Sustaining Engineering
Ingres Europe Limited
alex.hanshaw at ingres.com

PHONE +44 01753 559515
MOBILE+44 07793 757929

www.ingres.com <http://www.ingres.com/> 

This transmission is confidential and intended solely for the use of the recipient named above. It may contain confidential, proprietary, or legally privileged information. If you are not the intended recipient, you are hereby notified that any unauthorized review, use, disclosure or distribution is strictly prohibited. If you have received this transmission in error, please contact the sender by reply e-mail and delete the original transmission and all copies from your system. 

Alex Hanshaw
Manager, Sustaining Engineering
Ingres Europe Limited
alex.hanshaw at ingres.com

PHONE +44 01753 559515
MOBILE+44 07793 757929

www.ingres.com <http://www.ingres.com/> 

This transmission is confidential and intended solely for the use of the recipient named above. It may contain confidential, proprietary, or legally privileged information. If you are not the intended recipient, you are hereby notified that any unauthorized review, use, disclosure or distribution is strictly prohibited. If you have received this transmission in error, please contact the sender by reply e-mail and delete the original transmission and all copies from your system. 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ingres.com/pipermail/opensource-infrastructure/attachments/20080501/ae3a2b1d/attachment-0001.html