[os-infrastructure] svn and piccolo

Fri May 23 07:48:53 PDT 2008

The idea that Steve raised (using Piccolo and svn and synching them) is
the only idea that I will support in the short term.  It has the least
impact from a staffing impact standpoint.  The OpenROAD team is a very
small group with a very large set of responsibilities.  At this time, I
do not see that a move to svn will help our ability to do our jobs
effectively.  This will probably change over time but I am specifically
talking about the next 12 months.

I agree that svn is a very impressive product with lots of tools that
support its use.  This is particularly true when viewed as the
foundation of an Open Source offering.  But I see no reason why the
entire processes of our internal engineering have to be disrupted when
svn could be used as our outward facing code management system and
Piccolo could continue to be used as our internal system.  As much as I
dislike having two source code management systems, using Piccolo for
internal processes and svn for external with synchronization between
this is the "Least Bad Thing" that we can do.

This approach could then be used to work on improving the diff/merge
capabilities of svn by using the idea that Steve has described.  A move
to svn for all of our systems would then be predicated on acceptance and
integration of a possible Ingres Corporation submission to the IP
holders of svn.

I see us moving off Piccolo eventually.  I just do not see it happening
now.  Using svn as our outward facing tool seems like a good idea.  

For the record, we have committed to several OpenROAD customers to add
features to OpenROAD that will allow the code management of the OpenROAD
4GL to be integrated with Subversion.  We recently signed an Engineering
partnership with a large European customer to jointly develop the
OpenROAD infrastructure to achieve this goal.

Durwin Wright | Sr. Architect | Durwin.Wright at ingres.com
<mailto:Durwin.Wright at ingres.com>  | Ingres | 500 Arguello Street |
Suite 200 | Redwood City | CA | 94063 | USA
<http://maps.google.com/maps?q=500+arguello+street,+94063&ll=37.487297,-
122.233200&spn=0.004602,0.012771&t=k&hl=en>   +1 650-587-5523 | fax: +1
650-587-5550 

________________________________

From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Stephen Ball
Sent: Thursday, May 22, 2008 10:34 PM
To: Discussions about the infrastructure needed to support a true
opensourcecommunity
Cc: David Reed; Robert Kibble
Subject: RE: [os-infrastructure] svn and piccolo

On the diff tool....I have to say that we have been spoiled with
piccolo, it is particularly good at integrating changes both to head
revisions (easy) and across branches (more difficult), mostly because
the piccolo merge was specifically designed to deal with the Ingres code
base and the style of the Ingres coding standard. People may pan piccolo
as an old "crusty" tool, but there are certain things it does incredibly
well and we have come to rely on them to make our processes less labor
intensive.

Here's a radical idea; we own the source to piccolo and it is reasonably
well modularized, and SVN is open source and it would appear it is
possible to plug in external mergers; how about we use the piccolo merge
functions in SVN. Something to think about long-term.

We all know Subversion isn't the perfect tool, and the requirements of a
DBMS are complex and very specific, which means nothing will ever be
perfect. The need for a code-management maintainer is very long
over-due, this was the case even when we were using piccolo, and I don't
think a product like ours can get away with using a third party code
management tool without regularly contributing and customizing it for
our needs, at least with Subversion this is possible; so the questions
to ask are:

*         Can we live with it for the time being?

*         How much work would it take for us to fix it the way we want
it?

*         Will Trigis accept our changes?

*         Can we use code from piccolo to help us?

Steve

From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Andrew Ross
Sent: Friday, May 23, 2008 2:00 AM
To: Discussions about the infrastructure needed to support a true
opensourcecommunity
Cc: David Reed; Robert Kibble
Subject: RE: [os-infrastructure] svn and piccolo

Understood about the replicated piccolo server & database company
comment Alex. Perhaps you would like to race... you setting up &
documenting work flow for replicated piccolo vs. me setting up a
mirrored svn repository? 

;-) 

Regarding svn's build in merge capabilities. No argument is needed. The
svn team is very up front about the fact they make no attempt to provide
a leading diff tool. Pick whichever diff tool you like and tell svn to
use it. If we believe this were a real show stopper, then we must feel
the countless people using svn are daft.

In general, regarding mixed revision requirement will be a challenge to
most version control systems on the market (closed source or open). It
sounds like the answer to my question is that you're in favour of
running two systems, correct?

Andrew

________________________________

From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Alex Hanshaw
Sent: May 22, 2008 11:05 AM
To: Discussions about the infrastructure needed to support a true
opensourcecommunity
Cc: David Reed; Robert Kibble
Subject: RE: [os-infrastructure] svn and piccolo

Hi Andrew

 So far my experience of svn's merge capabilities is that it is
dreadful. Merging a change that adds a line and deletes a line against
an earlier

revision of a file failed miserably and required a manual merge.
Mercurial documentation includes an overview of other software and the
merge

capabilities of svn were one of the big criticisms aimed at svn. SE
crosses all changes to all forward code lines. Poor merge means a lot of
manual work.

 svn does not have labels, it has tags that turn into a branch so you
can pulled a mixed code revision with and use it for further code
submissions.

The number of times I pull an entire build is very small. Regular
incremental pulls to head revs not only mean that HOQA tests are

against current change but mean that piccolo network performance is not
an issue. Mixed revision labels from which we can work is a requirement.

svn does not provide that requirement. You would need to pull a tag work
out wha the fix was and then pull a second build against the main

code repository in order to submit the change and not make the tag a
branch. So now I'd have to pull 2 copies locally.

 Given my experience of svn's attempts to merge changes I'm not so sure
one system would actually be less work.

 If piccolo performance is really that bad I'm sure "p read" could be
set up to pull from a local read only replicated DB. If only we were a

database company with a replication product.

 Alex

________________________________

From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Andrew Ross
Sent: 22 May 2008 15:42
To: Discussions about the infrastructure needed to support a true
opensourcecommunity
Cc: David Reed; Robert Kibble
Subject: RE: [os-infrastructure] svn and piccolo

Hi Alex, Bruce, All

I haven't had time to try it yet, but I'm fairly certain we can mirror
the repository to all offices. You'd pull code or update from a
read-only mirror that's available on the local (gigabit) network. This
should take a deterministic amount of time and greatly reduce
inter-office (VPN?) code pull traffic.

The read-only mirror repositories would sync when someone commits code
to the master.

Updates will be committed against the master repository. Since we're
mostly only pushing tiny diffs, even going over a slow network for this
should still be zippy.

Andrew

p.s. Following up on the disk space & svn discussion. The repository
today consumes only 266MB of disk space, even with the branches people
have created. Borrowing a line from Steve, disk space is indeed cheap.
Having issues with disk space because people are using the system *a
lot* is a good problem to have.

________________________________

From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Alex Hanshaw
Sent: May 22, 2008 10:23 AM
To: Discussions about the infrastructure needed to support a true
opensourcecommunity
Cc: David Reed; Robert Kibble
Subject: RE: [os-infrastructure] svn and piccolo

Hi Bruce

 Piccolo is throttled on the network, svn is not.

 Svn took over an hour to come down in the uk, vs 5 minutes for Andrew.

 There may be a problem in setting up the OpenRoad community repository
given the code is shares from

different versions of Ingres. Some discussion with Andrew may be
required.

 Alex

________________________________

From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Bruce A. Lunsford
Sent: 22 May 2008 15:17
To: Discussions about the infrastructure needed to support a true
opensourcecommunity
Cc: David Reed; Robert Kibble
Subject: RE: [os-infrastructure] svn and piccolo

Sounds good to me, particularly if the syncing of piccolo-opensrc and
svn is automated, which it seems like it could be.  One other advantage
is that we don't lose the history in piccolo by a 1-time switchover to
svn (or alternatively have to come up with a way to get the history
migrated to svn).  One downside is that I was really enjoying the
under-10-minute refresh of an entire source build area using svn as
opposed to 8 hours or so with piccolo (which is why "p need" will
continue to be the favored approach with piccolo versus blowing away a
build area and starting from scratch).

>From the perspective of EDBC, this approach also helps because the EDBC
server code is now at the point of being put into piccolo.  This was
never possible before because of the lack of a piccolo mainframe client,
which was recently completed.  Moving to another CM system would have
meant porting another CM client to the mainframe (or sticking with
piccolo).

Regards,

Bruce

________________________________

From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Alex Hanshaw
Sent: Thursday, May 22, 2008 3:49 AM
To: opensource-infrastructure at lists.ingres.com
Cc: David Reed; Robert Kibble
Subject: [os-infrastructure] svn and piccolo

Hi all

 Robert Kibble today suggested a rather neat solution to our svn and
piccolo dilemma.

 The idea is that we create a piccolo branch off of main that is
identical and kept in tight lock step with

the svn repository.

 Every submission in the svn repository could then be replicated in the
piccolo-opensrc branch with a file

copy of the affected files. No need to merge difs.

 Every change in the svn repository could then be easily integrated into
main through the use of an ineed against

the piccolo-opensrc branch.

 Any change in main that we wished to push through into the svn
repository could be done by locking the affected

files in the svn repository at head rev, crossing the change from main
into the piccolo-opensrc branch (identical

at head rev to svn) and then whole file copy the piccolo-opensrc files
to the svn repository for submission.

 By forcing a tight lockstep of the svn repository and the
piccolo-opensrc branch replicating the changes between

the two repositories would be as simple as copying files.

 If we adopt this approach:

1)       We have zero disruption to our paying customers.

2)       We can provide a community codeline using svn.

3)       We can easily pull changes between the community repository and
our piccolo repository.

4)       No need to rewrite existing scripts and procedures built on and
around piccolo.

5)       Significantly reduced svn training costs.

6)       Ingres staff wanting to contribute to the community edition can
working in svn or piccolo (removes the barrier of not knowing svn).

 Let me know what you think.

 Regards

 Alex

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ingres.com/pipermail/opensource-infrastructure/attachments/20080523/7d65469c/attachment-0001.html