[os-infrastructure] svn and piccolo

Fri May 23 18:30:24 PDT 2008

Yes, having never looked at the merge mechanism in SVN I don't know
whether it's possible; the code can be isolated from the piccolo end but
I don't know whether it would fit into Subversion.

From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Alex Hanshaw
Sent: Friday, May 23, 2008 11:20 PM
To: Discussions about the infrastructure needed to support a true
opensourcecommunity
Cc: David Reed; Robert Kibble
Subject: RE: [os-infrastructure] svn and piccolo

You can pick a diff tool for manually resolving the differences that svn
has not been able to merge.

I don't think you can swap in a different merge mechanism.

 Alex

________________________________

From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Andrew Ross
Sent: 23 May 2008 13:02
To: Discussions about the infrastructure needed to support a true
opensourcecommunity
Cc: David Reed; Robert Kibble
Subject: RE: [os-infrastructure] svn and piccolo

That's an interesting idea Steve. Is it practical to extract piccolo
diff'ing code into a stand alone diff tool? If so, then it would work
just fine as svn allows us to specify any diff tool we want. Otherwise,
we'll go through an exercise of reviewing other diff tools to see which
one is best. I have no doubt we'll find something useable.

Regarding piccolo code pull times, I agree it isn't *the* deciding
factor by any means, but pulling a new work area in 5-10 minutes is
definitely a noticeable difference and benefit. If it's an hour in the
UK office, it is likely the bandwidth causing the throttling. Has anyone
tried pulling code from home in the UK? And of course as mentioned
earlier, svnsync is always an option and a good idea.

Andrew

________________________________

From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Stephen Ball
Sent: May 23, 2008 1:34 AM
To: Discussions about the infrastructure needed to support a true
opensourcecommunity
Cc: David Reed; Robert Kibble
Subject: RE: [os-infrastructure] svn and piccolo

On the diff tool....I have to say that we have been spoiled with
piccolo, it is particularly good at integrating changes both to head
revisions (easy) and across branches (more difficult), mostly because
the piccolo merge was specifically designed to deal with the Ingres code
base and the style of the Ingres coding standard. People may pan piccolo
as an old "crusty" tool, but there are certain things it does incredibly
well and we have come to rely on them to make our processes less labor
intensive.

Here's a radical idea; we own the source to piccolo and it is reasonably
well modularized, and SVN is open source and it would appear it is
possible to plug in external mergers; how about we use the piccolo merge
functions in SVN. Something to think about long-term.

We all know Subversion isn't the perfect tool, and the requirements of a
DBMS are complex and very specific, which means nothing will ever be
perfect. The need for a code-management maintainer is very long
over-due, this was the case even when we were using piccolo, and I don't
think a product like ours can get away with using a third party code
management tool without regularly contributing and customizing it for
our needs, at least with Subversion this is possible; so the questions
to ask are:

*         Can we live with it for the time being?

*         How much work would it take for us to fix it the way we want
it?

*         Will Trigis accept our changes?

*         Can we use code from piccolo to help us?

Steve

From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Andrew Ross
Sent: Friday, May 23, 2008 2:00 AM
To: Discussions about the infrastructure needed to support a true
opensourcecommunity
Cc: David Reed; Robert Kibble
Subject: RE: [os-infrastructure] svn and piccolo

Understood about the replicated piccolo server & database company
comment Alex. Perhaps you would like to race... you setting up &
documenting work flow for replicated piccolo vs. me setting up a
mirrored svn repository? 

;-) 

Regarding svn's build in merge capabilities. No argument is needed. The
svn team is very up front about the fact they make no attempt to provide
a leading diff tool. Pick whichever diff tool you like and tell svn to
use it. If we believe this were a real show stopper, then we must feel
the countless people using svn are daft.

In general, regarding mixed revision requirement will be a challenge to
most version control systems on the market (closed source or open). It
sounds like the answer to my question is that you're in favour of
running two systems, correct?

Andrew

________________________________

From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Alex Hanshaw
Sent: May 22, 2008 11:05 AM
To: Discussions about the infrastructure needed to support a true
opensourcecommunity
Cc: David Reed; Robert Kibble
Subject: RE: [os-infrastructure] svn and piccolo

Hi Andrew

 So far my experience of svn's merge capabilities is that it is
dreadful. Merging a change that adds a line and deletes a line against
an earlier

revision of a file failed miserably and required a manual merge.
Mercurial documentation includes an overview of other software and the
merge

capabilities of svn were one of the big criticisms aimed at svn. SE
crosses all changes to all forward code lines. Poor merge means a lot of
manual work.

 svn does not have labels, it has tags that turn into a branch so you
can pulled a mixed code revision with and use it for further code
submissions.

The number of times I pull an entire build is very small. Regular
incremental pulls to head revs not only mean that HOQA tests are

against current change but mean that piccolo network performance is not
an issue. Mixed revision labels from which we can work is a requirement.

svn does not provide that requirement. You would need to pull a tag work
out wha the fix was and then pull a second build against the main

code repository in order to submit the change and not make the tag a
branch. So now I'd have to pull 2 copies locally.

 Given my experience of svn's attempts to merge changes I'm not so sure
one system would actually be less work.

 If piccolo performance is really that bad I'm sure "p read" could be
set up to pull from a local read only replicated DB. If only we were a

database company with a replication product.

 Alex

________________________________

From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Andrew Ross
Sent: 22 May 2008 15:42
To: Discussions about the infrastructure needed to support a true
opensourcecommunity
Cc: David Reed; Robert Kibble
Subject: RE: [os-infrastructure] svn and piccolo

Hi Alex, Bruce, All

I haven't had time to try it yet, but I'm fairly certain we can mirror
the repository to all offices. You'd pull code or update from a
read-only mirror that's available on the local (gigabit) network. This
should take a deterministic amount of time and greatly reduce
inter-office (VPN?) code pull traffic.

The read-only mirror repositories would sync when someone commits code
to the master.

Updates will be committed against the master repository. Since we're
mostly only pushing tiny diffs, even going over a slow network for this
should still be zippy.

Andrew

p.s. Following up on the disk space & svn discussion. The repository
today consumes only 266MB of disk space, even with the branches people
have created. Borrowing a line from Steve, disk space is indeed cheap.
Having issues with disk space because people are using the system *a
lot* is a good problem to have.

________________________________

From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Alex Hanshaw
Sent: May 22, 2008 10:23 AM
To: Discussions about the infrastructure needed to support a true
opensourcecommunity
Cc: David Reed; Robert Kibble
Subject: RE: [os-infrastructure] svn and piccolo

Hi Bruce

 Piccolo is throttled on the network, svn is not.

 Svn took over an hour to come down in the uk, vs 5 minutes for Andrew.

 There may be a problem in setting up the OpenRoad community repository
given the code is shares from

different versions of Ingres. Some discussion with Andrew may be
required.

 Alex

________________________________

From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Bruce A. Lunsford
Sent: 22 May 2008 15:17
To: Discussions about the infrastructure needed to support a true
opensourcecommunity
Cc: David Reed; Robert Kibble
Subject: RE: [os-infrastructure] svn and piccolo

Sounds good to me, particularly if the syncing of piccolo-opensrc and
svn is automated, which it seems like it could be.  One other advantage
is that we don't lose the history in piccolo by a 1-time switchover to
svn (or alternatively have to come up with a way to get the history
migrated to svn).  One downside is that I was really enjoying the
under-10-minute refresh of an entire source build area using svn as
opposed to 8 hours or so with piccolo (which is why "p need" will
continue to be the favored approach with piccolo versus blowing away a
build area and starting from scratch).

>From the perspective of EDBC, this approach also helps because the EDBC
server code is now at the point of being put into piccolo.  This was
never possible before because of the lack of a piccolo mainframe client,
which was recently completed.  Moving to another CM system would have
meant porting another CM client to the mainframe (or sticking with
piccolo).

Regards,

Bruce

________________________________

From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Alex Hanshaw
Sent: Thursday, May 22, 2008 3:49 AM
To: opensource-infrastructure at lists.ingres.com
Cc: David Reed; Robert Kibble
Subject: [os-infrastructure] svn and piccolo

Hi all

 Robert Kibble today suggested a rather neat solution to our svn and
piccolo dilemma.

 The idea is that we create a piccolo branch off of main that is
identical and kept in tight lock step with

the svn repository.

 Every submission in the svn repository could then be replicated in the
piccolo-opensrc branch with a file

copy of the affected files. No need to merge difs.

 Every change in the svn repository could then be easily integrated into
main through the use of an ineed against

the piccolo-opensrc branch.

 Any change in main that we wished to push through into the svn
repository could be done by locking the affected

files in the svn repository at head rev, crossing the change from main
into the piccolo-opensrc branch (identical

at head rev to svn) and then whole file copy the piccolo-opensrc files
to the svn repository for submission.

 By forcing a tight lockstep of the svn repository and the
piccolo-opensrc branch replicating the changes between

the two repositories would be as simple as copying files.

 If we adopt this approach:

1)       We have zero disruption to our paying customers.

2)       We can provide a community codeline using svn.

3)       We can easily pull changes between the community repository and
our piccolo repository.

4)       No need to rewrite existing scripts and procedures built on and
around piccolo.

5)       Significantly reduced svn training costs.

6)       Ingres staff wanting to contribute to the community edition can
working in svn or piccolo (removes the barrier of not knowing svn).

 Let me know what you think.

 Regards

 Alex

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ingres.com/pipermail/opensource-infrastructure/attachments/20080523/dc79361b/attachment-0001.html