[os-infrastructure] [os-engineering]: Model discussion boiled dry(er)

Sat Jun 28 10:10:23 PDT 2008

Let's take a step back and look at what we have been discussing from
another perspective.  Let's see if this makes sense to anyone.

Essentially Ingres Corporation had already adopted a "Fedora-like" model
at the time that OpenROAD forked from the Ingres codeline.  This
happened in the in the early 1990's -- before most of us where involved
in Ingres or OpenROAD.  This fork has persisted until now.  The also
company made a deliberate decision to not integrate (or harvest which is
more accurate) the OpenROAD contributions back into the Ingres codeline.
This is a fact.

I can outline the technical issues that we would need to solve before an
integration of OpenROAD and Ingres would be possible.  Here are only a
few of the notable problems that would need to be solved:  (1) OpenROAD
is not a single codeline; (2) It consists of two or three separate
codelines that are glued together using Piccolo map files and Piccolo
labels; (3) One of those codelines is a stable version of Ingres; (4)
The others are separate codelines for the OpenROAD 3GL and OpenROAD 4GL
and (5) Some of the OpenROAD source code is not really compatible with
any SCC.

We have started the smallest of steps towards solving these problems but
we still face at least two additional years of development effort before
we can even contemplate such integration.  I think it will be eventually
possible.  However, the lost of a stable version of Ingres in Piccolo
and the capabilities provided by Piccolo that OpenROAD relies on would
greatly impede our ability to support, develop and solve these problems
with our current resources.  From an OpenROAD standpoint, svn does not
meet our current needs.  It may in the future, but it does not meet them
now.  Piccolo does meet our current requirements.  

This does not mean that we cannot begin the task of changing our
processes and procedures in anticipation of the eventual replacement of
Piccolo.  This is why I want the emphasis to be on process and
procedures.  I understand that the decision to use svn to expose Ingres
to the outside world is driven by various reasons.  This is fair.

I do not believe that any model adopted by Ingres makes a lot of sense
for OpenROAD until we re-integrate the OpenROAD "contributions" back
into Ingres.  They are just different and are really forks that have
gone off in different directions over the last fifteen years.  (There
are many things that the DBMS could benefit that have already been
solved for years by OpenROAD.  Some examples are: (1) Declarative User
Defined datatypes; (2) Arrays; (3) Used defined method and (4) A
computationally complete 4GL language to create complex datatypes and
methods declaratively whose impedance matches that of Ingres.)

Since Ingres and OpenROAD are fundamentally different any solution that
is designed to expose each to the open source world must be investigated
separately.  We must be honest and be prepared for the possibility that
what works for one product is not appropriate for the other.

Before I continue I would like to find out if any of my previous
statements make sense.  Feel free to challenge or agree any of the
statements that I have made.

Durwin Wright | Sr. Architect | Durwin.Wright at ingres.com | Ingres | 500
Arguello Street | Suite 200 | Redwood City | CA | 94063 | USA  +1
650-587-5523 | fax: +1 650-587-5550 

-----Original Message-----
From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Andrew Ross
Sent: Saturday, June 28, 2008 4:38 AM
To: Joseph M. Abbate
Cc: Open Source Engineering; Discussions about the infrastructure needed
to support a true opensourcecommunity
Subject: RE: [os-engineering]FW: [os-infrastructure] Re: Model
discussionboileddry(er)

Hi Joe,

Regarding changes that community want that we don't necessarily want

back in the enterprise product:

This is another good example (the cluster one) which reminds me a lot of

something we're doing today with Karl for instance. 

I must say up front as there's a slight difference in that Karl

maintains perforce for his use. The rest of Datallegro uses svn. Also,

Datallegro has a business relationship so it isn't true community work

but more of a business partnership. The flow of content would be similar

so it's a valid use case to discuss.

Karl originally did development based on the tar files (ingres2006r2) we

provided on the web. By the time we got the changes to review and

integrate, the code was well over a year old. I integrated 72% of these

myself and came to celebrate heartily when there was a rare trivial

merge. There are a handful of others out there that had enough of a

taste of this to agree with me. If anyone disagrees with me, they're

quite welcome to take on a handful of outstanding submissions to help

them come around to my position. ;-)

We decided the tar files were grossly out of date and that it was best

to grant Karl (as a former employee) headrev access to piccolo via.

reactivating his account and VPN. We had no choice as there was no

public repository for us to use. This worked a lot better for getting

Karl content as he could get the latest bug fixes and features earlier.

It also made the merges go from almost certain manual integration to

almost certain trivial merge. This is a massive improvement.

The outstanding issue was that hands off qa wasn't available to Karl in

a practical sense. Thus, a key criteria

(http://community.ingres.com/wiki/Rules_of_the_road_for_contributing_cod

e_to_Ingres) for acceptance of the code has been an issue. We've done

work to provide set up tools and do a bit of tool cleanup. Now it's

possible for a community member to run the build, create an dbms

instance, and test it with a single command. This is another big

improvement in progress.

Let me recap the key parts of this relationship: Datallegro takes

content from us but holds some content back. We reject some of

Datallegro's content as it's meaningful for them, but not

needed/practical/safe for Ingres in general. Most importantly, access to

headrev on our side has made the flow of content go a lot easier.

Joe, this sounds a lot like your example. It's a case of how we've

handled it today, and the model I'm talking about has made it much

easier. Had we created a buffer branch, it would have just made for a

lot more work merging and not solved the content selection process. We'd

have to select and merge out stuff we don't want in any case.

Regarding tools and whether we need(ed) a public code repository:

Obviously the tar files were not the right fit with Karl. Also, they

would not have properly supported work between Warwick University,

Ilmenau University, and Ingres. Nor would it have properly supported

work between the many people interested in our new spatial support. Now

that we're establishing a convention of sharing the ported applications

out of the apps repository, it's much easier for others to contribute. A

public CM made this practical and much easier to do. Proper tools

clearly matter.

Regarding models:

Fedora is a very different animal than Ingres. First of all, Fedora is a

collection of countless vibrant open source projects (bash, screen, gcc,

apache, samba, etc.) The community was there and independent of RedHat

to begin with via. GNU and others. The community would have been there

independent of Fedora and RedHat (see Debian, Ubuntu, Suse, FreeBSD,

etc.) There was even a Fedora Linux project before the Fedora project

got started (there's no open source Ingres project grinding along

despite the company that I'm aware of). 

All of this aside, I believe in the value of allowing community to run

ahead. proposal 1 will support this.  A community spin off is possible

on demand at any time. The proposal lets us have this option without

committing us to all sorts of work to maintain a buffer branch from the

start. It's a model I've used with massive amounts of code (40M+ of

telecom platform/cluster code) and developers (100's I knew of well,

1000's beyond that I only knew were out there) before with success so I

know it works.

This is great discussion, detailed and specific so it's easier to dig

into what matters about it. Thanks Joe.

Andrew

-----Original Message-----

From: Joseph M. Abbate 

Sent: June 27, 2008 2:57 PM

To: Andrew Ross

Cc: Discussions about the infrastructure needed to support a true

opensource community; Open Source Engineering

Subject: Re: [os-engineering]FW: [os-infrastructure] Re: Model

discussionboiled dry(er)

Hi Andrew,

Andrew Ross wrote:

> This looks like the way it works today in piccolo, and has been that 

> way for a while. Have I misunderstood? If someone made a change to 

> dm2r.c in r2, and propped it to r3 by way of main (or made the change 

> in main and patched it to r3), then any feature content or bug fixes 

> affecting dm2r.c in main will always have to be contended with.

Yes, that is the way it works today and it's not a major problem because

"main" is generally a step or two ahead from the current enterprise

codelines. It *is* possible for somebody to make a change in r2 and then

manually integrate it into r3. Typically this doesn't happen between

currently active codelines, but does become necessary say between

oping20 and ingres2006r2, most likely because a feature was

significantly changed between 2.0 and 9.1 (clusters come to mind).

Which reminds me of another example: let's say some "community" 

contributor (perhaps with initials AR) decided to re-implement the Linux

cluster support in a community branch in a significantly different way

than is currently present in r2/r3/main. For argument sake, let's say

there wasn't much interest from our commercial customers on a Linux

cluster solution, but we do have several VMS cluster customers that are

depending on the current implementation (or the 2.0 implementation). How

would we handle these divergencies in a single consolidated repository,

containing both enterprise and community code?

> In general, I'm reading what you've wrote what do we do when we have 

> to fork certain files for good reason and how to deal with the 

> complexity it could cause. (please let me know if I've misunderstood) 

> The proposal

> 1 model is optimized for keeping the code synchronized rather than 

> optimizing it for scenarios where it needs to fork. There's an 

> implicit assumption that the latter is going to happen less often.

That is the latent issue in this entire discussion. I believe those

preferring a "Fedora model" think Ingres Database Community Edition

ought to be or is likely to become the "David Database" (Dah-vid, for

Jacques-Louis David), free to pursue its open source aspirations as seen

by its contributors. Those preferring otherwise (I don't want to call it

"fresh" because there can still be an immediately synchronized

repository pair in the Fedora model), believe the Community and

Enterprise editions ought to go along more or less at the same pace,

each including most of the features of the other, with Community perhaps

taking some riskier stances at first, but eventually being folded back

into Enterprise. The non-Fedora model also sees Ingres Corp as the

ultimate arbiter as to what goes into the official Community repository,

because after all, it is going to be integrated into Enterprise or

otherwise carry the Ingres brand name. OTOH, the Fedora model requires

that Ingres Corp place its trust on a somewhat separate entity, even if

initially we'll be guiding it along.

Joe

_______________________________________________

opensource-infrastructure mailing list

opensource-infrastructure at lists.ingres.com

http://lists.ingres.com/mailman/listinfo/opensource-infrastructure

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ingres.com/pipermail/opensource-infrastructure/attachments/20080628/5f5c56c2/attachment-0001.html