[os-infrastructure] State of the onion
Bodo Bergmann
Bodo.Bergmann at ingres.com
Fri Jun 6 12:27:12 PDT 2008
All this just to defend to something (handling Karl's changes) David
gave as an example for
one of the 4 key point he listed:
"yet to define how we actually prioritize and vet changes from the
outside" ???
But I don't see any argument in your reply that shows how it is defined.
Doing special things (e.g. branches) with/for Karl might help his
special situation,
but I don't think this is the kind of process for contributions we have
to follow.
I agree with David in his list of key points.
Bodo.
-----Original Message-----
From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
Andrew Ross
Sent: Friday, June 06, 2008 7:34 PM
To: Discussions about the infrastructure needed to support a true
opensourcecommunity
Subject: RE: [os-infrastructure] State of the onion
Hi David,
Regarding Karl's 200 updates.... As the guy who has worked most directly
with Karl to integrate his changes, I would like to share a few
observations to illustrate there is significant progress happening and
also that a big part of the 200 backlog is Karl's own doing.
The first time we accepted changes from external contributors, it took
us nearly a year to review, integrate, and test Karl's first batch of 67
changes. We agreed this was too long, and a problem, and we needed to do
something about it. Along the way we learned a few things including the
discovery that we need to prioritize reviewing his changes. We started
load balancing between support & my team so that bugs could be
prioritized and integrated more quickly. In fact, for the more recent
changes, we were waiting on Karl as he felt he'd like to take a second
kick at the updates.
We started working with patches generated from main rather than a
codeline more than a year old. We also put in place & published criteria
for the rejection of changes that anyone could see from the public wiki.
After these changes, some changes have flowed much more quickly
(integration in days). Others, such as a large compile warning cleanup
contribution got bogged down given the large scope.
In our defense, let me go on:
Despite requests to do so, Karl has not yet made talking to Ingres
people to pre-review his plans part of his practice. In the first 67, he
bundled nearly random cleanup with other changes. Also, he's been
batching the changes up into huge collections and holding them for
months. This makes it very hard to review. This also means, as is the
case with the 200, we haven't seen them yet. We don't even have an idea
of what they are. I've talked with him about this a few times. He knows
it's an issue. And we've talked about how we can remedy it. My point is
that Karl has significant responsibility to up his game so we can both
be successful. Had this been PostgreSQL, Linux, or other open source
projects, we wouldn't be having this conversation as the onus would be
strictly on Karl to shape up and meet our criteria.
I hope this is not misunderstood as speaking unkindly of Karl. Karl is a
talented and prolific developer. He is also a very active member of the
Ingres community. Thus, we feel without hesitation that it is worth it
to figure out a way to work effectively with him. Mike T and I had a
lengthy talk with Karl at the summit, and we are planning to leverage a
"karl branch" in svn as a way to work with Karl similar to keeping p &
svn in sync. We've done a lot of work to make the test suite run on a
machine without a huge amount of set up. This means we'll get fewer
surprises in terms of diffs we need to fix.
My point of sharing all this is so that people have a balanced view of
our work with Karl and some insight into the real improvements we have
made.
Andrew
-----Original Message-----
From: opensource-infrastructure-bounces at lists.ingres.com
[mailto:opensource-infrastructure-bounces at lists.ingres.com] On Behalf Of
David Tondreau
Sent: June 6, 2008 12:35 PM
To: opensource-infrastructure
Subject: Re: [os-infrastructure] State of the onion
OK, here it goes. I just can't hold back any more. This is going to be
a diatribe (and probably at first be perceived as negative). Regardless
of what you think, this is not from "an old horse not wanting to learn
new tricks." I am as committed to anyone else in this company to making
the organization successful. What I am looking for is a solution that
best aids the business in making money (as I for one, anyway, am a bit
attached to my paycheck). And what we are talking about here (to go
back to Durwin's statement) is the "cart <way> before the horse."
Let me start with a couple of key points:
1. No one has yet decided on the business model for the open source
version of the DBMS. Is there a community version with a separate
code line? Or are the crown jewels of "main" hung out for anyone
to contribute to. Are there separate builds for community and
commercial with a stable community build at some point or are we
going to continue with the current "community editions stop at
beta" nonsense. One would think we would allow the chosen
business model to drive the way we engineer, not vice versa.
2. No one has yet to define how we actually prioritize and vet
changes from the outside. Carl told me at the summit that he has
over 200 changes that he would like to push in and is originating
more every day. Can anyone tell me why Carl's changes aren't
making it into the source code? I'd venture to guess it has
NOTHING to do with the fact that the source code is hard to get
from him. He could probably send us a tarball of the changes in a
day. The problem is that a person inside actually has to review
the changes to see whether or not they merit being included in our
source code (or what has to be done to them to make them merit
that). All 200 of them! My guess is that we have neither the
time nor the priority to address this since everyone is 150%
booked and the likelihood is that Carl's changes, by his own
admission, are sometimes only 90% solutions. That is the ugly
truth about community contributions. All of them have to be
reviewed and many of them will take extra work to get to our
standard of engineering.
3. No one has yet to quantify the raw cost to the organization to
make this changeover and I challenge them to do so and present
that number to management. Calculate the time to provide
functional equivalence between piccolo and subversion, perform
software installation, training, convert user based build process,
test scripts, etc. This is a non-trivial process. One only need
to recall why Ingres was the only group in CA that never made the
move to Harvest...
4. Perhaps most important, from now until the end of time, 95% or
more of the code changes made to our product base will originate
from engineers within these walls. You can even put the code in
subversion, make it publicly available, make everybody in the
world a committer and still, 95% of the contributions will come
from within. Personally, I would be surprised if it were ever
more than 1 in 250 but 1 in 20 serves the purpose of this
exercise.
So why, pray tell, are we trying to wrap ourselves around a tree at an
unknown cost and unknown level of disruption to the business to satisfy
5% of the engineers working on the code? That simply doesn't make any
sense To hear some talk about it, the answer is "by gosh, we have got
to be an open source company and we have got to use subversion because
we need to use the most commonly accepted open source code management
tool or we will be the laughing stock of the world." Not to be unkind,
but this is all just plain BUNK. Lets take a look at some /_*big*_/
open source projects -- projects that Ingres compares to (mission
critical, huge code base, enterprise class solution) that /noone in
their right mind/ would accuse of not being "true open source":
*Mozilla*
Publicly accessible /read/only/ repository via CVS. Internally, they
decided in late 2007 to use Mercurial. Encourages download of source
code via tarball. From the Mozilla web site, here is the Mozilla
process for externally submitting a bug (read this entire thing):
"CVS is used to manage the documents on our web site. To submit
changes to a current document you can either use CVS to check out
the document <http://www.mozilla.org/contribute/writing/cvs#access>
and create a patch with cvs diff -u or you can click the "Edit this
page" link at the bottom of the document you are changing to make a
patch with Doctor.
Once you have a patch, submit the file to the owner of the document
by attaching it to a Bugzilla bug report and asking the owner to
review (and check in the changes, too, if you don't have access). If
the document doesn't list its owner, click on the "Document History"
link at the bottom of the page to see who has recently updated the
page and ask them. If all else fails, mail webmaster at mozilla.org
<mailto:webmaster at mozilla.org>.
As with the source code, if you establish a track record of good
work then you may be granted access to the repository, especially if
you contribute documents and become their official owner and
maintainer. To get access, you'll need to submit a CVS Contributor
Form <http://www.mozilla.org/hacking/form.html>. Read mozilla.org
Content and CVS <http://www.mozilla.org/contribute/writing/cvs> for
CVS information specific to our doc tree. The document Source Code
Via CVS <http://www.mozilla.org/cvs.html> explains where to get a
CVS client for your platform and has pointers to CVS documentation."
*OpenOffice.org*
Publicly accessible /read/only/ repository via CVS. Source available by
tarball. Submission process from the web site:
"How to submit code to OpenOffice.org
We ask that all code submitted to OpenOffice.org be submitted via
Issue Tracker
<http://qa.openoffice.org/issue_handling/project_issues.html>. In
your submission please list "Issue Type" as PATCH. Your code will be
sent to the committer for the appropriate project."
*MySQL*
Primary downloads are tarball snapshots. There is a read/only
subversion repository. Community contributions? Not really. From the
web site (note the word "envision" in the first sentence):
"This is how we envision the contribution process can flow:
1. Contributor suggests to work on new or existing WL, or Bug,
not in the list below.
1. This happens by emailing community-contributions <at>
mysql <dot> com.
2. We review and accept co-operation and include this
suggestion in the list below
2. . Contributor suggests to work on a WL or Bug in the list
below.
1. This happens by emailing community-contributions <at>
mysql <dot> com.
3. We sign up the Contributor, and assign the WL or Bug to
him/her.
4. We assign a MySQL mentor to the Contributor
5. The MySQL mentor will advice the Contributor on the work
process, and provide design and implementation guidance.
1. For WL's and more significant Bugs we might request to
understand your solution approach (or even design) before you should
start implementation, to ensure you don't spend time on something
that needs significant rewrites later
6. The Contributor provides patch for code-review, and MySQL
Mentor ensures timely feedback. This step might be repeated some
times as required.
1. Contributions should be provided under the Contributor
License Agreement (CLA), which the Contributor needs to approve of:
CLA
7. MySQL Mentor (or the Mentor's lead) approves the patch for
inclusion in Community Preview.
1. This Community Preview comes as snapshot binaries from
our build and test environment (called: Pushbuild). Main platforms
are supported (Linux, Windows, Solaris, perhaps Mac OS), other
platforms needs to be built from source.
2. MySQL is committed to providing regular snapshot updates
to this Community Preview, so that patches provided to us are made
available in a short timeframe.
8. Bugs towards the patch is filed in a normal fashion to
bugs.mysql.com (version = Community Preview) and by default bugs
will be assigned to Contributor.
9. After the patch has been seen successful in the Community
Preview, MySQL will decide on to include it in the appropriate MySQL
Server version. The default assumption is the current available
alpha version for WL's and the oldest relevant MySQL server version
(and forward) for Bugfixes.
The net-net here is that of these three well established open source
projects, all primary source download options are tarballs, two use cvs
and one uses subversion (hardly a quorum on subversion), all
repositories are read only and all use their bug tracking system to
allow external developers to submit code changes.
All this angst about public read/writable source repository and energy
expended to ensure our two copies of the truth are in sync is, in my
opinion, downright silly. I am fine if we want to figure out how to
make piccolo constantly refresh a publicly accessible read only
subversion or cvs code repository. That would put us on par with
Mozilla, OpenOffice.org and MySQL. But lets try to solve this public
committer thing when our inability to respond to public contributions to
the source code is eating our lunch (and stopping us from making money).
Until then, it seems to me that we are as open source as everyone else
-- just not very organized or transparent about it. And THAT, my
friends, is the bigger problem we should be trying to solve.
David
David Tondreau
Architect
*Ingres Corporation*
_david.tondreau at ingres.com <mailto:david.tondreau at ingres.com>_
*PHONE* +1 703.738.4811
*FAX* +1 650.587.5550
*_www.ingres.com_* <http://www.ingres.com/>
This transmission is confidential and intended solely for the use of the
recipient named above. It may contain confidential, proprietary, or
legally privileged information. If you are not the intended recipient,
you are hereby notified that any unauthorized review, use, disclosure or
distribution is strictly prohibited. If you have received this
transmission in error, please contact the sender by reply e-mail and
delete the original transmission and all copies from your system.
_______________________________________________
opensource-infrastructure mailing list
opensource-infrastructure at lists.ingres.com
http://lists.ingres.com/mailman/listinfo/opensource-infrastructure
_______________________________________________
opensource-infrastructure mailing list
opensource-infrastructure at lists.ingres.com
http://lists.ingres.com/mailman/listinfo/opensource-infrastructure
More information about the opensource-infrastructure
mailing list