[os-infrastructure] Re: [os-engineering] Closed source components

[Joe Abbate]

Hi Chris,

Chris Clark wrote:
> RE email trail below and keeping closed source away from the public.
>
> From a discussion I had with Andrew, for the create source (Ingres) 
> tar ball from main scripts/process we have a mechanism that :
>
>    * filters whole directories
>    * filters files from directories that are not filtered by above
>
>
> How this would work for a sync process would probably be non-trivial 
> but do-able. If we moved whole sale to a new scs we may be able to use 
> permissions in the scs to prevent access to these closed source 
> components (but we are no where near a migration yet, and I'm not sure 
> if [for example] svn  has this kind of granularity).
>
> So for modules, this is a solvable (and partially solved today) problem.
>
> The tricky part is when we have closed source mixed in the public 
> facing code (possibly like the B1 code). One answer to this is a hard 
> and simple rule of, "no closed source in public facing code." This 
> would mean we would have calls to closed source in the public facing 
> code (probably in ifdefs) with the closed source contained in a 
> filtered module, this would have to mean that all closed source code 
> would need to be plug-able. I'm not sure if there are any other options.
>
> If we went this route, ingres!main could be filtered/sync'd and made 
> available in the public scs (say svn), which is pretty much what both 
> Proposal 1 and 3 at http://community.ingres.com/wiki/Community_Model 
> is about (but doesn't talk about how the sync works).
>
> Personally I'm now leaning towards Proposal 1 which would allow us 
> (and/or the community) to move to Proposal 3 if/once the community 
> decides they need to have features that we Ingres Corp will never want 
> in the enterprise version. This is a big change for me as I was very 
> much in favor of the buffer approach. With minor changes to how we 
> handle closed source pieces I think we can do without the buffer. The 
> community can benefit from being closer to the enterprise code as they 
> do not have to worry about integrations unless they feel the need to 
> diverge (once they diverge, they have to handle the integration problem).
>
> Again this is for Ingres, and not for OpenROAD.

I don't see this as simply a technical matter of instituting mechanisms, 
automated or human, to prevent or disallow"pollution" of the open source 
code with closed code and vice versa.  As I tried to explain before to 
Andrew, Piccolo's ingres!main has always been a forward-looking 
codeline, so until now changes to it could always be branched to an 
Enterprise codeline, e.g., ingres!ingres2006r3.  If a fix was made in an 
earlier codeline, like ingres26 and it was merged to main in order to 
then merge it into ingresr30, someone looking at a 'p ineed' in 
ingres2006r3 (or the original cross-integrator) could always do a 'p 
ignore' if the change didn't need to get propagated upstream.  With 
Community edition changes flowing into ingres!main, from the Enterprise 
side, ingres!main becomes like a two-headed Hydra with changes 
intertwined into the same code stream.  For example, if some geospatial 
changes were already in ingres!main because they had been accepted as a 
Community edition contribution, but we deemed they weren't ready for our 
Enterprise customers, we'd have be extra careful about integrating 
changes from ingres!main to ingres2006r3 (we could 'p ignore' them if it 
was late in the cycle but what if we thought we'd eventually add them to 
the release?).

I don't see a problem with Community and Enterprise edition changes 
flowing into a common repository, but if Community has some features 
that Enterprise can't or doesn't want to take and vice versa, it makes 
sense to have two separate trunks (Community *and* Enterprise) that feed 
(flow into) the shared repository and from which each
trunk can select what it wants.  Andrew keeps insisting that ingres!main 
does not equate with Enterprise because Enterprise is vetted in a branch 
like ingres2006r3, but in effect ingres!main is currently the 
antechamber to Enterprise (otherwise Bob wouldn't be running nightly 
builds of main), i.e., as soon as ingres2006r3 (9.2) goes GA on the core 
platforms and we're ready for the next release, we'll do a mass branch 
into, say, ingres94 (the next Enterprise release).  By having SE flow 
through any fixes in earlier codelines through ingres!main, we also 
attempt to minimize regressions in Enterprise code.  Imagine a year down 
the road, when we're ready for the yet another Enterprise release.  If 
Community changes have also been flowing directly into ingres!main, and 
some of them are not desired for Enterprise, (a) we'll probably have do 
nightly builds of ingres!main in the context of Enterprise, assuming 
something like #ifdefs are used in source files shared by both, and (b) 
we'll have to be more selective about what we branch.  Will all this 
extra effort be beneficial in helping us build a better, more stable 
Enterprise product?

Joe