Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between pages "Talk:Callisto Coordinated Update Sites" and "Build Policies and Procedures"

(Difference between pages)
m (Responses to: Concurrent Builds & Releases)
 
(DTP Build Policies and Procedures Wiki)
 
Line 1: Line 1:
== Talk Page started by Kim Moir (platform-releng) :-) ==
+
=DTP Build Policies and Procedures Wiki=
 
+
←[[Data Tools Platform Project | Back to DTP Main Page ]]
I've done a bit of editing to try and keep this page organized, and show what's been resolved and what still needs some active work. If anyone ever see's I've edited too much, please say so, as I'm sure it was unintentional. Eventually we may want to move "resolved and stale" issue to some separate history page and some of the "news" topics may deserve a page of their own, eventually. And ... by all means ... all committers to Callisto should feel free to contribute to these pages by organizing the content, or provide links out to other pages, etc. The end goal is to end up with a great "main" page that everyone finds accurate and a useful reference.
+
==DTP 0.7 Endgame Plan==
 
+
===Build Frequency===
-- [[User:David williams|David williams]] 21:49, 11 February 2006 (EST)
+
* 2/20 (Monday): Build holiday
 
+
* 2/21 (Tuesday) -- 2/24 (Friday): Daily builds
== Page talks tips ==
+
* Week of 2/27: Builds on 2/27 (Monday), 3/1 (Wednesday), 3/3 (Friday)
 
+
* Week of 3/6: Daily builds, Monday -- Friday (inclusive)
I believe the little "+" symbol next to the edit button at top should be used for these talks.
+
* Week of 3/13: Daily builds, Monday -- Thursday (inclusive)
--[[User:Droy|Denis Roy]] 16:50, 6 February 2006 (EST)
+
===Key Dates===
 
+
* 2/24 (Friday): Stabilization on Eclipse 3.2M5
Which will add a new level 2 section at bottom of page.
+
* 3/1 (Wednesday): DTP 0.7 feature freeze
 
+
* 3/16 (Thursday): DTP 0.7 final build
=== Responses to page talk sections ===  
+
* 3/17 (Friday): DTP 0.7 promotion
 
+
===Policies===
I think it also seems to work fairly well to add a level 3 section to "respond to" a level 2 section. I do not know if there's an easy short cut for that, I've just been doing it "manually". (with 3 equal sign's (title) 3 equal signs)  
+
'''Features'''
 
+
* Feature work allowed into code line through 6AM ET on 3/1
-- [[User:David williams|David williams]] 21:49, 11 February 2006 (EST)
+
* Additional feature work allowed into code line 3/1 to 3/3 with
 
+
** Prior agreement by owning project lead(s)
=== author date and time ===
+
** Notification to dtp-pmc mailing list
 
+
* No feature work allowed into code line week of 3/6 and later through 0.7 unless
And, if you don't know if yet, 4 tildes in a row automatically gets converted to your name, and the current date and time, for easy marking of who and when. I guess the bracket(initials-username)end-bracket just shows how out of date we are with this fancy new Wiki stuff :)
+
** Proposed to PMC
 
+
** Accepted by PMC
-- [[User:David williams|David williams]] 21:49, 11 February 2006 (EST)
+
** Notification to dtp-dev
 
+
** No serious objections registered on dtp-dev (as determined by PMC)
 
+
'''Bug Fixes'''
== URL's for Callisto updates ==
+
* Bug fixes allowed into code line through midnight 3/3 (Friday)
[kmoir] There is a problem with the url
+
* Bug fixes allowed into code line 3/6 (Monday) through midnight 3/10 (Friday)
http://update.eclipse.org/updates/callisto/
+
** Prior agreement by owning project lead(s)
 
+
** Notification to dtp-pmc mailing list
Currently, the platform team stores all its updates in this directory
+
* Bug fixes allowed into code line 3/13 (Monday) through midnight 3/16 (Thursday)
 
+
** Proposed to PMC
http://update.eclipse.org/updates
+
** Accepted by PMC
 
+
** Notification to dtp-dev
and subdirectories such as these
+
'''Documentation and Test Cases'''
 
+
* Non-code based documentation changes allowed through midnight 3/16 (Thursday)
http://update.eclipse.org/updates/3.0
+
* Code-based (JavaDoc) documentation changes subject to ''Bug Fix'' policies above
http://update.eclipse.org/updates/3.1
+
* Test cases (automated or otherwise) separate from build plug-ins allowed at any time
 
+
* Test cases (automated or otherwise) incorporated into build plug-ins subject to ''Bug Fix'' policies above
So I don't know that this is the best url for Callisto
+
 
+
http://update.eclipse.org/updates/callisto/
+
 
+
given that
+
<li>we already have platform content in that directory</li>
+
<li>we can't change the Apache alias for platform because this has been hardcoded in our feature.xmls</li>
+
<li>the unix permissions on the directory mean that the platform team would need to retain ownership in order to continue update the content in the existing update sites.</li>
+
 
+
[dmw: agreed, we should have 'callisto' first in the URL, and keep it "parallel" other Eclipse projects]
+
 
+
<span style="color:DarkGreen">[denis: I can't host this on update.eclipse.org/callisto. How about download.eclipse.org/callisto?]</span>
+
 
+
[nickb] How about: http://update.eclipse.org/updates/3.2/callisto.xml (in the same folder as site.xml) ?
+
[dmw] I think we're leaning toward http://download.eclipse.org/callisto, but wanted to comment on the comment ... I think we should avoid having version numbers in (any) directory ... isn't the long term goal to have a constant update site for all time, and just keep putting new plugins and features in there, as needed, so to speak? ... and, hopefully, there may start to be more and more that don't actually change, say from 3.2 to 3.3?]
+
 
+
[kmoir]Until the code to resolve this bug is implemented, we can't ensure that we can just reuse the same update site from release to release.  https://bugs.eclipse.org/bugs/show_bug.cgi?id=123162
+
+
=== Resolved: URL's for Callisto updates ===
+
 
+
These issues have been resolved as best they can, and the main page updated to reflect the update URL's
+
for Callisto will be  http://download.eclipse.org/callisto/
+
 
+
-- [[User:David williams|David williams]] 21:49, 11 February 2006 (EST)
+
 
+
And now already changed so the "top level" one will be
+
http://download.eclipse.org/callisto/releases
+
 
+
to avoid having other directories permenanently underneath the main update site directory. They
+
should be siblings for easier management and future flexibility.
+
 
+
-- [[User:David williams|David williams]] 00:43, 17 February 2006 (EST)
+
 
+
== How all-or-nothing and few-or-many-choices should Callisto be ==
+
 
+
[kmoir] Providing Callisto as a single download comprising all 10 projects provides better exposure to the newer projects. By the same token, it also forces people to download stuff that they may not necessarily want or ever use, thus significantly increasing the strain on the distribution infrastructure. It is also rather overwhelming for a new eclipse user.  My point is, perhaps there is a better way to educate users and provide choice while simultaneously exposing the full breadth of functionality available with Callisto.
+
 
+
[dmw: yep, this is the question to be answered empirically and via community feedback ... but if we get one packaging working ... a few others should be easy --- though, I'd never want to get to the point of a Callisto user having more than, say, 4 or 5 choices ... those users that want more fine grained control than that will have to use project update URLs]
+
 
+
[nickb] I agree that making people download EVERYTHING is both overkill and will overtax the infrastructure. However, as David points out, "enabling" projects like EMF and GEF are required for the more newb-friendly projects, so could we have a set of screenshots just showing people how to use UM and the various requirements they need to check off in order to get the projects they want? I like what WTP has done for their [http://download.eclipse.org/webtools/updates/ UM site], for example.
+
 
+
=== Resolved: we'll start with the "completely selectable" approach ===  
+
 
+
Since its easier, and allows better incremental approach to the end goal (e.g. there could be one failing or delayed feature, without preventing other updates). As before, we'll assess as we go, and remain open to future enhancements.
+
 
+
-- [[User:David williams|David williams]] 00:48, 17 February 2006 (EST)
+
 
+
== Tool to track downloads ==
+
 
+
By the way, I'm working on a tool to track download stats for EMF (and later, other projects who want to implement it), which combines
+
 
+
* a PHP script which runs SQL queries at the eclipse.org download stats database,
+
* a set of 3 cronjobs (actually, one shell script w/ three different options) for nightly/weekly/monthly querying using that interface, and
+
* a [http://emf.torolab.ibm.com/emf/downloads/downloads.php web UI] to display those results for trending and overall results (eg., combining monthly data across a year or weekly data across a quarter; or comparing 4 consecutive weeks to see if downloads increase or decrease over time)
+
 
+
The purpose for us is:
+
 
+
* to track how many hits we get in a day/week/month (survey says: [http://emf.torolab.ibm.com/emf/downloads/downloads.php?type=File&groups%5B%5D=groupSmall&groups%5B%5D=groupType&range=ym&rangeLimit=-1 over 2 million EMF jar downloads and 80,000 zip downloads in Dec & Jan])
+
* to track how those hits change overtime
+
* to determine which files are more popular than others (ie., should the Callisto UM site include XSD? survey says: [http://emf.torolab.ibm.com/emf/downloads/downloads.php?type=File&groups%5B%5D=groupSmall&thresh=1000&groups%5B%5D=groupProject&groups%5B%5D=groupType&range=ym "*xsd*"&&!"*emf*" to "*emf*" =~ 20:1 (zips)], which means EMF-only and EMF SDK (incl. XSD) zip downloads are 20x more downloaded than XSD-only zips)
+
* to compare UM installs vs. zip downloads, to see which is more prominently used (survey says: [http://emf.torolab.ibm.com/emf/downloads/downloads.php?type=File&groups%5B%5D=groupSmall&groups%5B%5D=groupType&range=ym&rangeLimit=-1 jars to zips =~ 19:1])
+
 
+
If you feel these tools/stats would be of value to other projects, I'd be happy to volunteer the code (and time to help set it up) for any other Callisto projects you'd like to track. See /cvsroot/org.eclipse/www/emf/downloads/
+
 
+
=== Responses: Tool to track downloads ===
+
 
+
Nick, thanks. First, your tool does sound very interesting ... I personally won't have time to investigate it much for a while, but I'd think we'd want these sorts of stats for <b>all</b> Eclipse projects, especially those in Callisto!
+
 
+
== Sizes involved with Callisto ==
+
 
+
Since several have voiced concerned over size of Callisto, I guess we should start to estimate its size.
+
 
+
Here's my off hand estimates of "runtime" sizes of several components (in MegaBytes).
+
These are mostly guesses at this point (but a few I'm "updated" recently), but if enough people provide estimate, we'll get a little better idea of what tradeoffs are.
+
 
+
[<b>Callisto meisters:</b> feel free to update your portions of table to be accurate ... using current milestones, or previous release milestones as estimates. Any rough estimate will do.]
+
 
+
[nickb] You can get numbers for your most recent builds like this:
+
 
+
* ssh <i>username</i>@dev.eclipse.org
+
* cd /home/data/httpd/download.eclipse.org/tools/emf/downloads/drops/2.2.0/I200602020000
+
* find . -name "*.zip" -exec du -sh {} \;
+
 
+
[/nickb]
+
 
+
If we literally included each (sub) project as a choice, that would be 15 choices ... which I think is too many for what I've thought Callisto was for. Perhaps someone could propose some concrete "groupings" that would satisfy majority of users (though, I'm pessimistic if we could reach much agreement?)
+
 
+
If we go that route, though, I don't think we would need to do anything, except maybe to get the platform plugin to provide all 10 "discovery sites"?
+
 
+
Also, maybe our <b>friendly webmaster</b> could comment on if these "increases in size" are of the size to be of concern .... such as is 75 Megs, say, that much different than 150? Is 150 that much different that 300? (I know "twice as big" sounds like a lot ... but .. its not like 10 times as much). And with our improved "nearest mirror" support, maybe it wouldn't matter?).
+
 
+
<span style="color:DarkGreen">[denis: also consider that the bigger it is, the longer it will take to reach all our mirrors. The good news is that Mike okay'd an extra 200Mb (!!) of bandwidth for (but not limited to) Callisto, in addition to and separate of the 100Mb for eclipse.org, so we'll be in good shape for actual distribution.]</span>
+
 
+
<table border="1" align="center">
+
 
+
 
+
                <tr valign="top">
+
                        <th>Project</th>
+
                        <th>Runtime Size</th>
+
                        <th>SDK Size</th>
+
                </tr>
+
 
+
 
+
        <tr valign="top">
+
                <td>Business Intelligence and Reporting Tools (BIRT)</td>
+
                <td>3</td>
+
                <td>?</td>
+
        </tr>
+
        <tr valign="top">
+
                <td>C/C++ IDE (CDT)</td>
+
                <td>10</td>
+
                <td>?</td>
+
        </tr>
+
        <tr valign="top">
+
                <td>Data Tools Platform (DTP)</td>
+
                <td>6</td>
+
                <td>?</td>
+
        </tr>
+
        <tr valign="top">
+
                <td>Eclipse Modeling Framework (EMF, SDO, XSD)</td>
+
                <td>EMF/SDO: 3.4, XSD: 1</td>
+
                <td>EMF/SDO/XSD: 23</td>
+
        </tr>
+
        <tr valign="top">
+
                <td>Eclipse Project (Platform only)</td>
+
                <td>15</td>
+
                <td>?</td>
+
        </tr>
+
        <tr valign="top">
+
                <td>Eclipse Project (PDE, JDT)</td>
+
                <td>60</td>
+
                <td>?</td>
+
        </tr>
+
 
+
        <tr valign="top">
+
                <td>Graphical Editing Framework (GEF)</td>
+
                <td>3</td>
+
                <td>?</td>
+
        </tr>
+
        <tr valign="top">
+
                <td>Graphical Modeling Framework (GMF)</td>
+
                <td>6</td>
+
                <td>?</td>
+
        </tr>
+
        <tr valign="top">
+
                <td>Test and Performance Tools Platform (TPTP)</td>
+
                <td>10</td>
+
                <td>?</td>
+
        </tr>
+
        <tr valign="top">
+
                <td>Visual Editor (VE, JEM)</td>
+
                <td>5</td>
+
                <td>?</td>
+
        </tr>
+
        <tr valign="top">
+
                <td>Web Tools Platform (WTP, WST, JST)</td>
+
                <td>25</td>
+
                <td>45</td>
+
        </tr>
+
                <tr valign="top">
+
                        <th>Totals</th>
+
                        <th>150</th>
+
                        <th>300</th>
+
                </tr>
+
 
+
</table>
+
 
+
 
+
== Concurrent Builds &amp; Releases ==
+
 
+
[https://bugs.eclipse.org/bugs/show_bug.cgi?id=116912 Release Train Cascade / RSS Notification & Response]
+
 
+
[nickb] There's been a fair amount of discussion about getting the Callisto projects to align in terms of their releases (before the subject of UM was broached). Should this be a page by itself, rather than a footnote here, by all means please do. The link above shows what's been done so far in terms of building a build cascade automation scheme, along with some comments from the community. Please post comments about this plan into that bug so that I
+
can integrate your requests into it, in order to make this solution more portable across all Callisto projects.
+
 
+
=== Responses to: Concurrent Builds &amp; Releases ===
+
 
+
Yes, I'd agree the information in the above referenced [https://bugs.eclipse.org/bugs/show_bug.cgi?id=116912 bug report] would make a good a Wiki page that spelled out the "how to" information ... step by step directions, links to example scripts, etc. I myself am not too concerned about cascaded builds, per se (we do "continuous builds", which would sort of lead to a combinatorial explosion if we and everyone followed it literally. Also, I haven't checked recently, but that discussion did not seem to address "cascading updates" ... which would be nice for developers (and by this I mean notifications of some new "weekly" update being available. That way, developers could perhaps automatically update their targets, with minimal effort.
+
<p>Just agreeing (I'm not volunteering :).</p>
+
<p>-- [[User:David williams|David Williams]] 19:57, 11 February 2006 (EST)</p>
+
 
+
David:
+
 
+
True, we haven't addressed cascading updates (in terms of RSS and cron scripts, anyway), but if all the projects publish builds & UM jars at the same time (like EMF, UML2 and EMFT does), then this UM cascade could happen automatically - or at least would be available for those who want to take advantage. I'm all for putting it out there, publishing where it is, making it easy to get it, and letting whoever wants it to get it, either via download page, RSS (link to download page or directly to build 's URL), or UM site.
+
<p>-- [[User:Nick Boldt|Nick Boldt]] 11:56, 24 February 2006 (EST)</p>
+
 
+
== Cross project requirements - best practices ==
+
 
+
[kmoir] On the same note, I couldn't find a document that gathered best practices for projects participating in Callisto
+
 
+
For instance, from the platform
+
 
+
http://www.eclipse.org/equinox/documents/plugin-versioning.html Using qualifiers<br>
+
http://www.eclipse.org/eclipse/platform-core/documents/3.1/run_from_jars.html Using jarred plugins and features<br>
+
signing jars ....etc.<br>
+
 
+
=== Responses to: Cross project requirements - best practices ===
+
 
+
Thanks Kim, I've adding the plugins-as-jars link to the main page's [[Callisto_Coordinated_Update_Sites#Related_Links_for_Further_Enjoyable_Reading_and_Reference]]
+
<br /> Thanks for pointing it out. (In general, all Callisto Committers should free to add references or annotations to
+
[[Callisto_Coordinated_Update_Sites#Related_Links_for_Further_Enjoyable_Reading_and_Reference]])
+
 
+
 
+
The versioning document was already linked there.
+
 
+
I didn't actually see much info in the jars document about "how to" sign plugins-as-jars. Has that effort or information moved further along?
+
 
+
And, by all means, I'm sure the community would use any "best practices" Wiki pages. If real general, beyond Callisto, I suggest you add pointers to them on
+
[[Development_Conventions_and_Guidelines]]. For example, there you will find a link to a kind-of-out-of-date
+
[[User_Interface_Guidelines]] which we in Callisto should also pay attention to.
+
 
+
<p>-- [[User:David williams|David Williams]] 20:13, 11 February 2006 (EST)</p>
+
 
+
<p>[kmoir] 11:01, 24 February 2006 (EST)</p>
+
There is a process to sign jars but it is not complete yet...releng, equinox/runtime team and foundation are working on it.
+
 
+
== Default download should be binary not SDK ==
+
 
+
The SDK is too big. This is bad because: a) most people don't need all of that stuff, b) it wastes bandwidth, and c) it makes us look bad compared to other products. Thus I recommend that the default set of features be the Platform runtime binary plus the JDT runtime binary. The key part here being "binary". So no source, no PDE, no doc on how to develop plug-ins, just user doc.
+
 
+
Furthermore I recommend that the default for all other features like WTP also be "binary". The source / developer stuff should be in its own feature, listed on the update site, easy to download, but just not the default. We need to make Eclipse/Callisto look "lean and mean" not "huge and ungainly".
+
--[[User:Ebb|Ebb]] 10:13, 24 February 2006 (EST)
+

Revision as of 14:29, 24 February 2006

DTP Build Policies and Procedures Wiki

Back to DTP Main Page

DTP 0.7 Endgame Plan

Build Frequency

  • 2/20 (Monday): Build holiday
  • 2/21 (Tuesday) -- 2/24 (Friday): Daily builds
  • Week of 2/27: Builds on 2/27 (Monday), 3/1 (Wednesday), 3/3 (Friday)
  • Week of 3/6: Daily builds, Monday -- Friday (inclusive)
  • Week of 3/13: Daily builds, Monday -- Thursday (inclusive)

Key Dates

  • 2/24 (Friday): Stabilization on Eclipse 3.2M5
  • 3/1 (Wednesday): DTP 0.7 feature freeze
  • 3/16 (Thursday): DTP 0.7 final build
  • 3/17 (Friday): DTP 0.7 promotion

Policies

Features

  • Feature work allowed into code line through 6AM ET on 3/1
  • Additional feature work allowed into code line 3/1 to 3/3 with
    • Prior agreement by owning project lead(s)
    • Notification to dtp-pmc mailing list
  • No feature work allowed into code line week of 3/6 and later through 0.7 unless
    • Proposed to PMC
    • Accepted by PMC
    • Notification to dtp-dev
    • No serious objections registered on dtp-dev (as determined by PMC)

Bug Fixes

  • Bug fixes allowed into code line through midnight 3/3 (Friday)
  • Bug fixes allowed into code line 3/6 (Monday) through midnight 3/10 (Friday)
    • Prior agreement by owning project lead(s)
    • Notification to dtp-pmc mailing list
  • Bug fixes allowed into code line 3/13 (Monday) through midnight 3/16 (Thursday)
    • Proposed to PMC
    • Accepted by PMC
    • Notification to dtp-dev

Documentation and Test Cases

  • Non-code based documentation changes allowed through midnight 3/16 (Thursday)
  • Code-based (JavaDoc) documentation changes subject to Bug Fix policies above
  • Test cases (automated or otherwise) separate from build plug-ins allowed at any time
  • Test cases (automated or otherwise) incorporated into build plug-ins subject to Bug Fix policies above

Back to the top