Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between pages "Update Site Optimization" and "Adaptor Hooks"

(Difference between pages)
 
 
Line 1: Line 1:
−
== The Problem ==
+
== Overview ==
−
The Eclipse Install/Update design concept includes grouping artifacts called features which are published on an Update Site located on a remote server. A feature consists of the feature manifest file and other resources placed in a single JAR archive. When directed at the update site, Eclipse Update Manager must download each of these JARs and parse the manifest in order to perform activities such as site browsing, searching, dependency checking etc.
+
Since Eclipse 3.0 the Framework Adaptor API has been available in the Equinox OSGi Framework. A framework adaptor implementation is called upon by the Equinox OSGi Framework to perform a number of tasks.  A framework adaptor may be used to add functionality to the framework.
  
−
This approach works reasonably well for moderate update sites, but does not scale well for large sites like [http://www.eclipse.org/projects/callisto.php Callisto].  Each of the feature JARs is small, but opening a connection and downloading this small JAR is costly and adds up. Even worse, users need to pay this price BEFORE they even decide if they want to install anything from the site. A solution is needed to reduce the number of connections simply to browse or search the update site.
+
A single framework adaptor is specified when the framework is launched.  By default in Eclipse 3.0 this is set to the EclipseAdaptor. In order to add new functionality in an adaptor in Eclipse 3.0 and 3.1 it is required that the adaptor implementation either re-implement the complete framework adaptor API or extend one of the existing framework adaptor implementations.  This makes it impossible for two parties to add new functionality to the framework in separate adaptors at the same time because the Equinox OSGi Framework can only be configured to use one adaptor.
  
−
Once the features to install have been selected, Update needs to physically download plug-in JARs onto user's machine. At this point, payload size ceases to be trivial - a full Callisto download is several hundred megabytes. A technique to reduce the payload size would benefit users who are downloading the full Callisto set.
+
In Eclipse 3.2 a new hookable adaptor has been included that is used by default as the framework adaptor.  The framework adaptor API has remained unchanged for the most part in Eclipse 3.2.  What has changed is the actual implementation of the adaptor API. A new implementation of the adaptor API is now included which provides hooks that others can implement to provide functionality to the adaptor implementation.
  
−
== The Solution ==
+
== Hookable Adaptor ==
−
The solution comes in two parts: the site digest, and the use of [[Pack200]].  The site digest is produced by merging all the information needed for browsing and searching a site into one file that is archived for size and can be downloaded using one connection instead the many separate connections needed to download the features.  Pack200 is a jar compression utility that is part of J2SE 5.0 that will reduce the size of the jars significantly.
+
The hookable adaptor is implemented in the package org.eclipse.osgi.baseadaptor.  This adaptor implementation provides all of the default behavior required of a FrameworkAdaptor to provide an OSGi R4 compliant Framework.  It also provides many hooks that allow others to insert additional functionality into the framework through what are called framework extension bundles.  See the OSGi Core Specification chapter 3.15 "Extension Bundles" for more information.
  
−
Both these solutions require enhancements of the Install/Update code to make Update capable of consuming these artifacts. However these performance enhancements are optional and Install/Update should continue to perform as normal in their absence.
+
Framework extension bundles are fragments of the system bundle (org.eclipse.osgi).  As a fragment they can provide extra classes which the framework can use.  A framework extension bundle can define a set of hook implementations that are configured with the hookable adaptor (using a hookconfigurators.properties file).
  
 +
=== The Base Adaptor ===
 +
The class org.eclipse.osgi.baseadaptor.BaseAdaptor implements the interface org.eclipse.osgi.framework.adaptor.FrameworkAdaptor.  This class is used by default as the adaptor of the framework.  You should avoid extending this class, instead you should use hook implementations to add functionality to the BaseAdaptor.
  
−
== Builds, Update Sites and the Site Optimizer ==
+
In some cases it may be impossible to do what you want with the current set of adaptor hooks.  In this case you may be forced to extend the BaseAdaptor class to provide your own adaptor implementation.  If you find yourself in this situation then you should open a bug against Framework Equinox requesting a new hook method or interface.
−
<p>There are two sides to this solution, steps that must be taken during a component's build, and steps that are taken on the update site itself.</p>
+
−
<p>
+
−
To ensure that the jars downloaded from an update site are the same as jars downloaded in a zip distribution, the jars need to be normalized (or repacked) during the build process (see the [[Pack200|Pack200 wiki page]]).  This is especially true if the jars will be signed.  If the jars are being [[JAR Signing|sent to the Eclipse Foundation to be signed]], then this repacking will be done at that time.  The actual build of the digest and packing of the jars can be considered a separate step and can be done on the update site itself.</p>
+
−
<br>
+
−
===The Site Optimizer===
+
−
The org.eclipse.update.core bundle provides an application extension named org.eclipse.update.core.siteOptimizer which can be invoked from the command line.
+
−
<pre>
+
−
java -jar /eclipse/startup.jar -application org.eclipse.update.core.siteOptimizer [options]
+
−
</pre>
+
−
The site optimizer application exposes the digest builder and the jar processor.  The digest builder is the tool that creates the actual site digest, the jar processor is a tool that can repack, sign, pack or unpack a jar and all its nested jars recursively.
+
−
<p>
+
−
The site optimizer can be used during a build to do the repacking of the jars.  Exactly when it should be called depends on how the build is organized.  If the build first builds update jars that are repackaged into the download zips, then the optimizer should be run on those update jars before they are repackaged.  If the build produces the download zips first, then the optimizer should be run on the download zips.  In both cases, we have either a zip full of jars, or a zip full of directories that contain jars.  The site optimizer can take this zip as input and output a similarly shaped zip containing the repacked (and optionally signed) jars:
+
−
<pre>
+
−
java -jar /eclipse/startup.jar -application org.eclipse.update.core.siteOptimizer -jarProcessor
+
−
  -repack -outputDir ./out sdk.zip
+
  
−
java -jar /eclipse/startup.jar -application org.eclipse.update.core.siteOptimizer -jarProcessor
+
=== The Hook Registry ===
−
  -repack -sign sign_script.sh -outputDir ./out sdk.zip
+
The hook registry is implemented in the class org.eclipse.osgi.baseadaptor.HookRegistry. The hook registry is used to store all the hooks configured in the framework.  When the hook registry is initialized it will discover all the hook configurators installed in the framework and will call on them to add hooks to the registry.  The BaseAdaptor uses the hook registry at runtime to lookup and use the different hooks configured with the registry.
−
</pre>
+
 
−
</p>
+
==== Hook Configurators ====
−
<p>See the [[Pack200#Jar Processor|jar processor]] page for details on the options available for the jar processor.</p>
+
Hook configurators must implement the org.eclipse.osgi.baseadaptor.HookConfigurator interface.  Hook configurators can add one or more hook implementations to the hook registry using the various add methods on the registry.
−
<br>
+
 
−
===The Update Site===
+
==== Discovering Hook Configurators ====
−
<p>
+
In order for a hook configurator to be discovered by the hook registry its implementation must be accessable by the framework's classloader.  This implies that hook configurators must be built into the framework itself (org.eclipse.osgi) or be supplied by a framework extension bundle.  Again a framework extension bundle is really just a fragment to Framework (i.e org.eclipse.osgi or the System Bundle).
−
If the update site is going to contain packed jars, then the site.xml file should specify that it supports pack200 by setting the pack200 attribute: <code><site pack200="true"></code>.  This lets the Update Manager know that the site contains packed jars, and it will look for a .jar.pack.gz file beside the .jar file that it would normally download.  If the .jar.pack.gz file is found, it will be downloaded and unpacked, otherwise the .jar file is downloaded as normal.</p>
+
 
−
<p>
+
A hook configurator also must be declared in one of the following ways to indicate to the hook registry which classes should be loaded as hook configurators.
−
The site optimizer is used on the update site to build the digest and do the actual packing of the jars:
+
 
−
<pre>
+
===== hookconfigurators.properties files =====
−
java -jar /eclipse/startup.jar -application org.eclipse.update.core.siteOptimizer -digestBuilder
+
A hookconfigurators.properties file can be used to declare a list of hook configator classes.  The key hook.configurators is used in a hook configurators properties file to specify a comma separated list of fully qualified hook configurator classes.  For example, the Equinox Framework (org.eclipse.osgi) is shipped with a default set of hook configurators specified a hookconfigurators.properties file included in the org.eclipse.osgi.jar:
−
  -digestOutputDir=/eclipse/digest -siteXML=/eclipse/site/site.xml  -jarProcessor -pack -outputDir /eclipse/site /eclipse/site
+
 
−
</pre>
+
<code>
−
This command will build the digest and traverse the /eclipse/site directory structure and pack all the jars it finds.  The output of a pack is a .pack.gz file, so the result is that beside each jar, there will be a jar.pack.gz file.</p>
+
hook.configurators=
−
<br>
+
    org.eclipse.osgi.internal.baseadaptor.BaseHookConfigurator,
−
== Related Pages ==
+
    org.eclipse.osgi.internal.baseadaptor.DevClassLoadingHook,
−
*[[Callisto Coordinated Update Sites]]
+
    org.eclipse.core.runtime.internal.adaptor.EclipseStorageHook,
−
*[[JAR Signing]]
+
    org.eclipse.core.runtime.internal.adaptor.EclipseLogHook,
−
*[[Pack200|Pack200 and the Jar Processor]]
+
    org.eclipse.core.runtime.internal.adaptor.EclipseErrorHandler,
 +
    org.eclipse.core.runtime.internal.adaptor.EclipseAdaptorHook,
 +
    org.eclipse.core.runtime.internal.adaptor.EclipseClassLoadingHook,
 +
    org.eclipse.core.runtime.internal.adaptor.EclipseLazyStarter,
 +
    org.eclipse.core.runtime.internal.stats.StatsManager,
 +
    org.eclipse.osgi.internal.verifier.SignedBundleHook
 +
</code>
 +
 
 +
Quite a few hook configurators are automatically enabled by default withing the Equinox Framework.  The only hook configurator required for a fully functional OSGi R4 Framework is the org.eclipse.osgi.internal.baseadaptor.BaseHookConfigurator.  All other configurators declared above add extra functionality needed by eclipse and can be disabled.
 +
 
 +
Extension bundles may provide their own hookconfigurators.properties file to specify additional hook configurators.  The hook registry will discover all hookconfigurator.properties files on its classpath and will merge all declared configurator classes into one list.
 +
 
 +
===== osgi.hook.configurators property =====
 +
The osgi.hook.configurators configuration property is used to specify the list of hook configurators.  If this property is set then the list of configurators specified will be the only configurators used.  If this property is set then the hookconfigurators.properties files will not be processed for additional configurators. This property can be used in a config.ini to lock down the set of configurators to a specific set.
 +
 
 +
===== osgi.hook.configurators.include property =====
 +
The osgi.hook.configurators.include configuration property is used to add additional hook configurators.  This is helpful for configuring optional hook configurators.  Hooks that should be enabled by default should be included in a hookconfigurators.properties file. This property is ignored if the osgi.hook.configurators is set.
 +
 
 +
===== osgi.hook.configurators.exclude property =====
 +
The osgi.hook.configurators.include configuration property is used to exclude any hook configurators.  This is helpful for disabling hook configurators that are specified in hook configurator properties files.  This property is ignored if the osgi.hook.configurators is set.
 +
 
 +
== Hook interfaces ==
 +
 
 +
=== Adaptor Hook ===
 +
 
 +
=== Bundle File Factory Hook ===
 +
 
 +
=== Bundle File Wrapper Factory Hook ===
 +
 
 +
=== Bundle Watcher Hook ===
 +
 
 +
=== Class Loading Hook ===
 +
 
 +
=== Class Loading Stats Hook ===
 +
 
 +
=== Storage Hook ===
 +
 
 +
== Bundle Files ==
 +
 
 +
== Class Loaders ==
 +
 
 +
== Examples ==

Revision as of 11:48, 22 April 2006

Overview

Since Eclipse 3.0 the Framework Adaptor API has been available in the Equinox OSGi Framework. A framework adaptor implementation is called upon by the Equinox OSGi Framework to perform a number of tasks. A framework adaptor may be used to add functionality to the framework.

A single framework adaptor is specified when the framework is launched. By default in Eclipse 3.0 this is set to the EclipseAdaptor. In order to add new functionality in an adaptor in Eclipse 3.0 and 3.1 it is required that the adaptor implementation either re-implement the complete framework adaptor API or extend one of the existing framework adaptor implementations. This makes it impossible for two parties to add new functionality to the framework in separate adaptors at the same time because the Equinox OSGi Framework can only be configured to use one adaptor.

In Eclipse 3.2 a new hookable adaptor has been included that is used by default as the framework adaptor. The framework adaptor API has remained unchanged for the most part in Eclipse 3.2. What has changed is the actual implementation of the adaptor API. A new implementation of the adaptor API is now included which provides hooks that others can implement to provide functionality to the adaptor implementation.

Hookable Adaptor

The hookable adaptor is implemented in the package org.eclipse.osgi.baseadaptor. This adaptor implementation provides all of the default behavior required of a FrameworkAdaptor to provide an OSGi R4 compliant Framework. It also provides many hooks that allow others to insert additional functionality into the framework through what are called framework extension bundles. See the OSGi Core Specification chapter 3.15 "Extension Bundles" for more information.

Framework extension bundles are fragments of the system bundle (org.eclipse.osgi). As a fragment they can provide extra classes which the framework can use. A framework extension bundle can define a set of hook implementations that are configured with the hookable adaptor (using a hookconfigurators.properties file).

The Base Adaptor

The class org.eclipse.osgi.baseadaptor.BaseAdaptor implements the interface org.eclipse.osgi.framework.adaptor.FrameworkAdaptor. This class is used by default as the adaptor of the framework. You should avoid extending this class, instead you should use hook implementations to add functionality to the BaseAdaptor.

In some cases it may be impossible to do what you want with the current set of adaptor hooks. In this case you may be forced to extend the BaseAdaptor class to provide your own adaptor implementation. If you find yourself in this situation then you should open a bug against Framework Equinox requesting a new hook method or interface.

The Hook Registry

The hook registry is implemented in the class org.eclipse.osgi.baseadaptor.HookRegistry. The hook registry is used to store all the hooks configured in the framework. When the hook registry is initialized it will discover all the hook configurators installed in the framework and will call on them to add hooks to the registry. The BaseAdaptor uses the hook registry at runtime to lookup and use the different hooks configured with the registry.

Hook Configurators

Hook configurators must implement the org.eclipse.osgi.baseadaptor.HookConfigurator interface. Hook configurators can add one or more hook implementations to the hook registry using the various add methods on the registry.

Discovering Hook Configurators

In order for a hook configurator to be discovered by the hook registry its implementation must be accessable by the framework's classloader. This implies that hook configurators must be built into the framework itself (org.eclipse.osgi) or be supplied by a framework extension bundle. Again a framework extension bundle is really just a fragment to Framework (i.e org.eclipse.osgi or the System Bundle).

A hook configurator also must be declared in one of the following ways to indicate to the hook registry which classes should be loaded as hook configurators.

hookconfigurators.properties files

A hookconfigurators.properties file can be used to declare a list of hook configator classes. The key hook.configurators is used in a hook configurators properties file to specify a comma separated list of fully qualified hook configurator classes. For example, the Equinox Framework (org.eclipse.osgi) is shipped with a default set of hook configurators specified a hookconfigurators.properties file included in the org.eclipse.osgi.jar: 

hook.configurators= 
   org.eclipse.osgi.internal.baseadaptor.BaseHookConfigurator,
   org.eclipse.osgi.internal.baseadaptor.DevClassLoadingHook,
   org.eclipse.core.runtime.internal.adaptor.EclipseStorageHook,
   org.eclipse.core.runtime.internal.adaptor.EclipseLogHook,
   org.eclipse.core.runtime.internal.adaptor.EclipseErrorHandler,
   org.eclipse.core.runtime.internal.adaptor.EclipseAdaptorHook,
   org.eclipse.core.runtime.internal.adaptor.EclipseClassLoadingHook,
   org.eclipse.core.runtime.internal.adaptor.EclipseLazyStarter,
   org.eclipse.core.runtime.internal.stats.StatsManager,
   org.eclipse.osgi.internal.verifier.SignedBundleHook

Quite a few hook configurators are automatically enabled by default withing the Equinox Framework. The only hook configurator required for a fully functional OSGi R4 Framework is the org.eclipse.osgi.internal.baseadaptor.BaseHookConfigurator. All other configurators declared above add extra functionality needed by eclipse and can be disabled.

Extension bundles may provide their own hookconfigurators.properties file to specify additional hook configurators. The hook registry will discover all hookconfigurator.properties files on its classpath and will merge all declared configurator classes into one list.

osgi.hook.configurators property

The osgi.hook.configurators configuration property is used to specify the list of hook configurators. If this property is set then the list of configurators specified will be the only configurators used. If this property is set then the hookconfigurators.properties files will not be processed for additional configurators. This property can be used in a config.ini to lock down the set of configurators to a specific set.

osgi.hook.configurators.include property

The osgi.hook.configurators.include configuration property is used to add additional hook configurators. This is helpful for configuring optional hook configurators. Hooks that should be enabled by default should be included in a hookconfigurators.properties file. This property is ignored if the osgi.hook.configurators is set.

osgi.hook.configurators.exclude property

The osgi.hook.configurators.include configuration property is used to exclude any hook configurators. This is helpful for disabling hook configurators that are specified in hook configurator properties files. This property is ignored if the osgi.hook.configurators is set.

Hook interfaces

Adaptor Hook

Bundle File Factory Hook

Bundle File Wrapper Factory Hook

Bundle Watcher Hook

Class Loading Hook

Class Loading Stats Hook

Storage Hook

Bundle Files

Class Loaders

Examples

Retrieved from "https://wiki.eclipse.org/index.php?title=Update_Site_Optimization&oldid=4220"

Back to the top