Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between pages "PTP/planning/remote" and "Adaptor Hooks"

< PTP‎ | planning(Difference between pages)
(Future)
 
 
Line 1: Line 1:
= Evaluation of Related Works for Remote Service Support =
+
== Overview ==
 +
Since Eclipse 3.0 the Framework Adaptor API has been available in the Equinox OSGi Framework.  A framework adaptor implementation is called upon by the Equinox OSGi Framework to perform a number of tasks.  A framework adaptor may be used to add functionality to the framework.
  
List of Authors:
+
A single framework adaptor is specified when the framework is launched.  By default in Eclipse 3.0 this is set to the EclipseAdaptor. In order to add new functionality in an adaptor in Eclipse 3.0 and 3.1 it is required that the adaptor implementation either re-implement the complete framework adaptor API or extend one of the existing framework adaptor implementations.  This makes it impossible for two parties to add new functionality to the framework in separate adaptors at the same time because the Equinox OSGi Framework can only be configured to use one adaptor.
    Tianchao Li (lit@in.tum.de)
+
    Greg Watson (gwatson@lanl.gov)
+
    (future authors)
+
  
== Background ==
+
In Eclipse 3.2 a new hookable adaptor has been included that is used by default as the framework adaptor.  The framework adaptor API has remained unchanged for the most part in Eclipse 3.2.  What has changed is the actual implementation of the adaptor API.  A new implementation of the adaptor API is now included which provides hooks that others can implement to provide functionality to the adaptor implementation.
  
=== The Need for Remote Compile, Launch and Debug ===
+
== Hookable Adaptor ==
The current implementation of PTP (v1.0) provides basic support for the coding, compiling, launching and debugging of parallel jobs on a local parallel machine. However, a more typical situation involves one or more of these activities ocurring on a remote machine. In a practical configuration, the development of parallel programs often involves four environments:
+
The hookable adaptor is implemented in the package org.eclipse.osgi.baseadaptor.  This adaptor implementation provides all of the default behavior required of a FrameworkAdaptor to provide an OSGi R4 compliant Framework. It also provides many hooks that allow others to insert additional functionality into the framework through what are called framework extension bundles.  See the OSGi Core Specification chapter 3.15 "Extension Bundles" for more information.
  
; User desktop : the user environment that supports coding and provides UI for compiling, launching and debugging
+
Framework extension bundles are fragments of the system bundle (org.eclipse.osgi).  As a fragment they can provide extra classes which the framework can use.  A framework extension bundle can define a set of hook implementations that are configured with the hookable adaptor (using a hookconfigurators.properties file). 
; File server : the machine on which the project files reside
+
; Build environment : the machine on which the program is to be built, which provides an environment with the same libraries and services available on the parallel machine
+
; Parallel machine : the machine on which the program is executed and debugged
+
  
It is possible that any combination of these machines may be physically remote, however it is normal for the user's desktop to be remote from the other machines. PTP must be able to support this range of configurations if it is to meet the needs of application developers.
+
=== The Base Adaptor ===
 +
The class org.eclipse.osgi.baseadaptor.BaseAdaptor implements the interface org.eclipse.osgi.framework.adaptor.FrameworkAdaptor.  This class is used by default as the adaptor of the framework. You should avoid extending this class, instead you should use hook implementations to add functionality to the BaseAdaptor.
  
=== Project Configuration Alternatives ===
+
In some cases it may be impossible to do what you want with the current set of adaptor hooks.  In this case you may be forced to extend the BaseAdaptor class to provide your own adaptor implementation.  If you find yourself in this situation then you should open a bug against Framework Equinox requesting a new hook method or interface.
  
There are three different possible ways that projects may be hosted in Eclipse:
+
=== The Hook Registry ===
 +
The hook registry is implemented in the class org.eclipse.osgi.baseadaptor.HookRegistry. The hook registry is used to store all the hooks configured in the framework.  When the hook registry is initialized it will discover all the hook configurators installed in the framework and will call on them to add hooks to the registry.  The BaseAdaptor uses the hook registry at runtime to lookup and use the different hooks configured with the registry.
  
; Local Project : the project and all code of the parallel program is stored in the local file system of the user's desktop.
+
==== Hook Configurators ====
; Remote Project : the project and all code is physically located on a remote machine (either a dedicated machine or one of the build machine or parallel machine).
+
Hook configurators must implement the org.eclipse.osgi.baseadaptor.HookConfigurator interfaceHook configurators can add one or more hook implementations to the hook registry using the various add methods on the registry.
; Hybrid Project : the user desktop physically holds the project (in particular the project configuration information) but some or all of the code is physically located on a remote machine.
+
  
The process of building, launching and debugging applications differs for each combination of local and remote resources.
+
==== Discovering Hook Configurators ====
 +
In order for a hook configurator to be discovered by the hook registry its implementation must be accessable by the framework's classloader.  This implies that hook configurators must be built into the framework itself (org.eclipse.osgi) or be supplied by a framework extension bundle.  Again a framework extension bundle is really just a fragment to Framework (i.e org.eclipse.osgi or the System Bundle).
  
==== Local Project ====
+
A hook configurator also must be declared in one of the following ways to indicate to the hook registry which classes should be loaded as hook configurators.
  
In the case where the user's desktop is the central location where the whole project is physically located, the following capabilities are required to support remote build, launch and debug:
+
===== hookconfigurators.properties files =====
 +
A hookconfigurators.properties file can be used to declare a list of hook configator classes.  The key hook.configurators is used in a hook configurators properties file to specify a comma separated list of fully qualified hook configurator classes.  For example, the Equinox Framework (org.eclipse.osgi) is shipped with a default set of hook configurators specified a hookconfigurators.properties file included in the org.eclipse.osgi.jar:
  
'''Remote Build'''
+
<code>
 +
hook.configurators=
 +
    org.eclipse.osgi.internal.baseadaptor.BaseHookConfigurator,
 +
    org.eclipse.osgi.internal.baseadaptor.DevClassLoadingHook,
 +
    org.eclipse.core.runtime.internal.adaptor.EclipseStorageHook,
 +
    org.eclipse.core.runtime.internal.adaptor.EclipseLogHook,
 +
    org.eclipse.core.runtime.internal.adaptor.EclipseErrorHandler,
 +
    org.eclipse.core.runtime.internal.adaptor.EclipseAdaptorHook,
 +
    org.eclipse.core.runtime.internal.adaptor.EclipseClassLoadingHook,
 +
    org.eclipse.core.runtime.internal.adaptor.EclipseLazyStarter,
 +
    org.eclipse.core.runtime.internal.stats.StatsManager,
 +
    org.eclipse.osgi.internal.verifier.SignedBundleHook
 +
</code>
  
# The code and build information (e.g. Makefiles) must be first transferred to the build machine (possibly through project export).
+
Quite a few hook configurators are automatically enabled by default withing the Equinox Framework. The only hook configurator required for a fully functional OSGi R4 Framework is the org.eclipse.osgi.internal.baseadaptor.BaseHookConfigurator.  All other configurators declared above add extra functionality needed by eclipse and can be disabled.
# The build command is invoked on the build machine.
+
# The build log is transferred back to the local machine and used to update the local models and views (problem view, marker view etc).
+
# The binary is transferred back to the local machine.  
+
  
'''Remote Launch & Debug '''
+
Extension bundles may provide their own hookconfigurators.properties file to specify additional hook configurators.  The hook registry will discover all hookconfigurator.properties files on its classpath and will merge all declared configurator classes into one list.
  
# The control/monitoring/resource management system is invoked on the parallel machine.
+
===== osgi.hook.configurators property =====
# The binary is transferred to the parallel machine.
+
The osgi.hook.configurators configuration property is used to specify the list of hook configurators. If this property is set then the list of configurators specified will be the only configurators used. If this property is set then the hookconfigurators.properties files will not be processed for additional configurators.  This property can be used in a config.ini to lock down the set of configurators to a specific set.
# Input data files are transferred to the parallel machine.
+
# The launch command is invoked on the parallel machine.
+
# The user desktop can then control the execution and debugging of the program.
+
# The stdio and stderr is trapped and sent back to the user desktop.
+
# Any output data files are transferred back to the user desktop.
+
  
==== Remote Project ====
+
===== osgi.hook.configurators.include property =====
 +
The osgi.hook.configurators.include configuration property is used to add additional hook configurators.  This is helpful for configuring optional hook configurators.  Hooks that should be enabled by default should be included in a hookconfigurators.properties file.  This property is ignored if the osgi.hook.configurators is set.
  
In the case that a remote machine physically holds the project code, the following capabilities are required to support remote build, launch and debug:
+
===== osgi.hook.configurators.exclude property =====
 +
The osgi.hook.configurators.include configuration property is used to exclude any hook configurators.  This is helpful for disabling hook configurators that are specified in hook configurator properties files.  This property is ignored if the osgi.hook.configurators is set.
  
'''Code Edit'''
+
== Hook interfaces ==
# The user environment accesses the project files through the remote file system.
+
  
'''Remote Build'''
+
=== Adaptor Hook ===
  
# The code and build information (e.g. Makefiles) should be transferred to the build machine (possibly through project export), if the build machine is different from the machine that physically holds the project.
+
=== Bundle File Factory Hook ===
# The build command is invoked on the build machine.
+
# The build log is transferred back to the local machine and updates local models and views (problem view, marker view etc).
+
# The binary is transferred to the remote machine that holds the project.
+
  
'''Remote Launch & Debug '''
+
=== Bundle File Wrapper Factory Hook ===
  
# The control/monitoring/resource management system is invoked on the parallel machine.
+
=== Bundle Watcher Hook ===
# The binary is transferred to the parallel machine, if different from build machine.
+
# Input data files are transferred to the parallel machine, if different from build machine.
+
# The launch command is invoked on the parallel machine.
+
# The user desktop can then control the execution and debugging of the program.
+
# The stdio and stderr is trapped and sent back to the user desktop.
+
# Any output data files are transferred back to the build machine, if necessary.
+
  
==== Hybrid Project ====
+
=== Class Loading Hook ===
  
If some project files reside on the user's desktop and some on a remote fileserver, then both sets of files need to be reconciled prior to building and/or launching. Once the files have been reconciled, the hybrid project can be treated as a remote project.
+
=== Class Loading Stats Hook ===
  
=== Required Functionality ===
+
=== Storage Hook ===
  
Additional functionality is required to handle each of the project configuration cases:
+
== Bundle Files ==
  
==== Local Project ====
+
== Class Loaders ==
  
# File transfer between local machine and remote machine (replicate project to remote machine and update project with remote changes, using import/export or team synchronization)
+
== Examples ==
# Execute commands on remote machines
+
# Access services on the remote machine (proxy via standard TCP/IP)
+
 
+
==== Remote Project ====
+
 
+
# Create project on remote file system
+
# Initiate transfer of files between remote machines from the local machine
+
# Execute commands on remote machines
+
# Access services on the remote machine (proxy via standard TCP/IP)
+
 
+
==== Hybrid Project ====
+
 
+
# Transfer files between local and remote machine and initiate transfer between remote machines
+
# Otherwise, same as remote project
+
 
+
== Related Work ==
+
 
+
There is already some work in both the Eclipse platform and the DSDP project that is relevant to PTP. Almost all of this work is still at a relatively primitive stage however, and we summarize below both their current status and proposed future development targets.
+
 
+
=== Alternate File System ===
+
The ability to support alternative file systems within Eclipse, including non local file systems, has already been requested by a number of people from different areas. Some work has already been undertaken to provide this support.
+
 
+
==== Status ====
+
Starting from Eclipse 3.2 M5, alternative file system support has been partially added. There are two extension points involved:
+
 
+
* org.eclipse.core.filesystem (defined in org.eclipse.core.filesystem plugin) - core support for file system
+
* org.eclipse.ui.ide.filesystemSupport (defined in org.eclipes.ui.ide plugin) - ui support for file system
+
 
+
Existing implementations provided by the Eclipse core for file system include: local, zip, memory.
+
The work on FTP file system seems to be going but the current code in the CVS is broken.
+
 
+
There is currently no UI extension for FTP file system. The UI support for memory file system is also not working. The UI support for zip file system is not complete. Bugs (137878, 137879) have been submitted.
+
 
+
For more information about the alternate file system, please reference http://www.eclipse.org/eclipse/platform-core/documents/3.2/flexible_workspaces/ and bug item 106176 and 109194.
+
 
+
RSE has also provided an extension to the org.eclipse.core.filesystem extension point, which is a wrapper of its remote file subsystem. However, no extension to org.eclipse.ui.ide.filesystemSupport has been provided and thus it is not possible to use it directly.
+
 
+
Projects and Linked Resources can now be created with alternative file systems (unfortunately only the zip file system is merely operable).
+
 
+
The 3.2 M5 release notes say that JDT has support non local file systems. However, no actual support is seen in the distribution. And bug 113373 indicates that the work is actually pending.
+
 
+
==== Future ====
+
I expect that the final release of Eclipse 3.2 and future 3.3 will provide better support for alternative file systems in both the platform level and the JDT level. However, other projects might need some time to catch up with such a change. Fundamental efforts might even be necessary to provide full support for some of the projects, like CDT, Photran etc.
+
 
+
=== Remote System Explorer ===
+
More specifically, RSE provides UI and API support for browsing and manipulating files on remote systems, execute commands on remote systems, monitoring processes on remote systems and searching
+
 
+
==== Status ====
+
The code currently in the CVS are solely those contributed by IBM. Many of the IDs are still prefixed by "com.ibm.rse". An experiment of the RSE shows that it basically works as it claimed. However, it does not meet our needs and possibly also does not meet the need of DSDP project:
+
 
+
# As the name suggests, the RSE is intended to be an explorer (or shell). The whole infrastructure is designed toward this aim. So, instead of trying to integrate and contribute to the underlying Eclipse platform, the RSE sets up a closed environment that are divided into a core and different subsystems. The design of UI components also towards the same target.
+
# The file subsystem and search subsystem of RSE are actually a duplicate functionality with the emerge of alternate file system support from the Eclipse core. Although a wrapper is implemented that exposes RSE file system, its implementation is not complete. Also, no UI integration has been provided.
+
 
+
Besides, there are also other drawbacks in the current implementation of RSE:
+
# The datastore feature is unlikely to be of interest to us, as it requires a specific datastore service to be started on the remote machine.
+
# The support for remote file and remote shell based on SSH is not currently available.
+
# The current code in the CVS combines the core model and UI implementation together in org.eclipse.rse.ui, and thus is difficult to read and understand.
+
 
+
Another possibility to improve RSE is to separate its dependency on the IDE into another plug-in. This will be helpful to use it within RCP programs.
+
 
+
==== Future ====
+
I'm expecting that the file subsystem and search subsystem of RSE should be faded out. The shell subsystem of RSE for remote execution is something also useful for remote file system support in the Eclipse core. So, I'll expect that part be adapted and finally become part of the Eclipse platform. And, only the UI part remains in RSE.
+
RSE is now part of the Target Management subproject of DSDP. Because of the similar challenge for both the PTP and the DSDP, I'm expecting that RSE will be adapted and extended to better support the need of target management for both embedded devices and parallel machines. We will have to collaborate with the developers from DSDP to speed up this process.
+
 
+
== Roadmap to PTP Remote Service Support ==
+
 
+
Based on the current status of Alternative File System and RSE, the following development tasks have been identified:
+
 
+
# Extend RSE to support remote execution via SSH [on hold while the status of similar projects is determined]
+
# Provide a remote file system implementation for core.filesystem. Initial implementation will use the 9P2000 protocol.
+
# Extend Eclipse platform to support project export to remote file system.
+
# Extend Eclipse platform to support project update from remote file system.
+
# Make sure the runtime and debug proxy implementations are appropriate for remote invocation.
+
# Modify CDT and Photran project build to support remote build [to be detailed]
+
# Modify PTP launch to support remote launch [to be detailed]
+
# Modify PTP debug to support remote debug [to be detailed]
+
 
+
== Summary ==
+
 
+
RSE can be a potential platform to enable remote service support for PTP. However, huge efforts are still demanded.
+
Collaborative development is also necessary to get the CDT and Photran prepared for remote building and launching.
+
 
+
== Open Issues ==
+
 
+
Resource management is being designed and developed in PTP, how does remote services support fits into the design remains to be discssed.
+

Revision as of 11:48, 22 April 2006

Overview

Since Eclipse 3.0 the Framework Adaptor API has been available in the Equinox OSGi Framework. A framework adaptor implementation is called upon by the Equinox OSGi Framework to perform a number of tasks. A framework adaptor may be used to add functionality to the framework.

A single framework adaptor is specified when the framework is launched. By default in Eclipse 3.0 this is set to the EclipseAdaptor. In order to add new functionality in an adaptor in Eclipse 3.0 and 3.1 it is required that the adaptor implementation either re-implement the complete framework adaptor API or extend one of the existing framework adaptor implementations. This makes it impossible for two parties to add new functionality to the framework in separate adaptors at the same time because the Equinox OSGi Framework can only be configured to use one adaptor.

In Eclipse 3.2 a new hookable adaptor has been included that is used by default as the framework adaptor. The framework adaptor API has remained unchanged for the most part in Eclipse 3.2. What has changed is the actual implementation of the adaptor API. A new implementation of the adaptor API is now included which provides hooks that others can implement to provide functionality to the adaptor implementation.

Hookable Adaptor

The hookable adaptor is implemented in the package org.eclipse.osgi.baseadaptor. This adaptor implementation provides all of the default behavior required of a FrameworkAdaptor to provide an OSGi R4 compliant Framework. It also provides many hooks that allow others to insert additional functionality into the framework through what are called framework extension bundles. See the OSGi Core Specification chapter 3.15 "Extension Bundles" for more information.

Framework extension bundles are fragments of the system bundle (org.eclipse.osgi). As a fragment they can provide extra classes which the framework can use. A framework extension bundle can define a set of hook implementations that are configured with the hookable adaptor (using a hookconfigurators.properties file).

The Base Adaptor

The class org.eclipse.osgi.baseadaptor.BaseAdaptor implements the interface org.eclipse.osgi.framework.adaptor.FrameworkAdaptor. This class is used by default as the adaptor of the framework. You should avoid extending this class, instead you should use hook implementations to add functionality to the BaseAdaptor.

In some cases it may be impossible to do what you want with the current set of adaptor hooks. In this case you may be forced to extend the BaseAdaptor class to provide your own adaptor implementation. If you find yourself in this situation then you should open a bug against Framework Equinox requesting a new hook method or interface.

The Hook Registry

The hook registry is implemented in the class org.eclipse.osgi.baseadaptor.HookRegistry. The hook registry is used to store all the hooks configured in the framework. When the hook registry is initialized it will discover all the hook configurators installed in the framework and will call on them to add hooks to the registry. The BaseAdaptor uses the hook registry at runtime to lookup and use the different hooks configured with the registry.

Hook Configurators

Hook configurators must implement the org.eclipse.osgi.baseadaptor.HookConfigurator interface. Hook configurators can add one or more hook implementations to the hook registry using the various add methods on the registry.

Discovering Hook Configurators

In order for a hook configurator to be discovered by the hook registry its implementation must be accessable by the framework's classloader. This implies that hook configurators must be built into the framework itself (org.eclipse.osgi) or be supplied by a framework extension bundle. Again a framework extension bundle is really just a fragment to Framework (i.e org.eclipse.osgi or the System Bundle).

A hook configurator also must be declared in one of the following ways to indicate to the hook registry which classes should be loaded as hook configurators.

hookconfigurators.properties files

A hookconfigurators.properties file can be used to declare a list of hook configator classes. The key hook.configurators is used in a hook configurators properties file to specify a comma separated list of fully qualified hook configurator classes. For example, the Equinox Framework (org.eclipse.osgi) is shipped with a default set of hook configurators specified a hookconfigurators.properties file included in the org.eclipse.osgi.jar:

hook.configurators= 
   org.eclipse.osgi.internal.baseadaptor.BaseHookConfigurator,
   org.eclipse.osgi.internal.baseadaptor.DevClassLoadingHook,
   org.eclipse.core.runtime.internal.adaptor.EclipseStorageHook,
   org.eclipse.core.runtime.internal.adaptor.EclipseLogHook,
   org.eclipse.core.runtime.internal.adaptor.EclipseErrorHandler,
   org.eclipse.core.runtime.internal.adaptor.EclipseAdaptorHook,
   org.eclipse.core.runtime.internal.adaptor.EclipseClassLoadingHook,
   org.eclipse.core.runtime.internal.adaptor.EclipseLazyStarter,
   org.eclipse.core.runtime.internal.stats.StatsManager,
   org.eclipse.osgi.internal.verifier.SignedBundleHook

Quite a few hook configurators are automatically enabled by default withing the Equinox Framework. The only hook configurator required for a fully functional OSGi R4 Framework is the org.eclipse.osgi.internal.baseadaptor.BaseHookConfigurator. All other configurators declared above add extra functionality needed by eclipse and can be disabled.

Extension bundles may provide their own hookconfigurators.properties file to specify additional hook configurators. The hook registry will discover all hookconfigurator.properties files on its classpath and will merge all declared configurator classes into one list.

osgi.hook.configurators property

The osgi.hook.configurators configuration property is used to specify the list of hook configurators. If this property is set then the list of configurators specified will be the only configurators used. If this property is set then the hookconfigurators.properties files will not be processed for additional configurators. This property can be used in a config.ini to lock down the set of configurators to a specific set.

osgi.hook.configurators.include property

The osgi.hook.configurators.include configuration property is used to add additional hook configurators. This is helpful for configuring optional hook configurators. Hooks that should be enabled by default should be included in a hookconfigurators.properties file. This property is ignored if the osgi.hook.configurators is set.

osgi.hook.configurators.exclude property

The osgi.hook.configurators.include configuration property is used to exclude any hook configurators. This is helpful for disabling hook configurators that are specified in hook configurator properties files. This property is ignored if the osgi.hook.configurators is set.

Hook interfaces

Adaptor Hook

Bundle File Factory Hook

Bundle File Wrapper Factory Hook

Bundle Watcher Hook

Class Loading Hook

Class Loading Stats Hook

Storage Hook

Bundle Files

Class Loaders

Examples

Back to the top