|
|
| Line 1: |
Line 1: |
| − | Session tests in Eclipse are tests that require that a new Eclipse session
| + | On this page, please add any little tidbits that help with the versioning, site management, etc., that are not part of other documents. |
| − | be launched for every test case run. Thus, they have additional requirements
| + | |
| − | regarding controling the environment test cases are run (VM, set of plug-ins
| + | |
| − | available, configuration, instance location, command line parameters) and
| + | |
| − | how results are communicated back to the test runner. This FAQ-like document describes
| + | |
| − | how the Platform/Core implemented support for running session tests in the
| + | |
| − | hopes of encouraging adoption of this framework not only by all the Platform/Core
| + | |
| − | team members but also by members of other Eclipse teams interested in developing
| + | |
| − | their own session tests.
| + | |
| | | | |
| − | <ol>
| |
| − | <li>
| |
| − | <p><strong>What is a session test?</strong></p>
| |
| − | <p>It is a test case that needs to be run isolated in a separate VM instance.
| |
| − | Technical details: The VM running a set of test cases (the <em>controller
| |
| − | VM</em>) will spawn a second instance (the <em>target VM</em>) for each
| |
| − | session test case to be run. The test results are collected (by using a
| |
| − | socket) and presented to the test runner as if the test had run in the same
| |
| − | VM.</p>
| |
| | | | |
| − | </li>
| + | === Automatically versioning features appropriately === |
| − | <li>
| + | |
| − | <p><strong>Where can I get the framework?</strong></p>
| + | |
| − | <p> Check out the <code>org.eclipse.core.tests.harness</code> project from
| + | |
| − | dev.eclipse.org. The infrastructure for session tests is in the <code>org.eclipse.core.tests.session</code>
| + | |
| − | package. It does not hurt mentioning that besides requiring this plug-in,
| + | |
| − | you should have <code>org.junit</code> as a pre-requisite as well.</p>
| + | |
| | | | |
| − | </li>
| + | I haven't tried it yet, but in |
| − | <li>
| + | [[https://bugs.eclipse.org/bugs/show_bug.cgi?id=125801#c9 | bug 125801] |
| − | <p><strong>What it is the easiest way to transform my regular plug-in tests
| + | Andrew Niefer mentions that |
| − | into session tests?</strong></p>
| + | |
| − | <p> Change the test suite creation to use <code>org.eclipse.core.tests.session.SessionTestSuite</code>
| + | |
| − | instead of <code>junit.framework.TestSuite</code>. Like this:</p>
| + | |
| − | <p>Before:</p>
| + | |
| | | | |
| − | <pre>
| + | <p><cite> |
| − | public class MyTestCase extends TestCase {
| + | When using ".qualifier", the resulting feature version can take the form of |
| − | ...
| + | 1.2.3.qualifier_suffix, where qualifier will come from the cvs tag, or date the |
| − | public void test1() {
| + | same as for plugins. suffix will be generated based on the qualifiers of the |
| − | ...
| + | contained plugins. |
| − | }
| + | <br /><br /> |
| − | ...
| + | Whether or not to append the _suffix to the version is controlled by the |
| − | public static Test suite() {
| + | property "generateFeatureVersionSuffix" in the builder's build.properties. The |
| − | return new TestSuite(MyTestCase.class);
| + | default value is false. |
| − | }
| + | </cite> |
| − | }
| + | </p> |
| − | </pre>
| + | |
| − | | + | |
| − | <pre>
| + | |
| − | public class MyTestCase extends TestCase {
| + | |
| − | ...
| + | |
| − | public void test1() {
| + | |
| − | ...
| + | |
| − | }
| + | |
| − | ...
| + | |
| − | public static Test suite() {
| + | |
| − | return new SessionTestSuite("org.myplugin.tests", MyTestCase.class);
| + | |
| − | }
| + | |
| − | }
| + | |
| − | </pre>
| + | |
| − | <p>The plug-in id is necessary so the plug-in which the class containing the
| + | |
| − | test case should be loaded from can be determined. Note hat it is possible
| + | |
| − | to build an empty suite and then manually add test cases and/or test suites.
| + | |
| − | When running your tests again, every test case in the suite (included those
| + | |
| − | in nested test suites) will be run in a separate Eclipse session. Note also
| + | |
| − | that if a session test suite contains another session test suite (or any
| + | |
| − | other special types of test suite), the outer session test suite will rule.</p>
| + | |
| − | </li>
| + | |
| − | <li><strong>Why do I get a <em>"...SetupException: No setup descriptions
| + | |
| − | found. Ensure you are specifying the path for an existing setup file...</em>"?</strong>
| + | |
| − | <p> You are running your session test suite as a regular <em>JUnit test</em>
| + | |
| − | instead of as a <em>Plug-in JUnit test</em>. A default setup (in other words,
| + | |
| − | command line) for launching new Eclipse sessions when running session tests
| + | |
| − | can be computed when running the test suite as a plug-in test, so no further
| + | |
| − | configuration is necessary. It basically contains the same parameters as
| + | |
| − | the ones used to launch the controller instance, with a different instance
| + | |
| − | location (the -data parameter). When launching the controller instance as
| + | |
| − | a regular JUnit test suite, the setup for running session tests has to be
| + | |
| − | configured manually. The framework looks for a <code>default-setup.xml</code>
| + | |
| − | file in the current directory, but a different file name and location can
| + | |
| − | be specified by using the <code>setup.file</code> system property. If not
| + | |
| − | running in Eclipse, and no setup file can be found, the execution fails
| + | |
| − | with this exception. See also next FAQ.</p>
| + | |
| | | | |
| − | </li>
| + | I personally feel that <code>generateFeatureVersionSuffix</code> should be the default, but since it isn't, thought I'd document here. |
| − | <li>
| + | |
| − | <p><strong>Should I run my tests using a JUnit or PDE Junit launch configuration?</strong></p>
| + | |
| − | <p>Running with a PDE Junit launch configuration will allow you to test code
| + | |
| − | in your workspace. Also, launching with PDE JUnit frees you from having
| + | |
| − | to provide configuration parameters, since reasonable defaults will be obtained
| + | |
| − | from the controlling Eclipse. If you are still interested in running your
| + | |
| − | tests as a JUnit launch configuration, see the next FAQ.</p>
| + | |
| − | </li>
| + | |
| − | <li>
| + | |
| − | <p><strong>How do I run session tests from a JUnit (not PDE JUnit) launch
| + | |
| − | configuration?</strong> </p>
| + | |
| − | <p>Ensure you provide a valid set of setup files. Setup files describe option
| + | |
| − | sets that can be used when launching Eclipse. See question about options
| + | |
| − | supported by the framework.</p>
| + | |
| − | </li>
| + | |
| − | <li>
| + | |
| − | <p><strong>How do I run UI-oriented tests?</strong> </p>
| + | |
| − | <p>Make sure you also set the application when launching the target instance
| + | |
| − | to be <code>SessionTestSuite.UI_TEST_APPLICATION</code>:</p>
| + | |
| − | <pre>SessionTestSuite suite = new SessionTestSuite("org.myplugin.tests", MyTestCase.class);
| + | |
| − | suite.setApplicationId(SessionTestSuite.UI_TEST_APPLICATION);</pre>
| + | |
| − | </li>
| + | |
| | | | |
| − | <li><strong>How can I see the command-line used for the Eclipse instance where my session test runs?</strong>
| + | This is important since a feature's qualifier needs to not just change accourding to its own changes, but also needs to reflect the "most changed" plugin it contains. |
| − | <p>Use the <code>setup.debug</code> boolean system property (check the question about supported options).</p>
| + | |
| − | </li>
| + | |
| − | <li><strong>How do I debug a failing test case?</strong>
| + | |
| − | <p><strong>The easy way:</strong> run the failing test case in isolation as
| + | |
| − | a regular JUnit plug-in test. Ensure the command-line matches the one built
| + | |
| − | by the session test framework when running it as a session test.</p>
| + | |
| | | | |
| − | <p><strong>The hard way:</strong> you can configure the session test to launch the target VM in debug mode using the following
| + | === Do not use underscore in version qualifiers === |
| − | VM argument:</p>
| + | |
| − | <tt>-Dsetup.override.vmArgs=Xdebug;Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y</tt>
| + | |
| − | <p>If you are using the setup.files mechanism (see below), then use these VM arguments intead:</p>
| + | |
| − | <tt>-Dsetup.files=default-setup.xml -Dsetup.options=remote_debug</tt>
| + | |
| − | <p>When the target VM is launched in debug mode, you must connect the debugger to each test individually:
| + | |
| − | <ol>
| + | |
| − | <li>From the "Run" menu, select "Debug..."</li>
| + | |
| − | <li>Select the "Remote Java Application" category on the left hand side</li>
| + | |
| − | <li>Click the "New" button to create a new launch configuration</li>
| + | |
| − | <li>Leave all settings as default and click "Debug"</li>
| + | |
| − | </ol></p>
| + | |
| − | </li>
| + | |
| − | <li><strong>How do I run performance session tests?</strong>
| + | |
| − | <p>Use the <code>PerformanceSessionTestSuite</code> instead.</p>
| + | |
| − | </li>
| + | |
| − | <li><strong>How do I run session tests that have private instance locations?</strong>
| + | |
| − | <p>Use the <code>WorkspaceSessionTestSuite</code> instead. Every test suite
| + | |
| − | has its own instance location (workspace), all test cases are run on separate
| + | |
| − | sessions on the same workspace. After the last test case is run, that location
| + | |
| − | is deleted.</p>
| + | |
| − | </li>
| + | |
| − | <li><strong>I just want to try out some examples. Any pointers?</strong>
| + | |
| − | <p>The <code>org.eclipse.core.tests.session.samples</code> package has some
| + | |
| − | examples. If you load the org.eclipse.core.tests.runtime and org.eclipse.core.tests.resources
| + | |
| − | projects, you will find real-world examples of how to use this framework.
| + | |
| − | Just look for the subclasses of <code>SessionTestSuite</code>.</p>
| + | |
| | | | |
| − | </li>
| + | I think this will all work better post M5, but certainly on 3.1.x streams, and underscore is converted to a hyphen in some places, and will at first seeem to work, but then there's problems associating source with its code, etc., so ... use with care, if not outright avoid. See [[https://bugs.eclipse.org/bugs/show_bug.cgi?id=89428 | bug 89428]] for some dicussions. |
| − | <li><strong>What are all supported options and how to use them?</strong>
| + | |
| − | <p>The options for the session test framework are specified in the form of
| + | |
| − | system properties. <em>You only have to use these options if running with
| + | |
| − | the default settings (the currently target JRE and Eclipse and default command
| + | |
| − | line options) are not appropriate.</em></p>
| + | |
| − | <ul>
| + | |
| − | <li><strong>setup.options</strong> - only valid when setup files are available.
| + | |
| − | Setup files describe option sets, which are collections of command-line
| + | |
| − | options that can be used while running Eclipse. Examples:
| + | |
| − | <p><code>-Dsetup.options=sun_1.5,eclipse_3.1_M1,big_memory</code></p>
| + | |
| − | <p>would run Eclipse 3.1 M1 using Sun JRE 1.5 with whatever amount of
| + | |
| − | memory the <code>big_memory</code> option set specifies. While:</p>
| + | |
| | | | |
| − | <p><code>-Dsetup.options=ibm_1.4,eclipse_3.0,no_jit</code></p>
| + | === Use the platform releng tool === |
| − | <p>would run Eclipse 3.0 on IBM JRE 1.4 with Just-in-Time compilation
| + | |
| − | disabled</p>
| + | |
| − | <p>The meaning of every option set must be specified in setup files (see
| + | |
| − | below).</p>
| + | |
| − | </li>
| + | |
| − | <li><strong>setup.files</strong> - setup files provide option set definitions
| + | |
| − | for running the session tests. Setup files can be chained, so a more general
| + | |
| − | setup file (used by the whole team) can be customized with a user specific
| + | |
| − | setup file. This is useful since option sets must specify file system
| + | |
| − | locations, and every user has its own layout for JRE and Eclipse installs.
| + | |
| − | The expected use of this option is:
| + | |
| − | <p><code>-Dsetup.files=/tmp/default-setup.xml,~/user-setup.xml</code></p>
| + | |
| | | | |
| − | <p>Here, the <code>default-setup.xml</code> file is shared by the whole
| + | To update your map files automatically while releasing your plug-ins, the releng tool found as a download way at the bottom of the platform download page is quite helpful. The instructions are sparse, but it's fairly straightforward (right-click a project and select Team -> Release...). |
| − | team and defines all option sets that the team may use. The <code>user-setup.xml</code>
| + | |
| − | merely overrides location sensitive options to adapt them to the user
| + | |
| − | specific file system layout. Look [http://eclipse.org/eclipse/platform-core/documents/default-setup.xml here]
| + | |
| − | for an example of a default setup file and [http://http://eclipse.org/eclipse/platform-core/documents/user-setup.xml here] for an example of a user custom setup file. These examples are intended
| + | |
| − | to be self-explanatory and are actually useful as they are (the user-specifc
| + | |
| − | setup file has to be tweaked to support your specific file system layout.</p>
| + | |
| − | </li>
| + | |
| | | | |
| − | <li><strong>setup.override.eclipseArgs</strong> - allows the user to manually
| + | When using .qualifier at the end of your Bundle-Version manifest entries, your plug-ins will be versioned according to the tags entered in your map files automatically when using the release engineering tool with the PDE basebuilder. |
| − | override command-line options corresponding to Eclipse arguments. Example:
| + | |
| − | <p><code>-Dsetup.override.eclipseArgs=data=c:/workspace;debug;showlocation</code></p>
| + | |
| − | </li>
| + | |
| − | <li><strong>setup.override.vmArgs</strong> - allows the user to manually
| + | |
| − | override command-line options corresponding to VM arguments (other than
| + | |
| − | system properties). Example:
| + | |
| − | <p><code>-Dsetup.override.vmArgs=Xint;Xmx256m;hotspot;cp=startup.jar</code></p>
| + | |
| − | </li>
| + | |
| − | <li><strong>setup.override.systemProperties</strong> - allows the user to
| + | |
| − | manually override command-line options corresponding to system properties.
| + | |
| − | Example:
| + | |
| − | <p><code>-Dsetup.override.systemProperties=osgi.checkConfiguration;osgi.locking=none</code></p>
| + | |
| | | | |
| − | </li>
| + | === Guide to understanding versioning === |
| − | <li><strong>setup.debug</strong> - enables the session test framework debug
| + | |
| − | mode. In this mode, among other things, the command-line for every session
| + | |
| − | launched is dumped to the console, so it is useful to verify which option
| + | |
| − | sets are being selected and how they map to the command line.</li>
| + | |
| − | </ul>
| + | |
| − | <p></p>
| + | |
| − | </li>
| + | |
| − | <li><strong>What if I find problems using the session test framework or would
| + | |
| − | like to request/propose an enhancement?</strong>
| + | |
| − | <p>Please report them in Bugzilla against the Platform/Runtime component.</p>
| + | |
| | | | |
| − | </li>
| + | Just because this page cannot be linked enough, look here for detailed information on when and how to apply new version numbers to your plug-ins and features: [[http://www.eclipse.org/eclipse/platform-core/documents/plugin-versioning.html Plug-in Versioning]] |
| − | </ol>
| + | |
On this page, please add any little tidbits that help with the versioning, site management, etc., that are not part of other documents.
This is important since a feature's qualifier needs to not just change accourding to its own changes, but also needs to reflect the "most changed" plugin it contains.
I think this will all work better post M5, but certainly on 3.1.x streams, and underscore is converted to a hyphen in some places, and will at first seeem to work, but then there's problems associating source with its code, etc., so ... use with care, if not outright avoid. See [| bug 89428] for some dicussions.
To update your map files automatically while releasing your plug-ins, the releng tool found as a download way at the bottom of the platform download page is quite helpful. The instructions are sparse, but it's fairly straightforward (right-click a project and select Team -> Release...).
When using .qualifier at the end of your Bundle-Version manifest entries, your plug-ins will be versioned according to the tags entered in your map files automatically when using the release engineering tool with the PDE basebuilder.
Just because this page cannot be linked enough, look here for detailed information on when and how to apply new version numbers to your plug-ins and features: [Plug-in Versioning]
