Skip to main content

Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

TPTP-Test-Automation-20070924

Attendees

  • Present:
    • Paul
    • Alan
    • Joel
    • Jonathan
    • Joanna

Goals

  • Consolidation of our testing process, and infrastructure.
  • Consolidation of custom test frameworks (Test Dashboard, AC/Profiler test server, variants).
  • Reuse of Test Project framework (ASF, reporting, etc.) and tools (AGR, JUnit, etc.).
  • Automatic test execution and report generation with every build (BVT).

Benefits

  • Specialization of testing skills (e.g. builds).
  • Provide an infrastructure that makes it easy for committers to contribute automated test cases/suites.
  • Manual labor savings by using automated tests, which are automatically executed with every build.
  • Decrease time/cost to resolve defects since uncovered earlier in the code-test-build cycle.
  • Decrease testing overlap by a) localizing test executing and reporting to one group and b) using code coverage statistics.

Reference Platform

  • Target: IBM Java 5* and XP/x86.
  • Developers are expected to unit test on reference platform before check-in.
  • Nightly builds invoke automated test execution on reference platform.
  • Goal to use a second reference platform for better test coverage (target: Sun Java 5 and Linux/x86).
  • Move to Java 6 once a complete test pass is completed to expose initial defects.

Deliver dates

  • October 17:
    • Define the testing process, extending the existing TPTP Testing Strategy.
    • Define the common test infrastructure and its constraints.
    • Proof of concept (PoC) for integration of ASF and TPTP builds for PMC approval.
    • Profiler test suite automation.
    • Determine the machine for the target platform.
  • October 31:
    • Fully integrate PoC with TPTP builds.
    • Resolve infrastructure limitations.
    • Presentation to TPTP committers.
    • Measure cost savings.
  • November 14:
    • Each project automates their manual test suites and converge existing automated test suites over time to leverage the test infrastructure.
    • Measure cost savings.
    • Report cost savings to PMC.

Action Items

  • Alan: Review and revamp the existing TPTP Testing Strategy as the new test process.
  • Jonathan/Kendric/Paul: Port existing profiler automated tests to used the Test Project framework.
  • Joel: Locate a dedicate test machine for the target platform.
  • Joel: Integrate the ASF with the TPTP builds to check-out a 'Hello World' TPTP JUnit test suite from CVS, run the test suite on an Eclipse instance running on the target platform, and generate a BIRT report on the test execution using existing sample code, possibly reuse some function from the Test Dashboard..
  • Paul: Determine labor cost (PW) for a) full test pass and b) smoke test pass from project leads.
  • Paul: Encourage Project Leads to automate existing manual test suites.
  • Paul: Book another meeting in two weeks (October 15).

References

Back to the top