Skip to main content

Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

TPTP-Test-Automation-20071129

Attendees

  • Present:
    • Paul
    • Kiryl
    • Joanna
    • Jonathan
    • Joel
    • Alan

Deliverables

  • Testing Process:
    • Alan integrated the review comments and Paul posted the second draft of the Testing Process (part 1) document for review.
      • Allan will integrate review comments for the second draft of the Testing Process (part 1) document by Monday, December 3.
    • Alan will add more detailed content to the first draft of the Testing Process (part 2) by Wednesday, November 28.
    • Please review the latest drafts for both Testing Process documents for Monday, December 3.
  • Common Test Infrastructure:
    • Completed:
      • CVS check-out of all of the test plug-ins in the /test-results/<project>/* modules.
      • Configure and run the Agent Controller on the reference platform.
      • Paul created the root-level test suites for BVTs. The test infrastructure will execute the org.eclipse.hyades.tests/AllBVT<test type>Tests.testsuite test suites. We do not need to check-in the execution results to CVS due to disk space limitations and polluting of our test pass results. Each developer can rerun the automated tests to reproduce a failure.
      • Paul created the BVT deployment to the reference platform.
      • Automatically generating the Test Report based on the existing test execution results in CVS.
      • Paul modified the BIRT tabular test report to walk an execution result to generate a detailed test report from a root-level execution result.
    • Outstanding:
      • Check for Intel build to complete (every x hours) before launching the BVTs.
      • Automatically posting the Test Report to /tptp/test/reports/<release>.
      • Cleanup of the reference platform.
      • Documenting the test infrastructure for use in the second testing process document, which will be required by TPTP adopters or extenders.
      • Email notifications to component leads for failing test suites.
    • Issues:
  1. When generating reports, the report has no entries: Paul and Sheldon are investigating.
  2. Defect 208616: Fixed and verified by Kendric/Joel.
  3. Defect 200194: Targeted to i5.
  4. Defect 190453: Targeted to i6.
  5. Defect 210289: Targeted to i6.
  6. Defect 210292: Targeted to i6.
  7. Defect 210955: DTP has resolved the issue, available in the December 1st build. Paul has request a patch.
  8. Defect 211476 : Paul will fix in i4.
  • Metrics:
    • The monitoring project lead has replied to a request to estimate the current testing costs.
      • Joanna and Guru are gathering their estimates.

Action Items

  • Alan:
    • Integrate review comments for the second draft of the Testing Process (part 1) document by Monday, December 3.
    • Add more detailed content to the first draft of the Testing Process (part 2) by Monday, December 3.
  • Joel:
    • Outstanding issues for the PoC (integrate the ASF with the TPTP builds to check-out a 'Hello World' TPTP JUnit test suite from CVS, run the test suite on an Eclipse instance running on the target platform, and generate a BIRT report on the test execution using existing sample code).
  • Paul:
    • Determine labor cost (PW) for a) full test pass and b) smoke test pass from project leads.
    • Encourage Project Leads to automate existing manual test suites.

Upcoming Meeting

  • We will meet next on Monday, December 3 (9 AM ET).

Upcoming Discussion

Back to the top