Technology Compatibility Kit Reference Guide for Jakarta Core Profile

This guide is for implementors of the Jakarta Core Profile 10.0 technology to assist in running the test suite that verifies the compatibility of their implementation.

Before reading this guide, you should familiarize yourself with the Jakarta EE programming model, specifically the Jakarta Restful Webservices 3.1 and the Jakarta Contexts and Dependency Injection 4.0 specifications. A good resource for the Jakarta EE programming model is the Jakarta EE web site.

The Core Profile TCK is based on the following Jakarta technologies:

Before running the tests in the Core Profile TCK, optionally read and become familiar with the Arquillian testing platform. A good starting point could be a series of Arquillian Guides.

If you are running the Core Profile TCK for the first time, read Introduction (Core Profile TCK) completely for the necessary background information about the TCK. Once you have reviewed that material, perform the steps outlined in the remaining chapters.

A TCK, or Technology Compatibility Kit, is one of the three required pieces for any specification (the other two being the specification document and a compatible implementation). The TCK is a set of tools and tests to verify that an implementation of the technology conforms to the specification. The tests are the primary component, but the tools serve an equally critical role of providing a framework and/or set of SPIs for executing the tests.

The tests in the TCK are derived from assertions in the written specification document. The assertions are itemized in an XML document, where they each get assigned a unique identifier, and materialize as a suite of automated tests that collectively validate whether an implementation complies with the aforementioned assertions, and in turn the specification. For a particular implementation to be certified, all the required tests must pass (i.e., the provided test suite must be run unmodified).

A TCK is entirely implementation agnostic. Ideally, it should validate assertions by consulting the specification’s public API. However, when the information returned by the public API is not low-level enough to validate the assertion, the implementation must be consulted directly. In this case, the TCK provides an independent API as part of a porting package that enables this transparency. The porting package must be implemented for each Core Profile implementation.

The goal of any specification is to eliminate portability problems so long as the program which uses the implementation also conforms to the rules laid out in the specification.

Executing the TCK is a form of compatibility testing. It’s important to understand that compatibility testing is distinctly different from product testing. The TCK is not concerned with robustness, performance or ease of use, and therefore cannot vouch for how well an implementation meets these criteria. What a TCK can do is to ensure the exactness of an implementation as it relates to the specification.

Compatibility testing of any feature relies on both a complete specification and a complete compatible implementation. The compatible implementation demonstrates how each test can be passed and provides additional context to the implementor during development for the corresponding assertion.

The compatibility requirements for Jakarta Contexts and Dependency Injection Version 3.0 consist of meeting the requirements set forth by the rules and associated definitions contained in this section.

The Jakarta Core Profile TCK is designed as a portable, configurable and automated test suite for verifying the compatibility of an implementation of the Jakarta CDI specification. The test suite is built atop Junit5 framework and Arquillian platform.

Each test class in the suite acts as a deployable unit. The deployable units, or artifacts are packaged as a WAR.

Any test case (e.g., test class, @Test method), test case configuration (e.g., beans.xml), test beans, annotations and other resources may be challenged by an appeal.

What is generally not challengeable are the assertions made by the specification. The specification document is controlled by a separate process and challenges to it should be handled by sending an e-mail to Platform Dev List

To submit a challenge, a new issue should be created in the Jakarta Platform Project using the label challenge. Any communication regarding the issue should be pursed in the comments of the filed issue for accurate record.

The challenges will be addressed in a timely fashion by the platform dev team.

The worflow for TCK challenges is outlined in Jakarta EE TCK Process 1.2.

Periodically, an updated TCK will be released, containing tests altered due to challenges - no new tests will be added. Implementations are required to pass the updated TCK. This release stream is named 10.0.x, where x will be incremented.

You can obtain a release of the Jakarta Core Profile TCK project from the Jakarta EE download site. The release stream for Jakarta Profile TCK is named 10.0.x. The Jakarta Core Profile TCK is distributed as a ZIP file, which contains the TCK artifacts (the test suite binary and source, test suite configuration file) in the artifacts directory, and documentation in doc directory.

You can also download the current source code from GitHub repository.

Executing the TCK requires a Jakarta EE 10 or newer runtime environment (i.e., application server), to which the test artifacts are deployed and the individual tests are invoked. The TCK does not depend on any particular Jakarta EE implementation.

Naturally, to execute Java programs, you must have a Java SE runtime environment. The TCK requires Java SE 11 or newer, which you can obtain from the Java Software website.

The TCK requires the following software to be installed:

You should refer to EE 10 implementation instructions for how to install the runtime environment.

Unzipping the Jakarta Core Profile TCK archive will create a core-profile-tck-x.y.z root folder which contains the TCK contents. To complete the installation, cd into the artifacts directory and install the standalone TCKs and Core Profile TCK artifacts using the maven pom.xml file. From that directory run:

mvn install

to populate the local maven repository with the necessary dependencies.

The various TCKs will have properties or configuration variables that need to be set in order to enable running the TCK agains a compatible implementation. The examples directory in the TCK distribution illustrates sample properties setup using Maven.

Refer to the configuration section of the CDI, RESTful, JSON-P and JSON-B standalone TCK user guides for configuration specific to each TCK. The examples directory in the TCK distribution illustrates sample configuration setup using Maven.

When the Jakarta Core Profile TCK test suite is executed during the Maven test phase of the TCK runner project, TestNG is invoked indirectly through the Maven Surefire plugin. Surefire is a test execution abstraction layer capable of executing a mix of tests written for JUnit, TestNG, and other supported test frameworks.

Why is this relevant? It means two things. First, it means that you are going to get a summary of the test run on the commandline.

If the Maven reporting plugin that complements Surefire is configured properly, Maven will also generate a generic HTML test result report. That report is written to the file test-report.html in the target/surefire-reports directory of the TCK runner project. It shows how many tests were run, how many failed and the success rate of the test run.

The one drawback of the Maven Surefire report plugin is that it buffers the test failures and puts them in the HTML report rather than outputting them to the commandline. If you are running the test suite to determine if there are any failures, it may be more useful to get this information in the foreground. You can prevent the failures from being redirected to the report using the following commandline switch:

The unit test reports will be placed into target/failsafe or target/surefire depending on which TCK is being run.

Each required component TCK describes how to run its signature tests. The JSON-P, JSON-B and Restful standalone TCKs include a test that sets up and runs the signature tests as part of the Junit 5 tests. Running those standalone TCKs generates the signature test results.

The CDI TCK includes a pom file to execute the signature tests. See the CDI TCK user guide for how to run those tests.