Restlet editions

Introduction

Restlet is becoming an ubiquitous Web development framework. This has led us to provide various adaptations (ports) of the main Restlet code. So far we have been handling this process manually, but time has come to industrialize this process.

Requirements

  • Support all major Java-based deployment environments for Restlet applications
  • Support modular distributions
  • Automate the port of the main Restlet code to specific environments
  • Ensure that Restlet application portability is maximal
  • One base code managed by SVN
  • One code per edition
  • A set of distributions (zip, windows installer, maven, etc) per edition
  • Solve this issue related to the module version defined in the manifest (http://restlet.tigris.org/issues/show_bug.cgi?id=755)
  • Allow contributors to send "patches" (SVN patches?)

Analysis

  • The current usage of the org.restlet.ext" prefix for extension modules restricts the core packages
  • Depending on the target platform, the available core dependencies vary (ex: JAXP transformations aren't always available)
  • We should make sure that the Restlet core API provides the stable set across all target environments (GAE/J, Android, Java SE, etc.)
  • Express a set of transformation rules (modification, deletion, addition) from the base code to the code of an edition (several granularities: module, library, classe/file, method/attribute, body of a method, annotations of a method, java comments)
  • Specific dependencies (distinct libraries per edition, distinct versions per edition)
  • Customization of edition's distributions (e.g.: installation directory for the Windows installer)
  • Make the build.xml script more maintainable (e.g. when adding a new extension)
  • Decentralize the declaration of the modules.
  • Editions form a hierachy sharing common rules (ex: OSGi and Equinox, Java SE and Java EE)

References

Supported editions

  • Restlet for Android
  • Restlet for Google App Engine
  • Restlet for GWT
  • Restlet for Java SE
  • Restlet for Java EE

Design

  • Use consistent Java package names across all editions
  • Restlet core packages, without the "ext" suffix should be based on the smallest set of APIs possible (smaller than Java SE?)
  • Modules packaging (ex: Servlet extension) should be consistent across all editions to facilitate application porting
  • Modules, Libraries describe themselves with a single XML file
  • The build.xml is templated, in order to generate a single build.xml file per edition
  • A boot script is provided that aims at generating for each edition: the build.xml file (according to the XML descriptors file), the source code
  • Each build.xml file per edition is responsible to generate the edition's distributions (zip, exe, etc)

Overall file hierarchy, with XML files descriptors:

svn/trunk
  + build
  |   + project.xml
  + modules
  |   + org.restlet/module.xml
  |   + org.restlet.ext.freemarker/module.xml
  |   + ...
  + libraries
      + org.mortbay.jetty_6.1/library.xml

Overall build schema:

buildEditions

Implementation

  • We should aim at maintaining a single code base for the Restlet modules
  • Annotations could be used to indicate specificities of each Restlet edition e.g.: @Includes(Edition.GAE), @Excludes(Edition.GWT)
  • Take care of specific dependencies per edition, wich could not apply to the same source code
  • JDK's APT tool can process annotations but can modify the Java source code
  • INRIA's Spoon tool is much more powerful and allows Java source code transformation/customization
  • fmpp, freemarker for text generation (build.xml, properties files, etc)
  • ant
  • ivy, maven
  • Note: in order to specialize by edition, use parameters such as "includes" and "excludes" instead of nodes which could lead to duplicate large blocks of data.

Build script(s)

For the users that retrieves the whole project wfrom the svn repository, the structure stay the same. The root directory contains exactly the same directories (build, libraries, modules, etc). The main differences are locted in the apparition of new modules and libraries descriptors (described below), and the build script.
this script is a kind of starter script. It generates

  • the whole build script and required properties files in directories "temp/<edition>"
  • the source code for each target edition in the subdirectories "temp/<edition>/modules".

After that, each edition's build script is called. Thus, it helps the developers to get the source code for a target edition, and import it inside their development environment. Of course each update of the source code for a specific edition must be applied in the unique source code following the text transformation rules explained below.

Description of the project.xml file

  • list of supported editions?
  • description of each edition?
  • list of supported distributions (zip, installer for Windows, Maven, etc) per edition
<editions>
   <edition id="android|gae|gwt|jee|jse">
      <label>
         <full></full>
         <medium></medium>
         <short></short>
      </label>
      <description></description>
      <javadocs>
         <link href="http://java.sun.com/j2se/1.5.0/docs/api/" />
         [...]
      </javadocs>
      <distributions>
         <distribution id="classic" />
         <distribution id="maven" />
      </distributions>
   </edition>
</editions>

Name

Description

Notes

values

edition

Describes a single edition

edition#id

Identifier of the current edition

attribute

one of the following: android, gae, gwt, jee, jse

label.full

Full label for this edition

text

sample value: Java Standard Edition

label.medium

Shorter label

text

sample value: Java SE

label.short

Short label

text

JSE

description

Description of the current edition

text

javadocs

List of standard "link" entries for the and javadocs task

distributions

List of supported distributions for this edition

distribution#id

Identifier of a distribution

attribute

one of the following: classic, maven

Description of the module.xml file

Necessary data:

  • name (maven), bundle name (manifest) (nearly the same: "Restlet extension: Jetty"/"Restlet extension: Jetty 6")
  • description
  • version, bundle version (manifest)
  • dependencies (per edition)
  • exported packages (manifest) (per edition)
  • imported package (manifest) (per edition)
  • bundle activator (manifest, not supported in Android) (per edition)
  • bundle required environment (manifest) (per edition?)
  • others?
<module id="" type="core|standard|integration|connector" package="org.restlet[...]" includes="" excludes="">
   <label></label>
   <description></description>
   <distributions>
      <distribution id="classic" />
      <distribution id="maven" />
   </distributions>

   <dependencies>
      <dependency includes excludes="" type="library|module" id="" />
   </dependencies>
   <osgi>
      <export includes="" excludes=""></export>
      <import includes="" excludes=""></import>
      <activator  includes="" excludes="" />
   </osgi>
   <source edition="android|gae|gwt|jee|jse">
      <files-mappers>
<![CDATA[
]]>
      </files-mappers>
      <files-sets>
<![CDATA[
]]>
      </files-sets>
   </source>
   <stage edition="gwt" includesource="true">
      <files-filters>
<![CDATA[
]]>
      </files-filters>
      <files-sets>
<![CDATA[
]]>
      </files-sets>
      <files-mappers>
<![CDATA[
]]>
      </files-mappers>
   </stage>
</module>

Name

Description

Notes

values

module

Describes a single module

module#id

Identifier of the current module

attribute

sample values: core, freemarker, etc

module#type

Distinguish a core module from an extension, and specify also the kind of extension.

attribute

one of the following: core, standard, connector, integration

module#package

Name of the corresponding package

attribute

sample value: org.restlet.ext.freemarker

includes

List of editions this modules is included in. It uses ',' as separator.

attribute

can be '*' for all editions

excludes

List of editions this modules is not included in. It uses ',' as separator.

attribute

can be '*' for all editions (won't be part of any edition

label

Full label for this module

text

sample value: Integration with Grizzly NIO framework.

distributions

List of supported distributions for this module

distribution.id

Identifier of a distribution

attribute

one of the following: classic, maven

dependencies

List of libraries and modules, this module depends on.

dependency#id

Identifier of the library or module

attribute

sample values: core, freemarker

dependency#type

Type of the dependency

attribute

one of the following: library, module

dependency#includes

List of editions this dependency is valid.

attribute

dependency#excludes

List of editions this dependency is invalid

attribute

osgi

Not used for the moment

source

A collection of ant file sets, file mappers and file filter used during the generation of the source code for one specific edition from the unique source repository

source#edition

Identifier of the edition

attribute

source.file-mappers

Container of ant file mappers used to rename source files

text

sample value of one item:
<mapper classname="mapper.ReplaceStringMapper" classpathref="forge-path" from=".java.gwt" to=".java" />

source.file-sets

Container of ant file file-set rules used generally to exclude source file for the target edition

text

sample value of one item:
<exclude name="src/org/restlet/engine/internal/Activator.java" />

stage

A collection of ant file sets, file mappers and file filter used for completing the final jar of this module

stage#edition

Identifier of the edition

attribute

stage#includesource

Indicates if the source code must be included inside the final jar.

attribute

one of the following: true, false

stage.file-mappers

Container of ant file mappers used to rename files

stage.file-sets

Container of ant file file-set rules

stage.file-filters

Container of ant file file-filter rules used for text filtering during the copy of the source files

sample value of one item:

<tokenfilter>
    <replacestring from="com.google.gwt.emul." to=""/>
</tokenfilter>

Description of the library.xml file

Necessary data:

  • package
  • version
  • release number
  • id (in main cases "package"_"version")
  • home uri
  • download uri
  • maven goup id
  • maven artifact id
  • maven version
<library id="">
   <package id="" name="">
      <maven>
         <groupId /> optional (default to "org.restlet")
         <artifactId /> optional (default to package, always concatenated with "org.restlet.lib")
         <version /> optional (default to version.release or version)
      </maven>
   </package>
   <version></version>
   <release></release>
   <distributions>
      <distribution id="classic|maven" includes="" excludes="" />
   </distributions>
   <homeUri></homeUri>
   <downloadUri></downloadUri>
</library>

Name

Description

Notes

values

library

Describes a single library

library#id

Identifier of the current library

attribute

sample values: db4o, freemarker, etc

package

Description one jar inside this library module

package#id

Identifier of this package

attribute

sample value: db4o-nativequery

package#name

Optional name of the jar file

attribute

sample value: com.db4o.nativequery

package.maven

Specifies a set of properties values for the generation of the Maven artifact

package.maven.groupId

Only if distinct from the default value ("org.restlet")

sample value: com.db4o

package.maven.artifactId

Only if distinct from the default value ("org.restlet.lib.<packagename>")

package.maven.version

Only if distinct from the default value ("<version>.<release>" or "<version>")

version

Minor version of this library

text

sample value: 7.7

release

Release number of this library

text

sample value: 67

distributions

List of supported distributions for this module

distribution

Describes a distribution

distribution#id

Identifier of a distribution

attribute

one of the following: classic, maven

homeUri

Uri of the home page of the library's project

text

downloadUri

Uri of the download page

text

Sources of diagrams: buildEditions (image/svg+xml, 150.2 kB, info)

Distributions

Current supported editions:

  • jse
  • jee
  • gwt
  • gae
  • android

Maven group id: org.restlet.<edition>

Files distributions

  • zip: http://www.restlet.org/downloads/<minor version (2.0)>/restlet-<edition>-<compact version (2.0m4)>.zip
  • Installer for Windows: http://www.restlet.org/downloads/<minor version (2.0)>/restlet-<edition>-<compact version (2.0m4)>.exe

Code annotations

The aim of this set of code annotations is to provide text transformation rules in order to generate a new version of a class for a specific edition. Basically the following transformations are supported:

  • remove a specific bloc of code, or single instruction, or method, or member  for the current edition
  • add a specific bloc of code, or single instruction, or method, or member  for the current edition
  • specify that the whole class must not be part of the current edition

The implementation is not incarnated by a set of Java annotations which are not flexible enough. The idea was taken from the concept of conditional compilation used for other languages such as C. It consists only of simple Java comments:

  1. // [ifdef <list of editions>] <optional keywords>
  2. // [ifndef <list of editions>] <optional keywords>
  3. // [enddef]

The avantage of such comments is that they can be put everywhere in the source code, and still remain even if the source code is automatically formatted by the development environment. Note that unfortunately, Eclipse does not allow such comments in the section where are declared the "import"  classes.

Explanations

The "ifdef" and "ifndef" annotations mark the beginning of a block. Such blocks target a list of editions specified with a simple coma-separated list of editions id. According to the optional keywords, they may  require the end block marker: "enddef".

As sample code says a lot more than a flow of explanations, here is the way to indicates that the a block code apply only for the jee and jse editions:

// [ifdef jee,jse]
instructions1;
instructions2;
// [enddef]

On the contrary, the following code will be removed for the gwt edition, but will be applied for any other:

// [ifndef gwt]
instructions1;
instructions2;
// [enddef]

Now, imagine that the gwt edition requires a set of instructions that must not be be included in the unique source code. This code must be commented in the unique source code, but uncommented for the gwt edition. Let's introduce the "uncomment" keyword:

// [ifdef gwt] uncomment
// instructions1;
// instructions2;
// [enddef]

Note that the "uncomment" keyword applies for both "ifdef" and "ifndef" block markers and handle only single comments (starting with "//").

Finally, let's introduce the other available keywords. They aim at lighten the unique source code and get ride of the "enddef" blowk marker. At this time, three keywords are at disposal: member, method and instruction.

Note that these keywords can be used in conjunction with the mandatory ifdef/ifndef block markers and the optional "uncomment" keyword.

Imagine that you want to add an attribute (including its comment) for a specific edition, let's use the "member" one:

// [ifdef gwt] member uncomment
/** This is a specific attribute for the gwt edition. */
// private String attribute;

Now, you want to remove a whole method, only for the gae and gwt editions: 

    // [ifndef gae,gwt] method
    /**
     * Handles a call.
     * 
     * @param request
     *            The request to handle.
     * @return The returned response.
     */
    public final Response handle(Request request) {
        final Response response = new Response(request);
        handle(request, response);
        return response;
    }

Finally, let's introduce the "instruction" keyword. It helps handle a single instruction inside a method. The following shows how to remove a single instruction for any editions except gwt, and replace it by another one for gwt. 

        // [ifndef gwt] instruction
        this.context = context;
        // [ifdef gwt] instruction uncomment
        // this.context = (context != null) ? context : new Context();

Limitations

When the set of text transformation rules becomes too heavy, the source code for one class become less maintainable. In this case, we preferefer ot generate the whole source code for the target edition in another file.
This file is located in the same directory than the original code, it shares also the name except that it is suffixed by the edition identifier (ex: ByteUtils.java.gwt).
Then, the module descriptor must be completed in twh ways:

  • add a rule that explicitely rename edition specific files
  • add an exclusion rule for the original source file
   <source edition="gwt">
      <files-mappers>
<![CDATA[
      <mapper classname="mapper.ReplaceStringMapper" classpathref="forge-path" from=".java.gwt" to=".java" />
]]>
      </files-mappers>
      <files-sets>
<![CDATA[
         <exclude name="src/org/restlet/engine/io/ByteUtils.java" />
]]>
      </files-sets>
   </source>
Comments (0)