FlexDoc/Javadoc - Features

  1. How It Works
  2. Doclet API as virtual XML document
  3. Formatting
  4. Insertion of images
  5. Hypertext
  6. Output formats
  7. Doclet GUI
  8. Ready template applications
  9. Integrations

How It Works

Here is a UML component diagram that depicts the functional structure of FlexDoc/Javadoc:

The diagram was created with Eclipse Papyrus UML tool.

Main Components

On the diagram above you can see the package box titled “xyz.flexdoc.javadoc”. It encloses all main Java-hardcoded components that make FlexDoc/Javadoc:

Component Description
Doclet DSM Driver Serves as a bridge between Javadoc Doclet API and FlexDoc.XYZ core. The driver connects to Javadoc using Doclet API and provides data from it in a form understandable for FlexDoc.XYZ core, which is called Doclet DSM (DSM is for Data Source Model). To deliver it, the driver implements two interfaces:
  • “DSM” interface provides the Doclet DSM itself. It is used by the Template Processor to obtain data it operates.
  • “DSM Type” interface provides the datatype/structure information about this data source. It is used both by the Template Processor to know how to handle those data, and by the Template Designer to help during designing of templates.
FlexDoc Doclet Provides xyz.flexdoc.javadoc.Doclet class, which is the one to be specified with -doclet option on Javadoc command line. FlexDoc Doclet is basically a holder of the Template Processor. It parses the command line options received from the Javadoc, initiates the Doclet DSM Driver and starts the Template Processor. It also may invoke the Doclet GUI, which provides a user-friendly interface to specify most of parameters and settings used by the Template Processor.

See also: Documentation | FlexDoc Doclet

Template Processor It is launched by FlexDoc Doclet and receives from it: Template Processor executes the template and generates the output documentation.
Template Designer A GUI to create/modify templates. It also starts Doclet DSM Driver (however, without connection to Javadoc) and uses the datatype information provided by DSM Type interface to help designing templates. In the next section, we shall discuss what that information is.

Actual Doclets as Template Sets

Actual Java API documentation generators (that is doclets) are created as sets of special templates, executed by the Template Processor. The templates are effectively programs that implement the entire generation of the documentation output.

Templates are made of
  • Template components, which are functional programming elements that both generate single pieces of output and can be visualized graphically in a form resembling the output they generate. Template components are assigned with various properties that control their interpretation as well as specify the formatting of the output they produce. Some of such properties are static. Others are calculated dynamically by small scripts (with Java-like syntax) called FlexQuery-expressions, which provide access to lots of specialized functions, template parameters and properties of Generator Object Model.
  • Formatting styles, which are named bundles of formatting properties that as a whole can be assigned to a template component. They can be used to unify output formatting across all template set, as implicit parameters (by overriding in the calls of subtemplates) and to program the generation of named CSS rules (which allows styling the result documentation with custom CSS stylesheets).
  • Template parameters, which are used to pass to the template set various data and settings as well as for internal communication between subtemplates.
Note: The word «template» is not exactly suitable to describe what this everything actually is. FlexDoc/XML templates are rather a mixture of what is traditionally called «templates» and «scripts». For more details, please see: FlexDoc.XYZ | What are Templates?

There are basically four kinds of templates (or rather roles which they can play):

  1. Document template that generates the whole document (e.g. an HTML file).
  2. Fragment template that generates some parts/fragments of the documents.
  3. Procedure template, which generates nothing by itself, but may be used for some preparation (e.g. loading referenced XML files, creating element maps, etc.) or to call the document templates (particularly when the output documentation has no obvious root).
  4. Main template, which starts the generation of the entire documentation.
All those templates may call each other (and even in different roles).

Main templates are those specified to the generator (only one at a time). A template set may include several main templates to generate different kinds of documentation (e.g. single file, framed multi-file, etc.)

Template Designer

Although templates are stored as plain-text files (with an XML-like format), they are not supposed for editing manually. Rather, a special graphic Template Designer must be used, which visualizes templates in the form of components they are made of.

This screenshot shows the class.tpl template (from JavadocClassic template set) open in the Template Designer (click to see in full size).

The left pane represents the structural components called sections, which are responsible for most of data processing as well as the generation of integral features of the output (such as paragraphs, tables, lists etc.)

The right pane represents the components called controls, which generate particular content details (e.g. text, images, table cells etc.) Controls are visualized in a form resembling the output they generate.

Template Processor as Javadoc Doclet

Doclet is a replaceable Javadoc plugin used by Javadoc to generate any sort of output (typically some Java API documentation). The whole working of Javadoc can be described as follows:
  1. Parsing Java sources and collecting various information about them.
  2. Forming by that information some internal representation accessible via Doclet API.
  3. Calling a doclet and passing to it an instance of DocletEnvironment interface – the root object of the Doclet API.
By default, when no custom doclet is specified, Javadoc uses the Standard Doclet embedded in it.
FlexDoc/Javadoc provides the only doclet called FlexDoc Doclet, which is essentially a shell enclosing the Template Processor. Precisely, it does two things:
  1. Initializes the Doclet DSM Driver, which is a bridge between Doclet API and FlexDoc.XYZ core.
  2. Starts the Template Processor and passes to it the main template and the Doclet DSM, an XML DOM-like representation of the whole Doclet API provided by Doclet DSM Driver.

Template Processor is the most crucial and sophisticated part of the whole FlexDoc/Javadoc system. It consists of the template interpreter and the output generator, which work together as single unit:

Together the template interpreter + all output generators represent a single functional unit called simply generator. It is controlled by lots of command-line options.

Alternatively, most of these settings can be specified in a Doclet GUI interactively. Its main dialog is shown on the following screenshot:

For further details, please see: FlexDoc/Javadoc | Documentation | FlexDoc Doclet

Template Parameters

Since all the content and formatting of the generated documentation is programmed entirely in templates, what in ordinary doc-generators are command-line options now simply become template parameters.

The parameters are defined and accessed in templates to control dynamic properties of template components (or pass data to them). They may be of various data types including list parameters, which allow passing a vector of values associated with the same parameter name.

Because template parameters are easy to introduce and support in templates (most of time actually takes writing their descriptions), a large template application may have hundreds of parameters. This allows adjusting the generated documentation within a wide range of possibilities without changing the templates.

However, it also creates two problems:

  1. How to find and set a necessary parameter quickly?
  2. How to avoid settings lots of related parameters when just a few features need to be changed?
Those problems are solved as follows: On the left screenshot you can see how parameters are defined in a template. On the right is the Parameter Inspector created from those definitions (click on each screenshots to enlarge):

Doclet API as virtual XML document

Doclet DSM

When Javadoc is started, first it collects the information about the Java sources to be documented. Javadoc does this by parsing all Java classes (by calling the Java compiler from itself). Once the collecting of that information is finished, Javadoc calls a Doclet (by default, the Standard Doclet) and passes to it all Java source information organized in the form of a network of various objects provided via the Doclet API.

One may notice, however, that the structure of such a representation can be described also as a DOM (Document Object Model of an XML file). All data provided by Doclet API can be represented either as some elements or their attributes.

FlexDoc/Javadoc implements that idea. It maps all the Doclet API classes (such as javax.lang.model.element.TypeElement) to “elements” and the values returned by the methods of those classes – to the element's children and “attributes” (see below).

As a result, the entire Doclet API is mapped onto some XML-like representation called Doclet Data Source Model (or Doclet DSM). The elements and attributes, on which everything is mapped, are called DSM elements and DSM attributes.

Since Java 9, Javadoc has been switched from the old Doclet API, which was focused more literally on the representation of Java program elements, to the new API that consists of a rather loose collection of interfaces equally used by other parts of JDK. It means that the names of those interfaces are rather generic and the actual meaning of the objects they provide can be recognized only from their properties (like all sorts of getKind() methods). Moreover, some Java program elements may be represented by a combination of several such API objects, which from the point of Javadoc would be rather a single one.

For instance, a method of a Java class is now represented simultaneously by two Doclet API interfaces:

  1. javax.lang.model.element.ExecutableElement with getKind() == ElementKind.METHOD

    that provides the information about the method definition.

  2. javax.lang.model.type.ExecutableType

    that provides all types associated with this method (i.e. return type, parameters types and exception types) adjusted to the method's containing class.

Doclet DSM incapsulates all of that into a single DSM element 'Method'. Basically, it tries to follow the old Doclet API architecture, which was more meaningful for Javadoc.
Actually, that XML-like representation is not created in memory. Rather, it is maintained dynamically by the Doclet DSM Driver. When a particular element or attribute is needed, the driver just calls the corresponding method of a Doclet API class.

The methods of Doclet API classes (or more precisely, the data they return) are mapped according to the following rules:

The mapping of Doclet API to the XML-like representation (Doclet DSM) is called Doclet DSM Type. It consist of a number of DSM Element Types, which are in effect XML-like representations of particular Java interface of the Doclet API. The DSM elements themselves, in turn, represent the instances of those Java interfaces.

The Doclet DSM Type serves as a base for designing the templates and their processing. You can view it in the Template Designer by invoking FlexQuery Assistance Dialog (e.g. via the main menu: Help | Assistant) as shown on the following screenshot (click to see a more expanded form):

In effect, the tree visible on this picture represents all Doclet API in a very simple but equally powerful form.

Data mining by Location Paths / Location Rules / Element Maps

Most of data mining is based on Location Paths / Location Rules, which is a developed by us extension of XPath. Location Rules are particularly suitable for retrieval of various datasets (i.e. collections of DSM elements) immediately and directly from Doclet DSM. See Example 1.

However, some important datasets cannot be obtained in that way, particularly those that are actually the inversions of datasets obtainable by Location Rules. In that case element maps are used, which are hash-maps specifically adapted for DSM elements. They serve the role of indexes and need to be prepared in advance. See Example 2.

Example 1
On the following screenshots you can see how the generation of «Class Ancestor Tree» is programmed:
Example 2
The following screenshots demonstrate the generation of the list of «All Known Implementing Classes» for a particular Java interface:

The 'all-known-implementing-classes' element map is created preliminary in init.tpl template. It is done in a way inverse to what we want to get from that map:

The last step is shown on the screeenshot.

Formatting

WYSIWYG capabilities of template components

All formatting of the generated output is specified as properties of template components. In Template Designer, this is visualized in a form resembling the output those components generate.

On the left screenshot you can see how the generation of documentation navigation bar is programmed (navbar.tpl). On the right are samples what this template generates:

In that way the following formatting features are supported:

During the generation, the formatting properties of components are rendered with appropriate features available in the selected output format.

Formatting styles

Formatting properties of template components can be defined and stored independently of components themselves, in special definitions called template formatting styles:

Basically, each formatting style is a collection of component formatting properties assigned with a certain name. By that name the style can be assigned to a template component. When a component has a formatting style, those formatting properties not specified on the component directly are inherited from the style.

Each formatting style may include properties only from a specific group determined by the style type. There are following style types:

On the following screenshots you can see the definition of a paragraph style:

Formatting styles are template local things. That is, they are not imported from somewhere outside. However, when a template is called from another template, the styles defined in the caller template will override the equally named styles defined in the called one. When multiple templates are called one from another, that will work in a chain. So, the styles defined in the main template will override any equally named styles defined elsewhere. As a whole, formatting styles behave as additional template parameters passed implicitly by their names.

Custom CSS rules / CSS file

In case of HTML output, FlexDoc/Javadoc support the traditional way of styling the generated documentation through the custom CSS style sheet. It is done via the named CSS rules generated from template formatting styles.

On these screenshots you can see the same documentation with default (left) and custom (right) CSS. Click on each screenshot to see the real HTML.

See Also:

Rendering of HTML markup in Java comments

This means interpreting in the generated output the user HTML tags embedded in Java comments.

In case of HTML output, little needs to be done (if only fixing wrong HTML markup).

However, in case of RTF, the embedded HTML needs to be fully parsed and interpreted with the corresponding formatting features available in RTF, at that this everything must be integrated with the formatting imposed by the settings in templates. That is supported for almost all HTML tags practically usable in documentation comments (including inserting of images):

Text <span>, <b>, <strong>, <i>, <em>, <cite>, <code>, <tt>, <u>, <s>, <strike>, <sub>, <sup>, <font>, <br>
Paragraphs <div>, <p>, <center>, <pre>, <blockquote>, <h1>, <h2>, <h3>, <h4>, <h5>, <h6>
Lists <ul>, <ol>, <li>, <dl>, <dt>, <dd>
Table <table>, <caption>, <tbody>, <thead>, <tfoot>, <tr>, <th>, <td>
Other <hr>, <img>, <a>...</a>

Example:

The following are screenshots of pages of such an RTF JavaDoc (click to enlarge):

Using formatting styles to render CSS classes

As the HTML markup in Java doc-comments becomes more sophisticated, now the JDK API source code uses CSS classes to style various things.

For instance, the following HTML table markup can be found in the doc-comments of package-info.java for jdk.javadoc.doclet package (the left inset):

HTML table markup
<table class="striped">
  <caption>Short form options mapping</caption>
  <thead>
    <tr> <th rowspan="2" style="vertical-align:top">
             Older option
         <th colspan="4">
             Equivalent to these values with the new option
    <tr> <th>{@code --show-members}
         <th>{@code --show-types}
         <th>{@code --show-packages}
         <th>{@code --show-module-contents}
  </thead>
  <tbody>
    <tr> <th>{@code -public}
         <td>public
         <td>public
         <td>exported
         <td>api
    <tr> <th>{@code -protected}
         <td>protected
         <td>protected
         <td>exported
         <td>api
    <tr> <th>{@code -package}
         <td>package
         <td>package
         <td>all
         <td>all
    <tr> <th>{@code -private}
         <td>private
         <td>private
         <td>all
         <td>all
  </tbody>
</table> 
CSS rules for striped table
table.striped {
    margin-top: 10px;
    margin-bottom: 10px;
    border-collapse: collapse;
    border: 1px solid black;
}
table.striped > caption {
    font-weight: bold;
    font-size: smaller;
}
table.striped th, table.striped td {
    padding: 2px 5px;
}
table.striped > thead {
    background-color: #E3E3E3;
}
table.striped > thead > tr > th,
table.striped > thead > tr > td {
    border: 1px solid black;
}
table.striped > tbody > tr:nth-child(even) {
    background-color: #EEE
}
table.striped > tbody > tr:nth-child(odd) {
    background-color: #FFF
}
table.striped > tbody > tr > th,
table.striped > tbody > tr > td {
    border-left: 1px solid black;
    border-right: 1px solid black;
}
table.striped > tbody > tr > th {
    font-weight: normal;
}

Coupled with the CSS definitions in the inset on the right, that will be rendered by HTML browsers as the following striped table:

Since HTML tags and CSS rules cannot be interpreted directly in a non-HTML output (like RTF), to achieve a similar result in those formats, FlexDoc/Javadoc can render some HTML elements and CSS classes using template formatting styles. Here is how it is done for striped tables as the one above:

  1. An HTML Rendering Rule is defined to render <table> elements with 'striped' CSS class:
  2. The formatting styles to render striped tables. The 2nd and 3rd screenshots is the style for striped rows.
  3. Now, the same striped table can be rendered in RTF as well:

Insertion of images

Static Images

When the image file is already known at the time of template design (e.g. various icons), it can be specified directly in the template using Image Control. Then, such an image is instantly displayed in the designer pane (along with other controls).

On the left screenshot you can see a template with a static image, on the right is what it generates (click to enlarge):

HTML Images

Independently of what is specified about images in templates, they may come also from <img> HTML tags found in input Java comments.

Example:

/**
 * Mystic Flowers:
 * <p>
 * <img src="http://www.flexdoc.xyz/images/flowers.png">
 */
String MYSTIC_URL;

After processing such a comment, the generator will produce the following fragment of HTML documentation:

Mystic Flowers:

Supported Image Formats

FlexDoc/Javadoc generates no images by its own. Any images must either be prepared beforehand or come from various external sources. However, FlexDoc does need to read images in order to display them in Template Designer or, at least, to obtain some of their properties necessary to insert the image correctly in the generated output.

The following formats of incoming images are supported: GIF, PNG, JPG, WMF, EMF, BMP

Hypertext

Cross-Reference Links

Cross-links are produced by matching sets of special keys generated for both link source and destination (target). The keys represent actual data being linked. Their generation is specified in templates along with the generation of the corresponding pieces of output.

Example:

Suppose we need to generate a list of some classes and link the list items to the corresponding class details:

The list is generated by a template section like the one shown below on the left screenshot. Each list item is produced by the Data Control (selected) specified to print the component name. The hyperlink generation is defined here in the Data Control's properties (right screenshot). It is determined by a pair of keys: {contextElement.id, "detail"}. The first key 'contextElement.id' represents the component itself (it's the unique ID of the DSM element representing the class). The second key is the string "detail". It says that what we want to link to is the class' entire details. This is needed because there may be other sections in the documentation dedicated to the same Java class, which may be linked to as well (including some subsections within the detail).

The class detail is generated by class.tpl template shown below (left screenshot). The link target is defined in the template properties (right screenshot). It's again determined by the same pair of keys: {contextElement.id, "detail"}.

Such definitions allow the generator to create hyperlinks automatically. Each time the sets of keys produced by some possible link source and destination are matching, the actual hyperlink is generated. In that way anything can be cross-linked together. At that, if a particular set of keys cannot be matched, an alternative set of keys can be specified as well. That means, when the primary target is absent, the link can go to some alternative location related to the link source.

External Links

Similar to cross-links can be programmed the generation of outgoing hyperlinks to any external URLs. The link URL is produced by an expression specified in the properties of the link definition.

On the following screenshots you can see how the generation of class name hyperlink is specified. The first “DOC” definition (left screenshot) is for the cross-link to the local class detail in the generated documentation. If such an internal destination does not exist, the external hyperlink is generated (according to -link option). That is the second “URL” definition (selected). On the right screenshot you can see its properties. The external URL is produced by a FlexQuery-expressions specified in «Expression for URL» field.

The incoming hyperlinks from external pages into the generated documentation can be supported as well. Each hypertext target (anchor) is generated by a definition like shown below (left screenshot below). It is specified in the properties of the component whose output is targeted. Typically, the anchor name is generated automatically and, therefore, meaningless. But it can be any predefined string generated by an expression specified in the properties of the target definition (right screenshot below), which can be used in the URL of the incoming link.

On the right you can see the definition of both cross-link target (in «Target Keys» box) and the anchor with a specific name (the method signature) for external incoming hyperlinks:

Hyperlinks in Different Output Formats

Basically, hyperlinks can be generated in all output formats (included even plain text). However, there are some specifics:

Format What's special
HTML Hyperlink Title

It is a small text generated by a «Link Tile» expression specified in the properties of the hyperlink definition. In HTML, hyperlink titles are set using title attribute, like this:

<a ... title="link tooltip text"> ... </a>

Most browsers interpret it as a hyperlink tooltip (a tiny window with the text that appears near the hyperlink when the mouse cursor is moved over it):

RTF Page Number References

RTF supports similar hyperlinks as HTML. But this is good in electronic form. Once you have printed something on paper, you cannot click on it. So, how to navigated the printed documentation?

In RTF, cross-links can be represented also in the form of page number references, which is programmed as follows.

A Data Control is specified in a template:

to generate a “Page Number” document field:

The hypertext tab is specified the same as in case of cross-link:

That's all. The generated RTF will look as on this page (with page number references in “Page” column; click to enlarge):

TXT You can generate cross-hyperlinks in plain-text files too. What exactly a hyperlink is depends on the final format you generate using the plain-text output. FlexDoc/Javadoc just provides the necessary functionality to find the things you want to link from and to. This is implemented in the form of new FlexQuery functions and GOM types/properties ("GOM" stands for Generator Object Model):

Function/Type/Property Description
findHyperTarget() Finds a hyperlink target matching the specified set of keys that may exist anywhere in the whole generated documentation. Returns a GOMHyperTarget object describing the target.
GOMHyperTarget Provides information about a hyperlink target. This includes the output file containing the target and the target name.

More information about these functions and types you can find in the Template Designer | Help | Assistant:

  • Functions | By Category | Hypertext Functions
  • Generator Object Model | Object Types

Output formats

HTML 4.01 / XHTML 1.0

Both framed multi-file and single-file output can be generated. Almost all formatting is expressed with CSS. It is possible also to replace some of the generated CSS rules with custom ones (see FlexDoc.XYZ | Documentation | Usage of CSS in generated HTML).

RTF

version 1.6 - supported since Word 2000; can be further converted to PDF

TXT (plain text)

It may be particularly useful for various utilizations of the Java code information provided by the Doclet API (for instance, to generate XML files by it, or SQL files to import into a database).

Generation of XML

Since XML files are just text files, you can easily generate any sorts of XML output from the data provided by Doclet API using FlexDoc/Javadoc.

To generated XML, all you need is to design a template where the output produced by Data Controls (from the data obtained from Doclet API) is interchanged with the static text representing certain XML tags, which can be produced, for instance, by Text Controls. Then, you can simply generate a TXT output with such a template, and this will be XML.

Doclet GUI

Why doclet GUI?

A documentation generator implemented as {FlexDoc Doclet + template set} is controlled by a lot of settings, which include: Of course, all such settings can be assigned using options on the Javadoc command line (most, actually, have their default values). But there may be so many of them!

So, besides command line options, FlexDoc Doclet provides a more user-friendly way to control the template-based doclets interactively – the Doclet GUI.

Generator Dialog

When no -nodialog option has been specified on the command line, as soon as Javadoc finishes parsing Java sources and calls the doclet to generate the output, the FlexDoc Doclet invokes the following Generator Dialog (click to enlarge):

Here, you can fill in all required settings.

The «Params» button invokes the Parameter Inspector, where you can edit all parameters of the specified main template. The «Options» button invokes the Option Inspector corresponding the selected output format.

When all settings are prepared, you can start the generation by clicking «Run» button. Then, the generator dialog will transform itself to show the generation progress.

You can stop the generation at any moment by clicking «Cancel» button. Once the generation has finished (or cancelled), the generator dialog transforms itself back into its initial state.

After that, you can change any settings and start the generator again with a new/modified template set. This, in effect, allow you to change/modify your doclet on the fly. At that, you won't need to restart the whole Javadoc (with all the parsing of your Java sources). All the memory representation of your Java project provided via the Doclet API will remain the same.

That unique capability makes possible very fast designing/modification of your templates. Just change something in the Template Designer, restart the generator by clicking «Run» and you can immediate see how the result output looks now!

Template Parameter / Output Format Option Inspectors

The additional settings, such as template parameters and output format options can be assigned in the special property inspector dialogs invoked by clicking the buttons on the right (click to enlarge):

Here you can edit the parameter or option values according to their data types. The bottom panel in the inspector dialog shows the HTML-preformatted description of each parameter or format-specific generator option.

For further details, please see: FlexDoc/Javadoc | Documentation | FlexDoc Doclet | Doclet GUI

Ready template applications

JavadocClassic

Implements a powerful Java API documentation generator with the following features: For further details, please see: FlexDoc/Javadoc | JavadocClassic

Integrations