It is based on an idea that most of the data a typical application operate have a tree or network structure and generally can be represented in the form similar to Document Object Model (DOM) of XML files. (In fact, that's why XML is widely used to export/import those data and exchange between applications.)

Following that idea, it is possible to imagine that if we had some abstract interface that represents most of the practically used data sources in a from similar to DOM, we could process such data using a single tool, which is based only on the concepts of XML (possibly with a few enhancements) and employs available standards and approaches developed around it.

DocFlex technology was devised as an implementation of that idea for the task of representing the application data in some human-readable form, i.e. generating various reports and documentation.

Another idea related to that task and sought to achieve in DocFlex Technology is a universal approach for specifying formatting of the output documents and generating a common network of hyperlinks interconnecting them.

The natural way to achieve all those goals would be to introduce a new programming medium in the form of a scripting language or templates.

A scripting language is typically something intended for manual coding, compiling and running with an interpreter. Normally, this approach provides powerful expressive capabilities however in expense of a rather laborious way of using them. In addition, in the case of XML, such a scripting language already exists. It is XSL, which is an elaborated programming language (the second generation now). XSL probably represents almost all what could be achieved in that way. Yet, it has lots of limitations.

The templates may be thought as a less flexible approach rather than scripts, but exactly this makes them easier to create using some specialized visual designer tool.

That last approach was chosen as a base for DocFlex Technology. Actually, the DocFlex templates represent a mixture of both approaches, since small scripts called FlexQuery expressions can be used to dynamically calculate some properties of more rigid (but visually expressible) constructions called template components.

2. What are Templates?

The DocFlex templates are actually programs. The idea behind the template approach is the following.

If you consider how almost any Java library that generates something (and not only generates -- for instance, powers a GUI system) works, you will notice that, basically, it may be described like the following:

First, we need to create a certain object representation of some controlling structure (using special classes and methods of that library).

Further, we call an interpreter included in that library and pass to it that controlling structure along with some data source we want to process, e.g. some data file, some interface or whatsoever.

Actually, such an interpreter may be started just by calling some member method of the root object representing that controlling structure. For instance, in case of a GUI system, this may be as simple as dlg.show().

Finally, the interpreter processes that controlling structure and produces from the data source some useful output (or shows a GUI).

What the actual programming is all about here, when using such a system, is the first step. We need to encode the creation of that controlling structure! (And this may be a piece of work indeed...)

Now, let's imagine that such a controlling structure is serialized to a file (and, therefore, can be created from it). What is that file, then? One may call it a “template”!

But if that is a template, why not to create and edit it using some template designer GUI? That approach has a number of advantages:

A designer GUI can visualize everything. Now, rather than coding abstract method calls, we can visually construct the controlling structure using the notions of things that we expect to see in the generated output.
Since probably every modern document format in its core is based on the same obvious concepts like flows of text with specified fonts and colors, paragraphs with specified margins, borders and other properties, tables, bulleted/numbered lists, images and so on, most of such things can be defined in some universal format-independent way, which can be encoded in the form of template components.
Such template components can be visualized in the template designer GUI. This allows easy manipulation with them (creating/defining, modifying, moving, coping and so on). Doing the same by calling various Java methods and modifying Java code each time we need to change something would be a lot more cumbersome!
On other hand, the template interpreter will render those template components with the suitable features of the particular destination output format (such as HTML or RTF). So, we can be relieved from learning and programming lots of very complicated (and sometimes poorly documented) things specific to a particular format. Instead, we can more concentrate on our primary application task.
The template designer (to a large degree) ensures that the result controlling structure (the template) is valid. That is, it won't hang when interpreted and will produce some output anyway.

DocFlex templates are exactly an implementation of that idea!

The following screenshot shows a template open in the Template Designer (click to enlarge):

3. Data Sources

DocFlex core connects to one or several data sources provided by a Java application using special drivers written with the DocFlex API.

Any such driver represents a particular data source in the form structurally similar to W3C DOM and understandable for DocFlex core. Such a representation is called Data Source Model (or DSM) and any driver supporting it -- DSM Driver. Such a driver provides the DSM data via special DSM Interface, which it implements.

Any DSM may be considered as a virtual XML document. It contains elements (called DSM elements) and attributes (called DSM attributes). However, unlike elements/attributes in XML files, DSM elements and attributes are not real things. Actually, they are instances of specific interfaces implemented by the DSM Driver, which makes them capable to dynamically provide a lot more information from the underlying data source (rather than an ordinary XML document).

All possible data sources processed by the same DSM Driver are described by a certain DSM Type (which may be considered as an equivalent of XML DTD). The DSM driver provides the data structure and type information associated with this DSM Type via special DSM Type Interface.

Actually, some DSM Driver implementations may process different DSM Types. For instance, DocFlex/XML includes the XML Model Driver, which works with the data from XML files. That driver can be initialized from different DTDs (or XML Schemas) passed as initialization parameters so as to represent each time a different DSM Type associated with the given DTD (or XML Schemas).

In fact, all existing applications of the DocFlex Technology (such as DocFlex/XML, DocFlex/Javadoc and DocFlex/Together) represent little more than the DocFlex core plus one or more specific DSM drivers.

4. How Data Processed

Data processing scheme implemented in DocFlex in many ways is similar to that used in XSL Transformations. In brief, it can be described as the following.

DocFlex templates are built of nested components called sections. Most important of them are Element Iterators and Area Sections. An Element Iterator performs iterations by a set of elements which it produces from the context element received from the iterator's parent. The set of iterated elements is generated according to the Location Rules, which are settings based on a subset of XPath specification (however, with some important extensions that allow traversing element references). During the iterations, each iterated element is selected as a context element and the iterator's content is repeated for it. Area Sections do not perform iterations. Instead, they are programmed to generate some output from the context element. The template interpretation starts from the template's root section block. The root element of the processed XML document becomes the first context element.

It can be seen, that Element Iterators produce the same effect as <xsl:template> / <xsl:apply-templates> directives and DocFlex templates themselves are analogues of XSLT-scripts.

However, it should be noted that DocFlex is not directly based on XSL and does not use any XSLT processor internally. Instead, this is a completely new attempt to merge into a single tool:

The ability to work with both XML files and any data sources provided by Java applications in some universal way based on notions of XML and accompanying standards.
The capabilities of data retrieval and processing comparable with those of XSLT (and beyond).
The ability to process data from multiple independent data sources (both XML files and provided by a Java application) and to generate by them single output.
The simplicity of designing and generating the professionally formatted hypertext or plain output documents.

For further information, see Data Processing and Templates/Template Designer features, where you can find many details about DocFlex templates and data processing capabilities.

See also DocFlex/XML | Samples | Alternative to XSLT, which demonstrate how DocFlex can be used as an alternative to XSL Transformations.

5. Generated Output

There are two types of output that can be generated by DocFlex:

Single file documents
Framed multi-file documentation

For those output types there are two types of templates: document template and frameset template.

Each interpretation of a document template normally results in generation of a single output document. Document templates also are the places where all the output formatting specified.

The DocFlex architecture was devised so that to allow designing document templates independently on a particular destination format. The specific format is selected just at the moment of starting the generation. All instructions and settings specified in a document templates will be rendered with appropriate features of the selected output format. For more information about formatting techniques supported in DocFlex, see Formatting Features.

Each document template is associated with a particular DSM Type. This DSM Type is the source of information about the data the template is intended to manipulate. That information is used during the template design.

When the template is interpreted by the generator, an output document produced by it normally contains data from a single DSM for whose type the template was designed.

However, DocFlex provides a way for designing document templates whose output can contain data from several DSMs and even of different DSM Types. This can be achieved by calling from the main template the subordinate ones (or subtemplates).

Each subtemplate is just yet another document template and designed the same way as the main one. When interpreted alone it may also produce a single document. But when called from another document template (a master) it becomes the subtemplate and can be specified to produce not a separate document but a fragment of the main document generated by the master.

Since each document template can be assigned with its own DSM the resulting document will contain data from the multiple DSMs. Which particular DSM is passed to the subtemplate during a call is specified when defining the subtemplate call within the master template. It is also possible to generate hyperlinks between the document's parts produced by different DSMs. See Call Template for more detail.

Frameset templates are entirely based on calls of subtemplates. This time, each subtemplate call generates a separate subordinate document. A frameset template itself generates a special frameset document which contains no actual data, but instead, specifies the frame windows where subordinate documents are displayed.

A typical example of framed documentation is the output produced by Java 2 SDK Javadoc (Standard Doclet). DocFlex can do just the same (see DocFlex/Javadoc | Examples), but now you can design the frameset structure by yourself and specify what exactly is loaded in the frame windows.

Currently, the frameset templates can generate output only in HTML format.

6. DocFlex SDK

DocFlex SDK (Software Development Kit) is an implementation of DocFlex Technology and represents a tool for development of template-based documentation/report generators built into Java applications.

From the external view DocFlex SDK consists of three main components:

DocFlex API
DocFlex Template Designer
DocFlex Generator

DocFlex API

Provides Java-programming interfaces for integration of the DocFlex core with the particular Java application. These interfaces allow

Developing the DSM Drivers, which represent the application's data in the DOM-similar form understandable for DocFlex core.
Besides delivering the data (via DSM Interface) and providing the data structure and type information (via DSM Type Interface), a DSM Driver may also supply
- The graphic images, which may represent some data elements in a specific graphical form (for instance, diagrams, screenshots etc.) to be inserted into the generated documentation. The DSM Driver may also provide imagemaps for such images to generate hyperlink within them.
- The small and big icons representing the data elements, which may be useful for generation of various summaries and references in more illustrative form.
- The additional utility functions which can be called within the FlexQuery expressions in templates to allow access to some specific data and functionality supported by the application.
Launching the DocFlex Template Designer from the Java application.
Launch the DocFlex Generator from the Java application with the ability to track the generation progress and show it (by the application itself in the preferred way) as well as to cancel the generation at any moment.

DocFlex Template Designer

Represents a high quality Graphic User Interface that allows creating and designing DocFlex templates.

This is a ready to use component.

The Template Designer checks all registered DSM Drivers and collects the list of all available DSM Types supported by them.

Using the DSM Type information provided by a particular driver (via DSM Interface), the Template Designer allows to design a template for a selected DSM Type.

DocFlex Generator

This is another DocFlex SDK component, which represents an engine that produces the output documents.

Actually, the DocFlex Generator comprises a set of components because it is designed as some abstract generator plus a number of extensions for each supported output format. Each extension, in effect, represents a separate generator.

The Java application should start some of those generators and provide it with a specified template and a DSM Driver initialized to supply the application's data. The template should be designed for the DSM Type supported by this driver. Then, the generator connects to the DSM provided by the driver and by interpreting the instructions from the templates generates an output document.

This is a simple scheme which involves one template and one data source.

Actually, such a generation may be much more complicated. The application may register multiple DSM Drivers able to dynamically initialize connections to different DSMs according to the parameters they receive from the generator.

Then, the generator is started as previously with a some initial template (called main template) and an initial DSM. But now, the main template may call from itself another templates (subtemplates). Those subtemplates may be designed for absolutely different DSM Types (for instance, the main template may work with a specific application's internal data source, whereas subtemplates may be designed to take their data from some external XML files).

When processing such a call, the generator looks for a registered DSM Driver appropriate to the called subtemplate's DSM Type and initializes it with the parameters specified in the call (for instance, a particular XML file name). Then, the subtemplate is processed so it takes its data from the new DSM initialized for it (e.g. the external XML file).

As the processing of the subtemplate finished, the generator returns to the main template and continues processing it again. In the mean time, during all those changes, the generated output will remain the same. So, the result document will contain data both as directly from the application itself as from some external data sources (e.g. XML files).

Multiple DSM Drivers may be used also when a Java application, though being a single unit, may operate its data using very different interfaces (sometimes based on different notions and directly incompatible with each other). Anyway, there may be some indirect connections between the data provided by those interfaces and, correspondingly, a desire to generate some single documentation by those data.

DocFlex Technology may allow to handle such a task. For example, when developing DocFlex/Together, we have successfully integrated two absolutely different interfaces: RWI and UIBuilder (both provided by Together OpenAPI), so to make possible generating a single documentation from the data supplied by both of them (see DocFlex/Together | Examples | User Interface Documentation).

The user interface for launching the generator should be implemented by the particular Java application. Below, you can see a screenshot of the generator dialog invoked from the DocFlex/XML:

And this is how the dialog looks, when the generator is running: