Latest revision as of 10:25, 3 October 2011

The Content Manager service gives uniform access to content served by a variety of back-ends, both inside and outside the system. It is the central component of the gCube subsystem which deals with the organisation of content and related data.

We overview here the architecture, content model, and interfaces of the service . Related libraries, such as the Content Manager Library (CML) and the gCube Document Library (gDL), are discussed separately.

Architecture

The Content Manager is designed as an OCMA service. In OCMA terms, it classifies as a multi-type, 1-N adapter service:

it is a multi-type service because it supports two front types for, respectively, reading and writing content modelled as labelled trees.

Collectively, the front types and the tree content model form the gDoc access type of the service.

it is an adapter service because it adapts the gDoc access type to multiple back types, where each back type corresponds to the interface and content model of a class of remote repositories.

For this, the service employes an open architecture of type-specific plugins to which it delegates the creation and operation of its collection managers.

Plugins are dynamically deployed within service instances, and different instances may host different plugins. In addition, some plugins may support both service front types, i.e. grant read and write access to the corresponding class of repositories. Others may instead support read-only access or, less commonly, write-only access.

The figure below overviews the design and use of the service in the context of one its instances.

The instance exposes three stateful port-types:

the ReadManager port-type serves as the interface of collection managers that offer read-only operations over the content of the bound collection.

The operations of ReadManager resources and their Resource Properties are discussed here.

the WriteManager port-type serves as the interface of collection managers that offer write-only operations over the content of the bound collection.

The operations of WriteManager resources and their Resource Properties are discussed here.

the Factory port-type serves as the front-end of a single WS Resource that creates ReadManager and WriteManager resources .

The operations of the Factory resource and its Resource Properties are discussed here.

Internally ReadManagers and WriteManagers share common state in a CollectionResource, a stateful resource that acts as an internal model of a remote collection accessed through the collection managers. The CollectionResource remains private to the service is not published within the system.

All the stateful resources of the service, whether public or private, are logically extended by components provided by service plugins, the resource delegates. The interactions between service and plugins are discussed in detail here, but we can summarise them here as follows:

the factory delegate extends the Factory in order to handle requests which are specifically addressed to the plugin;
at each such request, the factory delegate processes plugin-specific parameters to create one ore more "collection delegates", one for each collection that should be exposed through the service in order to satisfy the client request.
the Factory receives such delegates and performs three key tasks:

it initialises the delegates, a process during which each collection delegate may create and stage read delegates and write delegates, i.e. logical extensions of ReadManagers and WriteManagers.
its creates CollectionResources, ReadManagers, and WriteManagers to match the available delegates, injecting collection delegates into CollectionResources so as to form the web of relationships shown in the figure above.
it publishes collection profiles, one for each collection modelled within this process.

later requests to ReadManager and WriteManager resources are handled by their read and write delegates, respectively, which translate the requests against the back-end repository that exposes the collection bound to the managers.

Finally, note that all stateful resources of the service are persistent and may thus be re-activated across restarts of the gCube Hosting Node:

the Factory persists the history of its activations, i.e. the activation records that it published and/or processed (so as to not process them again).
the CollectionResource persists its collection delegate and, with it, the read and write delegates.

Content Model

Architectural considerations aside, the most distinguished element in the design of the Content Manager is its content model. Rather than settle for a fixed set of document structures, the service adopts a generic structure that can act as a 'carrier' for an arbitrary number of concrete document models. In particular, the service deals with edge-labelled and node-attributed trees, the gDoc trees.

The expectation here is that producers (service plugins) and consumers (service clients) will convene on concrete document models and exchange gDoc trees with an agreed shape. The agreement may be bilateral or involve any number of parties, and it may apply to the entire document or to distinguished parts of it (e.g. document metadata, annotations, raw content packaging, etc). For maximum decoupling between consumers and producers, the agreement may reflect system-wide conventions and result in canonical tree forms, most noticeably those that define the gCube Document Model.

gDoc Trees

A gDoc tree has the following properties:

its nodes may have an identifier and a number of uniquely named attributes;

its edges have a label;

its leaf nodes may have a value;

its root may identify the collection of the corresponding document.

In particular:

identifiers, attributes, and leaves have text values;
attribute names and labels may be qualified with a namespace.

The figure below uses a graphical representation to show an example of a gDoc tree.

gDoc trees serialise to XML documents for exchange over the network. In particular:

nodes serialise to elements and attributes serialise to element attributes
- elements are named like the edges that enter the corresponding nodes, with the exception of the document element which is named http://gcube-system.org/namespaces/contentmanagement/gdoc:godc
the elements that correspond to inner nodes contain the elements that correspond to their children
the elements that correspond to leaves contain their value
node identifiers serialise to attributes called http://gcube-system.org/namespaces/contentmanagement/gdoc:id
collection identifiers serialise to attributes called http://gcube-system.org/namespaces/contentmanagement/gdoc:collID

For example, the gDoc tree above serialises as:

note gDoc trees inherit constraints from their XML serialisation. In particular, the names of edges, the names of attributes, the values of attributes, and the values of leaves are regulated by the definition of the format.

gDoc API

The XML serialisation of gDoc trees is 'natural', in that it does not employ dedicated element structures for the representations nodes, edges, attributes, etc. This streamlines its manipulation with standard XMl technologies (e.g. XPath, XSLT, XQuery, DOM, SAX, etc.) and does not inhibit object binding technologies (e.g. JAXB, XStream, etc).

As a native option, however, the service defines a bespoke object model and API for gDoc trees which offer:

dedicated support for tree processing requirements associated with the use of the service;
transparencies and optimisations for tree storage, construction, deconstruction, and input/output.

While the model is available to service clients, it also forms the basis of the interface between the service and its plugins. For this reason, its main features are overviewed here while its client-oriented features are discussed [[Content_Manager:_Stub_Distribution#Stub Distribution|elsewhere].

As the figure below illustrates, the model is defined in org.gcube.contentmanagement.contentmanager.stubs.model.trees in terms of the following components:

Node: an abstract base for nodes with an identifier, a state, and a map of QName-ed attributes.
State: an inner enumeration of Node for node states.
Edge: A QName-ed edge to a target Node.
InnerNode: a Node with a list of outgoing Edges.
Leaf: a Node with a value.
gDoc: an InnerNode with a collection identifier.
Nodes: a collection of static utilities to generate Nodes and Edges.
Bindings: a collection of static utilities to serialise and deserialise Nodess to and from DOM trees and/or character streams.
NodeView: a base class for JAXB bindings to Nodes.
GDocView: a NodeViewM for JAXB bindings to GDoc nodes.

The model API is illustrated by example in the rest of this Section. The full list of methods and their signatures can be found in the code documentation.

Building Trees

The first and obvious way to create gDoc trees is with the constructors of the concrete node classes (GDoc, InnerNode, Leaf). As a first example, the following code illustrates the creation of a tree with an attributed root and two leaf nodes:

GDoc doc = new GDoc("someid");
doc.setAttribute(new QName("x"), "1");
doc.setAttribute(new QName("someNS","y"), new Date().toString());
doc.collectionID("...");
 
Leaf leaf1 = new Leaf(null,"2"); //no identifier
Leaf leaf2 = new Leaf(null,"true");
 
Edge e1 = new Edge(new QName("a"),leaf1);
Edge e2 = new Edge(new QName("someNS","b"),leaf2);
 
doc.add(e1,e2);

While already more convenient than cross-language and format-oriented tree APIs (e.g. DOM), step-by-step construction is verbose, even in the case of small trees. For a first degree of improvement, the node classes offer rich suites of constructors and setter overloads that allow for more 'in-lined' tree constructions and absorb the creation of QNames:

GDoc doc2 = new GDoc("someid",
		new Edge("a", new Leaf(null,"2")),
		new Edge("someNS","b", new Leaf(null,"true")));
 
doc2.setAttribute("x", "1");
doc2.setAttribute("someNS","y", new Date().toString());
doc2.collectionID("somecollID");

For additional convenience, the Nodes class defines a large number of generators, i.e. factory methods that can be statically imported and then composed into an 'embedded expression language' for gDoc trees:

import static org.gcube.contentmanagement.contentmanager.stubs.model.trees.Nodes.*;
...
 
GDoc doc3 = attr(
		gdoc("somecollID","someid",
			e("a",2), 
			e("someNS","b",true)),
            a("x",1),a("someNS","y",new Date()));>

Here, gdoc, attr, e, a are examples of node, attribute, and edge generators. Besides allowing fully in-lined tree expressions without the use of the new operator, the generators offer QName creation transparencies and object-to-string conversion transparencies (cf. the int, boolean, and Date example above). The transparency of date conversions is particularly important here, as it ensures adherence to XML serialisation standards that are not natively adopted in Java (e.g. in the implementation of toString). See the code documentation for the full list of available generators, as well as for the additional examples that follow:

doc = gdoc();
doc = gdoc("someid");
doc = gdoc("collectionid","someid");
 
doc = gdoc("1",
              e("a", n("2",            //n() => inner node generator
              e("b",l("3",0)))));
 
doc = gdoc(    //no identifier
                e("a", attr(
                            n("2", 
                                e("b",l("3",0)),            //l()= explicit leaf generator for identity assignment
                                e("a",l("4",0))),
                          a("foo","0"))));
 
 
doc = attr(gdoc("1",
		e("a",l("2",5)),
		e("b",attr(
                             n("3",e("c",4)),
			  a("foo",0))),
		e("c",5)),
      a("x",0));
 
 
doc = attr(gdoc("1",
               e("a", n("2",
               e("b",n("$2")))),
                   e("a",n("a1",
                               e("c",n(
                                          e("d","..."),
                                           e("d",attr(                                       //l()= explicit leaf generator for attribute assignments
                                                        l("<xml>..</xml>"),
                                                     a("w",".."))))))),
               e("b",attr(
                            n("1:/2"),
                       a("w","...")))),
         a("x","http://org.acme:8080"),a("y","<a>...</a>"));

The literal construction of trees is particularly convenient in during testing, though it composes well with the programmatic construction in the development of production code:

Edge edge = ....
InnerNode node = ....;
 
attr( 
   node.add(e("before","..."), edge, e("after","..."))
), a("newattr","...");

note: the node classes override equals for equivalence-based comparisons, and hashCode for their correct use as keys within hash-based data structures, and toString for convenience of debugging.

Serialising and Deserialising Trees

The Bindings class offers static facilities to transform native models of gDoc trees into XML-based models. Two representations are supported natively, based on which other XML-based representation can be produced using standard platform facilities (e.g. TRAX):

Bindings.toElement(GDoc) converts native models of gDoc trees into equivalent DOM models.
Bindings.fromElement(Element) converts DOM models of gDoc trees into equivalent native models.
Bindings.toXML(GDoc, Writer, boolean?) converts the native model into XML document streams, optionally excluding document declarations.
Bindings.fromXML(Reader) converts XML document streams into gDoc trees.

note: DOM conversions of native models are implemented directly, as they are most commonly required for interactions with the Content Manager service. Stream conversions are instead derived from DOM conversions via TRAX, at an additional processing cost.

note: conversions from native models to XML-based models assign the conventional name http://gcube-system.org/namespaces/contentmanagement/gdoc:gdoc (cf. Bindings.GDOC_NS, and Bindings.GDOC_NAME constants) to the document element. Vice versa, conversion from XML-based representations to native models discard the name of the document element.

Here is a usage example, which shows that equivalence of native models is preserved under round-trip conversions.

import static org.gcube.contentmanagement.contentmanager.stubs.model.trees.Bindings.*;
...
GDoc doc = ....
 
//DOM conversion
GDOc doc2 = fromElement(toElement(doc));
assert doc.equals(doc2); //true!
 
//stream conversion
StringWriter w = new StringWriter();
toXML(doc,w);
GDOC doc3 = fromXML(w.toString());
assert doc.equals(doc3);   //true!

note: due to the treatment of root element names, equivalence of XML-based representations is not necessarily preserved after round-trip conversion. It is preserved only if the XML-based representations have been previously produced with the conversion routines.

note: in all the conversions above, null values in attribute and leaf values are serialised using a special constant (exposed programmatically as Node.NULL).

note: the conversions are also available at arbitrary inner nodes, not only roots (cf. Bindings.nodeToElement(Node, QName?), Bindings.nodeFromElement(Element),Bindings.nodeToXML(Node, Writer, QName), and Bindings.nodeFromXML(Reader).

Consuming Trees

The gDoc API offers simple means of procedural tree navigation. For declarative queries, clients can convert the model into an XML-based representation and leverage platform standards and popular offerings (e.g. XPath, XQuery, or XSLT implementations). If required, the gDoc API can then be reasserted on query outputs.

The Node class defines methods to expose the state common to all nodes of a gDoc tree:

id(): returns the identifier.
parent(): returns the parent.
ancestors(): returns the list of all nodes from the parent to the root.
ancestorsAndSelf(): behaves ancestors but the returned list includes and starts with the recipient node.
attributes(): returns a copy of the attributes, indexed by name.
attribute(QName): returns the value of an attribute with the given name (or fails).
hasAttribute(QName): checks for the existence of an attribute with a given name.

note: identifiers can only be set at node creation time. Attributes can be added, modified, and removed at any point (cf. Node.state(Node.State), setAttribute(QName,String), removeAttribute(QName)).

note: for convenience, all methods that take QNames are overloaded to accept local names as well as (namespace,local name) pairs.

note: invoking methods that take node types is simplified by statically importing the class constants defined in the Nodes class (cf. Nodes.N for InnerNode.class and Nodes.L for Leaf.class).

The Leaf class adds methods to read and set the value (cf. value(), value(String)).

The InnerNode class adds methods to navigate along edges and or identifiers:

children(): returns the list of children.
children(QName): returns the list of children under edges whose label matches a given label.
<T extends Node> children(class<T>): returns the list of children of a given node type.
<T extends Node> children(class<T>, QName): returns the list of children of a given node type under edges whose label matches a given label.
child(QName): returns the child under an edge whose label matches a given label (or fails if there are zero o more such children).
<T extends Node> child(Class<T>, QName):returns the child of a given node type under an edge whose label matches a given label (or fails if there are zero o more such children).
descendants(QName*): returns the list of descendants that can be reached following edges whose labels match a given label.
<T extends Node> descendants(Class<T>,QName*): returns the list of descendants of a given type that can be reached following edges whose labels match a given label.
edges(): returns the list of all the edges.
edges(QName): returns the list of edges whose label matches a given label.
edge(QName): returns the list of edges whose labels match a given label (o fails if there are zero or more such edges).
hasEdge(QName): checks for the existence of an edge whose label matches a given label.
labels(): returns the list of all edge labels.
labels(QName): returns the list of labels that match a given label.

note: edges can be added or more removed at any time (cf. add(Edge*), removeEdge(Edge*), removeEdge(QName)).

note: all matches on qualified names are based on arbitrary regular expressions, both on the namespace and the local name of the label.

note: as above, methods that take QNames have overloads that accepts local names and, where appropriate, overloads that accept (namespace,local name) pairs.

note: as above, invoking methods that take node types is simplified by statically importing the corresponding constants in Nodes (cf. Nodes.N, Nodes.L).

The GDoc class adds a method to read and set the collection identifier (cf. collectionID, collectionID(String)).

Finally, the Edge class exposes its label and target (cf. label(), target()).

The following example illustrates some of the supported idioms, do check the code documentation for detailed information about method signatures:

import static org.gcube.contentmanagement.contentmanager.stubs.model.trees.Nodes.*;
 
GDoc doc = attr(gdoc("1",
		e("a",l("2",5)),
		e("b",attr(
				n("3",e("c",4)),
			  a("foo",0))),
		e("c",5)),
           a("x",0));
 
//typed children
String val = doc.child(L,"a").value();
 
//typed descendant
String val2 = doc.descendant(N,"3").child(L,"c").value();
 
for (InnerNode node : doc.children(N))
	for (QName l : node.labels())
		//process label
 
for (Node d : doc.descendants("b","e"))
	for (Edge siblingEdge : d.parent().edges()) 
		if (siblingEdge.target()!=d)
		//process sibling of descendant

Binding Trees

Clients that expect gDoc trees of a given form may wish to bind them to objects. The API offers two classes to streamline JAXB object bindings in the package org.gcube.contentmanagement.contentmanager.stubs.model.views. In particular, it includes two base classes for node and document bindings to XML serialisations of gDoc trees:

NodeView is a base class for node bindings. The view binds and exposes the identifier of the node as well as the URL of the node, if one exists (cf. getID(), getURL()). Node URLs are discussed later.

GDocView extends NodeView as a base class for document bindings. The view binds and exposes the collection identifier of the root node (cf. getCollID()), in addition to what already bound and exposed via its superclass.

Clients can extend these classes and the corresponding bindings. The following example illustrates:

@XmlRootElement(name=Bindings.GDOC_NAME,namespace=Bindings.GDOC_NS)
class MyDocView extends GDocView {
	@XmlElement(namespace="http://acme.org") int i;
	@XmlElement(namespace="http://acme.org") MyDocComponent c;
 
 
class MyDocComponent extends MyNodeView {
	@XmlElement Date date;
}

MyDocView and MyDocComponent are toy examples of user-defined views over gDoc trees and tree nodes, and they should be familiar to JAXB users.

MyDocView extends GDocView and uses JAXB annotations to specify the qualified name of the document elements to which it will be bound. Here we have chosen a name that aligns with the serialisations produced by the Bindings class, as shown above, but different names may be specified if the binding target serialisations produced through different means.MyDocView then includes two fields in its own namespace, an integer field and a MyDocComponent field, both of which are bound to XML elements. MyDocComponent extends NodeView, specifies a single Date, and uses JAXB annotations to bind it to an XML element. In both classes, we have chosen simple JAXB annotations. For example, we have assumed that the gDoc trees that will come to binding have labels that match the field names. The full range of JAXB facilities is of course available to customise bindings to less aligned trees.

Suppose now MyDocView is to be bound to the gDoc tree below. Wee use the generators of the gDoc API to denote the tree, but this is just for convenience of exposition; the tree may have been generated through any suitable means.

GDoc doc = gdoc("collID","123",
                   e(NS,"i",3),
                   e(NS,"c", n("789",
                              e("d",l("1",new Date())),
                              e("b",l("2",15)))),
                   e(NS,"d",new Date()), 
                   e(NS,"b",n("456")));

Clearly, the tree contains a subset that matches the binding expectations of the classes above. As with all JAXB clients, the binding would require steps similar to the following:

JAXBContext context = JAXBContext.newInstance(MyDocView.class);
...
 
//assuming a DOM binding to the tree has already occurred (other JAXB inputs could have been used instead, e.g. character streams)
Element docElement = ....
 
//bind
MyDocView mv = (MyDocView) context.createUnmarshaller().unmarshal(docElement);
 
 
...mv.id()...
...mv.collID()...
...mv.url()...
...mv.i...
...mv.c...
...mv.id()...
...mv.url()...
...mv.c.d...
 
//serialiase (again to DOM)
Document dom = ....;
Marshaller m = context.createMarshaller();
m.marshal(mv,dom);

gDoc Predicates

The gDoc model is untyped, in that neither the topology of trees nor the values of their attributes or leaves are subjected to constraints (beside those dictated by the XML serialisation). Types are reintroduced later, under the view that they can be projected on gDoc trees at the point of consumption.

Type projections serve two main purposes in the context of the Content Manager:

to validate the content of gDoc trees.

The main use case for validation is at the point of content ingestion through the write operations of the Content Manager. In particular, a plugin may project a type on incoming gDoc trees, with a view to rejecting those that fail the projection.

to identify the data of interest within gDoc trees.

The main use case for content identification is at the point of content retrieval through the read operations of the Content Manager. Through the service, in particular, a client may ask plugins to return only the portion of the data that succeeds the projection, and to discard the rest. Content pruning results in minimal bandwidth consumption and delivers content to client in forms which are optimal for their own object bindings.

Accordingly, support for type projections requires:

a language of tree types with which clients and plugins can capture the required shape and content of gDoc trees.
the ability to project such types over gDoc trees with both validation and pruning semantics.

XML schema languages are natural candidates for the choice of tree types. However, the also introduce complexity - both conceptually and in terms of tooling - which is not required when working with the subset of XML that corresponds to the gDoc model. As importantly, schema languages are strongly associated with validation and there are no implementations that use them towards document pruning (or indeed content extraction).

Accordingly, the tree API includes a native language of tree types, the gDoc predicates, as well as support for projecting them over content for validation and pruning purposes. gDoc predicates, in particular, can be used to constrain:

the topology of gDoc trees, including the labels and cardinality of edges (e.g. the existence of at least one edge whose label matches a given label).
the values of leaves, so that they conform to the textual literal of a range of atomic types (e.g. numbers or boolean values) or simply verify some type-specific predicate.

note: Support for predicates on attributes is forthcoming.

Predicate API

gDoc predicates are defined in the packages org.gcube.contentmanagement.contentmanager.stubs.model.predicates and org.gcube.contentmanagement.contentmanager.stubs.model.constraints, the main components of which are the following:

Predicate: the interface of all node predicates, defines match(Node) and prune(Node) methods for validation-based and pruning-based projection semantics.
- AnyPredicate: a Predicate that specifies no constraints on nodes, i.e. matches any node and prunes nothing from it.
- TreePredicate: a Predicate that specifies a list of EdgePredicates on inner nodes.
- LeafPredicate: a Predicate that specifies a Constraints on the value of leaf nodes.
  - Bool: an LeafPredicate that specifies a boolean Constraint on the value of leaf nodes.
  - Num: an LeafPredicate that specifies a numeric Constraint on the value of leaf nodes.
  - Text: an LeafPredicate that specifies a textual Constraint on the value of leaf nodes.
  - Date: an LeafPredicate that specifies a date Constraint on the value of leaf nodes.
  - URI: an LeafPredicate that specifies a URI Constraint on the value of leaf nodes.
  - ID: a LeafPredicate on the identifier of an inner node .
EdgePredicate: a predicate that specifies a node Predicate on the targets of edges whose labels match a given label.
- - One: an EdgePredicate that asserts the existence of exactly one edge whose label matches a given label and whose target matches a given predicate.
  - Opt: an EdgePredicate that asserts the existence of zero or one edges whose labels match a given label and whose targets match a given predicate.
  - AtLeast: an EdgePredicate that asserts the existence of one or more edges whose labels match a given label and whose targets match a given predicate.
  - Many: an EdgePredicate that asserts the existence of zero or more edges whose labels match a given label and whose targets match a given predicate.
  - Only: an EdgePredicate that asserts that all the edges whose labels match a given label match also a given predicate.
Predicates: factory methods for an expression language of tree predicates.

Constraint: the interface of all constraints over values of leaf nodes.
- Same: the Constraint that is satisfied by values that are equivalent to a given value.
- Match: the Constraint that is satisfied by values that match a given regular expression.
- More: the Constraint that is satisfied by values that are numbers strictly greater than a given number.
- Less: the Constraint that is satisfied by values that are number strictly smaller than a given number.
- Before: the Constraint that is satisfied by values that are earlier dates than a given date.
- After: the Constraint that is satisfied by values that are later dates than a given date.
- Not: the Constraint that is satisfied by values that do not satisfy another Constraint.
- Either: the Constraint that is satisfied by values that satisfy at least one of a number of other Constraints.
- All: the Constraint that is satisfied by values that satisfy a number of other Constraints.
Constraints: factory methods for an expression language of Constraints.

Building Predicates

Similarly to gDoc trees, gDoc predicates may be built with classic constructor-based idioms and/or else with predicate generators, a collection of factory methods in the Predicates class and Constraints classes which can be statically imported and then composed into a pseudo expression language for gDoc predicates. We concentrate here on predicate generators as the preferred way to build gDoc predicates. See the code documentation for the constructors available in predicate and constraint classes.

Consider this first example:

import static org.gcube.contentmanagement.contentmanager.stubs.model.constraints.Constraints.*;
import static org.gcube.contentmanagement.contentmanager.stubs.model.predicates.Predicates.*;
 
Predicate p = tree(
               one("a",
                   num(more(6))));

Here, tree() generates a TreePredicate which characterises trees which satisfy a single EdgePredicate. This latter predicate requires that the trees have exactly one outgoing edge with label a and with a leaf target. This leaf must in turn satisfy a Num predicate, i.e. its text value must represent a number and this number must satisfy a More constraint, which requires it to be greater than 6. In summary, we are characterising trees with a single a-edge that ends in a leaf with a number greater than 6.

The following example showcases a range of other predicates and constraints:

import static org.gcube.contentmanagement.contentmanager.stubs.model.constraints.Constraints.*;
import static org.gcube.contentmanagement.contentmanager.stubs.model.predicates.Predicates.*;
 
Predicate p = tree(
                one("a",any()),
                one("b",text(either(is("abc"),is("efg")))),
                atleast("c",bool(is(true))),
                opt("d",tree()),
                many("e",date(future())),
                one("f", uri(matches("^http.*"))),
                many("g", num(all(less(5),more(10)))),
                one("h", text()),
                one("j",text(not(is("somestring")))),
                one("k",id("12345",tree())),
                only("l", num()));

Here, the predicate characterises trees with:

a single a-edge that ends in any type of node, inner node or leaf. (any() is a generator of AnyPredicates);
a single b-edge that ends in a leaf whose value is either one of two strings;
one or more c-edges that end in leaves with a boolean value of true;
zero or one d-edges that end in inner nodes;
zero or more e-edges that end in leaves whose values are dates in the future;
a single f-edge that ends in a leaf whose value is an absolute http URI;
zero or more g-edges that end in leaves whose values are numbers between 5 and 10;
a single h-edge that ends in a leaf, not characterised further;
a single j-edge that ends in a leaf whose value differs from a given string;
a single k-edge that ends in an an inner node with an identifier of 12345;
zero or more l-edge that all end in leaves with numeric values;

note: predicates can nest recursively to match the structure of trees.

note: the Edge-predicates above use plain strings to match edge labels, but they may more generally use qualified names with regular expressions on both namespace and local parts. For example, the following predicate:

Predicate p = tree(
                 atleast("^part.*",tree(
                    one(".*acme.org$",".*",num()))));

characterises trees that have one or more edges whose labels begin with part and whose targets contain edges with no more than one edge with in a namespace that ends with acme.org and with a numeric value.

For a full list of available predicate and constraint generators, see the code documentation of the Predicates and Constraints classes.

Matching and Pruning

A gDoc predicate can be projected over a gDoc tree using the methods match() and prune() common to all Predicates. The first indicates whether tree satisfies the predicate, the second prunes it of all the nodes that are not matched by the predicate.

Consider this simple example:

import static org.gcube.contentmanagement.contentmanager.stubs.model.constraints.Constraints.*;
import static org.gcube.contentmanagement.contentmanager.stubs.model.predicates.Predicates.*;
 
Date d = new Date();
 
GDoc doc = gdoc(
         e("a",-1),
         e("a",1),
         e("a",2),
         e("b","..."),
         e("b",n(
             e("b1","..."))),
             e("c",n(
                  e("c1",d),
                  e("c2","..."))),
          e("d","..."));
 
Predicate p = tree(
          many("a",num(more(0))),
          atleast("b",tree()),
          one("c", tree(
                          one("c1",date()))));
 
assert p.matches(doc)==true;
 
GDoc pruned = gdoc(
              e("a",1),
              e("a",2),
              e("b",n(
                 e("b1","..."))),
              e("c",n(
                   e("c1",d))));
 
assert pruned.equals(p.prune(doc));

Here, it is easy to see that the gDoc tree satisfies the predicate. Accordingly, match() returns true and prune() successfully reduces the tree to include only the paths from the root which are directly described by the predicate (i.e. a tree equivalent to pruned in the example). If the tree had had a second c-edge, for example, match would have returned false and prune() would have failed with an exception. Note that, If the tree had had no b-edges, or if its b-edges had all ended in leaves, or in fact under any of a number of alternative assumptions, the outcome would have been equally negative.

note: match() does never fail, as a gDoc tree either satisfies the predicate or it does not. In contrast, prune fails whenever the tree does not match the predicate. In other words, prune subsumes match and reacts with a failure to mismatches.

note: Edge predicates are applied in order and each predicate can match any edge that has not been matched by previous predicates.

In the projection above, all the constraints applied to the data of interest, i.e. parts of the trees that were not to be pruned. Often, however, the requirement is to characterise parts of tree while retaining others. For this, some predicates at the edges can be marked as conditions, as shown in the following example:

Date d = new Date();
 
GDoc doc = gdoc(
         e("a",-1),
         e("a",1),
         e("a",2),
         e("b","..."),
         e("b",n(
             e("b1","..."))),
             e("c",n(
                  e("c1",d),
                  e("c2","..."))),
          e("d","..."));
 
Predicate p = tree(
          many("a",num(more(0))),
          cond(atleast("b",tree())),          one("c", tree(
                          one("c1",date()))));
 
GDoc pruned = gdoc(
              e("a",1),
              e("a",2),
              e("c",n(
                   e("c1",d))));
 
assert pruned.equals(p.prune(doc));

Here the predicate includes a condition on b-edges. At prune time, the condition is used to match the tree but it does not imply that matching edges ought to be preserved.

note: condition predicates do not alter the semantics of match(), only prune().

Another common requirement for pruning projections is to preserve all the children of a given node, as long as some children satisfy some constraints. As an example, consider the following document:

GDoc doc = gdoc(
             e("a",n(
                 e("b",-1),
                 e("c","..."),
                 e("d","..."),
                 e("e","..."))),
              e("a",n(
                 e("b","notanumber"),
                 e("c","..."),
                 e("d","..."),
                  e("e","..."))));

One may wish to prune this document so as to retain the a nodes whose b child contains a number. One way to achieve this is to use regular expressions and AnyType to preserve all the children of a nodes that do not need to be explicitly characterised, as shown below:

p = tree(many("a",
           tree(
             one("b",num()),tail()))); 
 
GDoc pruned = gdoc(
             e("a",n(
                 e("b",-1),
                 e("c","..."),
                 e("d","..."),
                 e("e","..."))));
 
p.prune(doc);
 
assertEquals(pruned, doc);

Here, Predicates.tail() is a factory method for the common Edge predicate many(".*",any()), which achieves the desired result.

Finally, pruning projections can include cut predicates that discard all the children of matching inner nodes, which is useful to reduce the amount of data that returned by prune() when the properties of matching nodes are not required for further processing (e.g. when matching nodes need only to be counted). The following example illustrates:

p = tree(many("a", 
           cut(              tree(one("b",num())))));
 
GDoc pruned = gdoc(e("a",n()));
 
p.prune(doc);
 
assertEquals(pruned, doc);

Here, Predicates.cut() is a factory method that generates a cut predicate from a standard predicate on inner nodes.

Serialising and Deserialising Predicates

The predicate and constraint classes are ready for JAXB bindings to XML (i.e. contain appropriate JAXB annotations). In addition, the Predicates class encapsulates a JAXB context and exposes javax.xml.bind.Marshallers and javax.xml.bind.Unmarshallers ready for client use (cf. getMarshaller(), getUnmarshaller).

For example, a client that needs works with character streams may operate as follows:

import static org.gcube.contentmanagement.contentmanager.stubs.model.predicates.Predicates.*;
 
//serialise predicate
Predicate p1 = ...;
Writer w = ...
getMarshaller().marshal(p,w);
 
//deserialise predicate
Reader r =...
Predicate  p2 = (Predicate) getUnmarshaller().unmarshal(r);

note: clients who use the client libraries (such as the Stub Distribution Library and the gCube Document Library) do not explicitly need to worry about conversion to and from DOM representations of predicates. The libraries perform conversions on their behalf.

The following is an example of predicate serialisation (namespaces are omitted for simplicity):

The full schemas of gDoc tree predicates and constraints is available here.

Interfaces

As overviewed above, the interface of the Content Manager is distributed across three stateful port-types:

the Factory port-type, which is the interface of a single WS-Resource;
the ReadManager port-type, which is the read-only interface of many, collection-specific WS-Resources;
the WriteManager port-type, which is the write-only interface of many, collection-specific WS-Resources.

In this Section, we discuss in more detail the operations of the port-types and the Resource Properties of the corresponding WS-Resources. For clarity, we show and comment the signatures of operations in terms of their Java implementations, which mirror the WSDL definitions. We point directly to the WSDL for the precise definition of input and output structures (particularly for values of Resource Properties and records of ResultSets).

note: The service is associated with client-side libraries (such as the Content Manager Library (CML) and the gCube Document Library) (gDL) which operate at a higher level of abstraction than its public interface. The operations discussed below are of interest to clients that cannot afford those dependencies.

Factory

The Factory resource exposes two operations, both which are intended for clients that wish to create collection managers, i.e. ReadManager and/or WriteManager resources. The operations are not for generic use, in that clients are expected to target a specific plugin to which the Factory ought to delegate the creation of the collection managers. The operations, their inputs, and their outputs are defined in the WSDL of the port-type and we present them here informally.

With the first operation, clients block waiting for the creation of the collection managers:

public CollectionReferences create(CMSCreateParameters parameters) throws GCUBEFault

the operation creates managers for one of more collections, depending on the creation parameters, and returns a list of CollectionReference, each of which groups references to the endpoints of the managers created for a given collection. The operation is parameterised by:

the name of the plugin to which the service ought to dispatch the request, the target plugin.
the directive to the service to broadcast the request for state replication, in line with OCMA patterns.
an arbitrary payload of parameters specific to the target plugin. These may vary from plugin to plugin, but typically will contain sufficient information to directly or indirectly identify one or more collections and the repository that hosts them.

The create operation of the Factory resource

The second operation is instead asynchronous, and it is indicated when target plugins declare long creation times in their documentation:

String createAsync(CMSCreateParameters parameters) throws GCUBEFault

the operation takes the same input as its synchronous version but returns immediately with the locator of a ResultSet. This ResultSet will be populated by the Factory only when the creation of the collection managers completes. Specifically, the ResultSet will contain a single CMSCreateOutcome, which captures the outcome of the operation. This can be a success or else a failure. In the first case, clients obtain the CollectionReferences described above. In the second case, they obtain the fault that occurred during the creation of the managers.

The createAsync operation of the Factory resource

Whether synchronous or asynchronous, a request that comes with a broadcasting directive induces the Factory to publish a record of its activation in the system. An activation record is used by other instances of the service as a replication mechanism, i.e. to self-stage as if the request had been made directly against them. In particular, all service instances subscribe with services of the Information System to receive notifications about the creation of activation records, as illustrated in the figure below:

The replication of collection managers across service instances

An example of activation record is the following:

The replication mechanism of the service implies that each instance ought to recognise and ignore the notifications of the activation records that it previously published. This raises a requirement of persistence on the Factory of the instance, which stores the identifiers of its own activation records.

FInally, the Factory resource publishes a multi-valued Resource Property Plugin, which contains a PluginDescription for each of the plugins hosted by the running instance (see WSDL definition).

The Resource Properties of Factory resource

Read Managers

A ReadManager resource exposes read-only operations over the content of the bound collection. The operations are intended for generic use, in that clients are not required to know the repository that hosts the collection or the service plugin that mediates access to that repository. The operations, their inputs, and their outputs are defined in the WSDL of the port-type.

AnyHolder getByID(GetByIDParams params) throws UnknownDocumentFault,GCUBEFault

the operation returns a gDoc tree. Invocations are parameterised by:

the identifier of the tree root;
[optional] a tree predicate with respect to which the tree should be pruned before it is returned.

The getByID operation of ReadManager resources

String getByIDs(GetByIDsParams params) throws GCUBEFault

the operation returns the locator of a ResultSet of gDoc trees. Invocations are parameterised by:

the locator of a ResultSet of tree root identifiers;
[optional] a tree predicate with respect to which the trees should be pruned before they are returned.

note: tree predicates in input and gDoc trees in output are contained in an AnyHolder wrapper (see WSDL definition).

The getByIDs operation of ReadManager resources

String get(GetParams params) throws GCUBEFault

the operation returns the locator of a ResultSet of gDoc trees. Invocations are parameterised by:

[optional] a tree predicate with respect to which the trees should be pruned before they are returned.

note: invocations that specify no predicate effectively ask for the contents of the whole collection.

The get operation of ReadManager resources

AnyHolder getNode(Path path) throws UnknownPathFault,GCUBEFault

the operation returns the node of a gDoc. Invocations are parameterised by:

the path of node identifiers that connects the node to the root of the tree.

note: returned nodes are contained in an AnyHolder wrapper (see WSDL definition).

The getNode operation of ReadManager resources

String getNodes(String path) throws GCUBEFault

the operation returns the locator of a ResultSet of gDoc tree nodes. Invocations are parameterised by:

the locator of a ResultSet of paths of node identifiers that connect the returned nodes to the roots of the corresponding trees.

The getNodes operation of ReadManager resources

A ReadManager resource publishes the following Resource Properties:

CollectionID: the identifier of the bound collection, as per OCMA requirements;
TypeID: the access type of the port-type, as per OCMA requirements. The value of this property is constant: gDocRead;
Plugin: the name of the bound plugin;
Cardinality: an estimate of the number of documents in the bound collection;
LastUpdate: an estimate of the time in which the bound collection was last updated;

The Resource Properties of ReadManager resources

Write Managers

A WriteManager exposes four write-only operations over the content of a bound collection. The operations are intended for generic use, in that clients are not required to know the repository that hosts the collection or the service plugin that mediates access to that repository. The operations, their inputs, and their outputs are defined in the WSDL of the port-type.

String add(AnyHolder holder) throws InvalidDocumentFault, GCUBEFault

the operation adds a gDoc tree to the bound collection and returns the identifier assigned to the tree root as a result.

note: the operation assumes that all nodes of the input tree acquire identifiers as a result of a successful addition. Depending on the plugin that serves the request, the operation may fail if the nodes already have an identifier.

The add operation of WriteManager resources

String addRS(String locator) throws GCUBEFault

the operation adds zero or more gDoc trees to the bound collection and returns a locator of a ResultSet of AddOutcomes, one for each tree in input (see WSDL definition). In the case of successful outcome, clients obtain the identifier assigned to the tree root. In the case of failure, they obtain the stack trace of the fault that prevented the addition of the tree.

note: again, the operation assumes that all nodes of the input tree acquire identifiers as a result of a successful addition. Depending on the plugin that serves the request, the operation may fail if the nodes already have an identifier.

The addRS operation of WriteManager resources

VOID update(AnyHolder holder) throws UnknownDocumentFault, InvalidDocumentFault, GCUBEFault

the operation updates a gDoc tree in the bound collection (the target tree). Invocations are parameterised by delta trees, gDoc trees that capture all the updates to be applied to target trees, including addition or removal of attributes, changes to attribute values, addition and removal of nodes, and changes to leaf values. In more detail, a delta tree is comprised of nodes marked with an attribute http://gcube-system.org/namespaces/contentmanagement/gdoc:status and includes:

all the nodes of the target tree which have changed, with the corresponding identifiers and with a status of MODIFIED or DELETED;

note: attributes of the target tree that have been removed occur in the delta tree with a value of _null;

note: DELETED nodes are either leaves with a value of _null_ or inner nodes with no children.

note: nodes that have no identifiers cannot be referred to in the delta tree, hence cannot be updated.

all the nodes that should be added to the target tree, without identifiers and with a status of NEW.

note: NEW nodes are leaves or inner nodes with NEW descendants.

note: depending on the bound plugin, NEW nodes may be assigned identifiers upon being added to the target tree.

The update operation of WriteManager resources

As an example, consider the following gDoc tree, where status attributes are depicted as node markers:

Here, the delta tree captures a set of updates to the descendants of a gDoc tree:

the root has acquired or changed an attribute x;
node 2 has lost its child 5 and all its descendants;
node 3 has acquired a child and its descendants;
node 4 has lost the attribute z and the child 6, while the value of its child leaf 7 has changed.

note: all the nodes of the target tree that have not changed do not occur in the delta tree.

String updateRS(String locator) throws GCUBEFault

the operation takes the locator of a ResultSet with delta trees for zero or more target gDoc trees and returns the locator of a ResultSet of UpdateFailures (see WSDL definition), one for each delta tree that could not be processed into an update of the corresponding target tree. In particular, clients obtain the identifier of the root of the target tree and the stacktrace of the error that occurred during its update.

note: delta trees in input are contained in AnyHolder wrappers (see WSDL definition).

The updateRS operation of WriteManager resources

A WriteManager resource publishes the following Resource Properties:

CollectionID: the identifier of the bound collection, as per OCMA requirements;
TypeID: the access type of the port-type, as per OCMA requirements. The value of this property is constant: gDocWrite;
Plugin: the name of the bound plugin;
Cardinality: an estimate of the number of documents in the bound collection;
LastUpdate: an estimate of the time in which the bound collection was last updated;

The Resource Properties of WriteManager resources

Difference between revisions of "Content Manager"

Latest revision as of 10:25, 3 October 2011

Contents

Architecture

Content Model

gDoc Trees

gDoc API

Building Trees

Serialising and Deserialising Trees

Consuming Trees

Binding Trees

gDoc Predicates

Predicate API

Building Predicates

Matching and Pruning

Serialising and Deserialising Predicates

Interfaces

Factory

Read Managers

Write Managers

Navigation menu

Views

Personal tools

gCube Wiki

gCube features

gCube documentation

Integration and Distribution

Search

Tools

@@ Line 1: / Line 1: @@
 The Content Manager service gives uniform access to content served by a variety of back-ends, both inside and outside the system. It is the central component of the gCube [[GCube_Information_Organisation_Services_(NEW)|subsystem]] which deals with the organisation of content and related data.
+We overview here the architecture, content model, and interfaces of the service .
+Related libraries, such as the [[Content_Manager_Library|Content Manager Library]] (CML) and the [[gCube_Document_Library|gCube Document Library]] (gDL), are discussed separately.
-= Design Overview =
+= Architecture =
 The Content Manager is designed as an [[OCMA:_The_Open_Content_Management_Architecture|OCMA]] service. In OCMA terms, it classifies as a [[OCMA:_The_Open_Content_Management_Architecture#Service_Classes|''multi-type'', ''1-N adapter'']] service:
@@ Line 13: / Line 16: @@
 :Plugins are dynamically deployed within service instances, and different instances may host different plugins. In addition, some plugins may support both service front types, i.e. grant read and write access to the corresponding class of repositories. Others may instead support read-only access or, less commonly, write-only access.
-The figure below overviews the design and use of the service in the context of one its instances. The instance exposes three stateful port-types:
+The figure below overviews the design and use of the service in the context of one its instances.
+[[Image:CMDesign.jpg|center|Collection Manager Design Overview]]
+The instance exposes three stateful port-types:
 * the ''<code>ReadManager</code>'' port-type serves as the interface of collection managers that offer read-only operations over the content of the bound collection.
-:The interface defines the <code>gDocRead</code> front type of the service.
+:The operations of <code>ReadManager</code> resources and their Resource Properties are discussed [[#Read_Managers|here]].
-: The front type and the identifier of the bound collection are published as Resource Properties of the manager, in accordance with OCMA patterns for publication and discovery of service state. A third Resource Property is the name of the ''bound plugin'', i.e. the plugin to which the manager delegates the resolution of its requests.
 * the ''<code>WriteManager</code>'' port-type serves as the interface of collection managers that offer write-only operations over the content of the bound collection.
-:The interface defines the <code>gDocWrite</code> front type of the service.
+:The operations of <code>WriteManager</code> resources and their Resource Properties are discussed [[#Write_Managers|here]].
-: Again, the type, the identifier of the bound collection, and the name of the bound plugin are published as Resource Properties of the manager.
 * the ''<code>Factory</code>'' port-type serves as the front-end of a single WS Resource that creates <code>ReadManager</code> and <code>WriteManager</code> resources .
-: The resource is created at the activation of the service instance in the gCube Hosting Node.
+:The operations of the <code>Factory</code> resource and its Resource Properties are discussed [[#Factory|here]].
-: During its lifetime, it publishes creation requests as [[OCMA:_The_Open_Content_Management_Architecture#Activation Records|activation records]]. Conversely, it subscribes for the activation records that are published by other instances of the service, in line with OCMA patterns for replication of service state.
-:The resource also publishes as a Resource Property a list of summary descriptions of the plugins which are hosted by the service instance.
+Internally <code>ReadManager</code>s and <code>WriteManager</code>s share common state in a <code>CollectionResource</code>, a stateful resource that acts as an internal model of a remote collection accessed through the collection managers. The  <code>CollectionResource</code> remains private to the service is not published within the system.
-Service plugins logically extend factory and collection manager resources with corresponding ''resource delegates''. In particular:
+All the stateful resources of the service, whether public or private, are logically extended by components provided by service plugins, the ''resource delegates''. The interactions between service and plugins are discussed in detail [[#Plugins|here]], but we can summarise them here as follows:
-* the ''factory delegate'' extends the <code>Factory</code> resource in order to handle requests which are specifically addressed to the plugin;
+* the ''factory delegate'' extends the <code>Factory</code> in order to handle requests which are specifically addressed to the plugin;
-* at each such request, the factory delegate processes plugin-specific parameters to create one ore more ''read delegates'' and/or ''write delegates'', which the service instance uses to create and extend corresponding collection managers;
+* at each such request, the factory delegate processes plugin-specific parameters to create one ore more "collection delegates", one for each collection that should be exposed through the service in order to satisfy the client request.
-* future requests to the managers are then handled by their delegates, which translate the requests against the back-end repository that exposes the collection bound to the managers.
+* the <code>Factory</code> receives such delegates and performs three key tasks:
+:* it initialises the delegates, a process during which each collection delegate may create and stage ''read delegates'' and ''write delegates'', i.e. logical extensions of <code>ReadManager</code>s and <code>WriteManager</code>s.
+:* its creates <code>CollectionResource</code>s, <code>ReadManager</code>s, and <code>WriteManager</code>s to match the available delegates, injecting collection delegates into <code>CollectionResource</code>s so as to form the web of relationships shown in the figure above.
+:* it publishes ''collection profiles'', one for each collection modelled within this process.
+* later requests to <code>ReadManager</code> and <code>WriteManager</code> resources are handled by their read and write delegates, respectively, which translate the requests against the back-end repository that exposes the collection bound to the managers.
+Finally, note that all stateful resources of the service are persistent and may thus be re-activated across restarts of the gCube Hosting Node:
-Finally, note that factory and collection managers are persistent resources and may thus be re-activated across restarts of the gCube Hosting Node:
+* the <code>Factory</code> persists the history of its ''activations'', i.e. the activation records that it published and/or processed (so as to not process them again).
+* the <code>CollectionResource</code> persists its collection delegate and, with it, the read and write delegates.
-* the factory persists the history of its ''activations'', i.e. the activation records that it published and/or processed (so as to not process them again).
+= Content Model =
-* the collection managers persist their delegates.
-[[Image:CMDesign.jpg|center|Collection Manager Design Overview]]
-== Content Model ==
 Architectural considerations aside, the most distinguished element in the design of the Content Manager is its content model. Rather than settle for a fixed set of document structures, the service adopts a generic structure that can act as a 'carrier' for an arbitrary number of concrete document models. In particular, the service deals with edge-labelled and node-attributed trees, the <code>gDoc</code> trees.
-The expectation here is that producers (service plugins) and consumers (service clients) will convene on concrete document models and exchange <code>gDoc</code> trees with an agreed shape. The agreement may be bilateral or involve any number of parties, and it may apply to the entire document or to distinguished parts of it (e.g. document metadata, annotations, raw content packaging, etc). For maximum decoupling between consumers and producers, the agreement may reflect system-wide conventions and result in [[Canonical gDoc Forms|''canonical tree forms'']].
+The expectation here is that producers (service plugins) and consumers (service clients) will convene on concrete document models and exchange <code>gDoc</code> trees with an agreed shape. The agreement may be bilateral or involve any number of parties, and it may apply to the entire document or to distinguished parts of it (e.g. document metadata, annotations, raw content packaging, etc). For maximum decoupling between consumers and producers, the agreement may reflect system-wide conventions and result in ''canonical tree forms'', most noticeably those that define the [[GCube_Document_Model|gCube Document Model]].
-=== gDoc Trees ===
+== gDoc Trees ==
 A <code>gDoc</code> tree has the following properties:
@@ Line 81: / Line 90: @@
 * collection identifiers serialise to attributes called <code>http://gcube-system.org/namespaces/contentmanagement/gdoc:collID</code>
-For example, the <code>gDoc</code> tree above may serialise as:
-<source lang="xml">
+For example, the <code>gDoc</code> tree above serialises as:
-<g:gdoc xmlns:g="http://gcube-system.org/namespaces/contentmanagement/gdoc"
-	g:id="1" x="..." y="..." g:collID="...">
-   <a g:id="2">
+[[Image:Samplegdocxml.jpg|center|A sample <code>gDoc</code> tree XML serialisation]]
-     <b g:id="5"/>
-  </a>
-  <a g:id="3">
-     <c>
-        <d >...</d>
-        <d w="..">...</d>
-     </c>
-  </a>
-   <b g:id="4" w="..." />
-</g:gdoc>
-</source>
-Note that <code>gDoc</code> trees inherit constraints from their XML serialisation. In particular, the names of edges, the names of attributes, the values of attributes, and the values of leaves are regulated by the definition of the format.
+'''note''' <code>gDoc</code> trees inherit constraints from their XML serialisation. In particular, the names of edges, the names of attributes, the values of attributes, and the values of leaves are regulated by the definition of the format.
-=== gDoc API ===
+== gDoc API ==
 The XML serialisation of <code>gDoc</code> trees is 'natural', in that it does not employ dedicated element structures for the representations nodes, edges, attributes, etc. This streamlines its manipulation with standard XMl technologies (e.g. XPath, XSLT, XQuery, DOM, SAX, etc.) and does not inhibit object binding technologies (e.g. JAXB, XStream, etc).
@@ Line 110: / Line 108: @@
 * transparencies and optimisations for tree storage, construction, deconstruction, and input/output.
-While the model is available to service clients, it also forms the basis of the interface between the service and its plugins. For this reason, its main features are overviewed here while its client-oriented features are discussed [[#Stub Distribution|later on]].
+While the model is available to service clients, it also forms the basis of the interface between the service and its plugins. For this reason, its main features are overviewed here while its client-oriented features are discussed [[Content_Manager:_Stub_Distribution#Stub Distribution|elsewhere].
 As the figure below illustrates, the model is defined in <code>org.gcube.contentmanagement.contentmanager.stubs.model.trees</code> in terms of the following components:
@@ Line 127: / Line 125: @@
 [[Image:Treepkg.jpg|center|The <code>...model.tree</code> package]]
 The model API is illustrated by example in the rest of this Section. The full list of methods and their signatures can be found in the code documentation.
-==== Building Trees ====
+=== Building Trees ===
 The first and obvious way to create <code>gDoc</code> trees is with the constructors of the concrete node classes (<code>GDoc</code>, <code>InnerNode</code>, <code>Leaf</code>).
@@ Line 163: / Line 162: @@
 </source>
-For additional convenience, the <code>Nodes</code> class defines a large number of ''generators'', i.e. factory methods that can be statically imported and then composed into a ''pseudo literal syntax'' for <code>gDoc</code> trees:
+For additional convenience, the <code>Nodes</code> class defines a large number of ''generators'', i.e. factory methods that can be statically imported and then composed into an 'embedded expression language' for <code>gDoc</code> trees:
 <source lang="java">
@@ Line 232: / Line 231: @@
 </source>
-'''note''': the node classes override <code>equals</code> for equivalence-based comparisons, and <code>hashCode</code> for correct use as keys within hash-based data structures, and <code>toString</code> for convenience of debugging.
+'''note''': the node classes override <code>equals</code> for equivalence-based comparisons, and <code>hashCode</code> for their correct use as keys within hash-based data structures, and <code>toString</code> for convenience of debugging.
-==== Serialising and Deserialising Trees ====
+=== Serialising and Deserialising Trees ===
 The <code>Bindings</code> class offers static facilities to transform native models of <code>gDoc</code> trees into XML-based models. Two representations are supported natively, based on which other XML-based representation can be produced using standard platform facilities (e.g. TRAX):
@@ Line 271: / Line 270: @@
 '''note''': the conversions are also available at arbitrary inner nodes, not only roots (cf. <code>Bindings.nodeToElement(Node, QName?)</code>, <code>Bindings.nodeFromElement(Element)</code>,<code>Bindings.nodeToXML(Node, Writer, QName)</code>, and <code>Bindings.nodeFromXML(Reader)</code>.
-==== Consuming Trees ====
+=== Consuming Trees ===
 The <code>gDoc</code> API offers simple means of procedural tree navigation. For declarative queries, clients can convert the model into an XML-based representation and leverage platform standards and popular offerings (e.g. XPath, XQuery, or XSLT implementations). If required, the <code>gDoc</code> API can then be reasserted on query outputs.
@@ Line 356: / Line 355: @@
 </source>
-==== Binding Trees ====
+=== Binding Trees ===
 Clients that expect <code>gDoc</code> trees of a given form may wish to bind them to objects. The API offers two classes to streamline JAXB object bindings in the package <code>org.gcube.contentmanagement.contentmanager.stubs.model.views</code>. In particular, it includes two base classes for node and document bindings to XML serialisations of <code>gDoc</code> trees:
@@ Line 422: / Line 421: @@
 </source>
-=== gDoc Predicates ===
+== gDoc Predicates ==
-The <code>gDoc</code> model is untyped, in that neither the topology of trees nor the values of their attributes or leaves are subject to constraints (beside those dictated by the XML serialisation).
+The <code>gDoc</code> model is untyped, in that neither the topology of trees nor the values of their attributes or leaves are subjected to constraints (beside those dictated by the XML serialisation).
-Types are reintroduced later, under the view that they can be ''projected'' on <code>gDoc</code> trees at the point of consumption. Type projections serve two main purposes in the context of the Content Manager:
+Types are reintroduced later, under the view that they can be ''projected'' on <code>gDoc</code> trees at the point of consumption.
+Type projections serve two main purposes in the context of the Content Manager:
 * to validate the content of <code>gDoc</code> trees.
@@ Line 446: / Line 447: @@
 '''note''': Support for predicates on attributes is forthcoming.
-==== Predicate API ====
+=== Predicate API ===
 <code>gDoc</code> predicates are defined in the packages <code>org.gcube.contentmanagement.contentmanager.stubs.model.predicates</code> and <code>org.gcube.contentmanagement.contentmanager.stubs.model.constraints</code>, the main components of which are the following:
@@ Line 459: / Line 460: @@
 *** <code>Date</code>: an <code>LeafPredicate</code> that specifies a date <code>Constraint</code> on the value of leaf nodes.
 *** <code>URI</code>: an <code>LeafPredicate</code> that specifies a URI <code>Constraint</code> on the value of leaf nodes.
-* <code>EdgePredicate</code>: a predicate that specifies a node <code>Predicate</code> on the targets of edges with a given label.
+*** <code>ID</code>: a <code>LeafPredicate</code> on the identifier of an inner node .
-*** <code>One</code>: an <code>EdgePredicate</code> that specifies the existence of a single edge with a given label.
+* <code>EdgePredicate</code>: a predicate that specifies a node <code>Predicate</code> on the targets of edges whose labels match a given label.
-*** <code>Opt</code>: an <code>EdgePredicate</code> that specifies the existence of at most on edge with a given label.
+*** <code>One</code>: an <code>EdgePredicate</code> that asserts the existence of ''exactly one'' edge whose label matches a given label and whose target matches a given predicate.
-*** <code>AtLeast</code>: an <code>EdgePredicate</code> that specifies the existence of one or more edges with a given label.
+*** <code>Opt</code>: an <code>EdgePredicate</code> that asserts the existence of ''zero or one'' edges whose labels match a given label and whose targets match a given predicate.
-*** <code>Many</code>: an <code>EdgePredicate</code> that specifies the existence of zero or more edges with a given label.
+*** <code>AtLeast</code>: an <code>EdgePredicate</code> that asserts the existence of ''one or more'' edges whose labels match a given label and whose targets match a given predicate.
-*** <code>ID</code>: an <code>EdgePredicate</code> that specifies a given identifier for the source.
+*** <code>Many</code>: an <code>EdgePredicate</code> that asserts the existence of ''zero or more'' edges whose labels match a given label and whose targets match a given predicate.
-* <code>Predicates</code>: a set of facilities to build tree predicates.
+*** <code>Only</code>: an <code>EdgePredicate</code> that asserts that ''all'' the edges whose labels match a given label match also a given predicate.
+* <code>Predicates</code>: factory methods for an expression language of tree predicates.
 * <code>Constraint</code>: the interface of all constraints over values of leaf nodes.
@@ Line 477: / Line 479: @@
 ** <code>Either</code>: the <code>Constraint</code> that is satisfied by values that satisfy at least one of a number of other <code>Constraint</code>s.
 ** <code>All</code>: the <code>Constraint</code> that is satisfied by values that satisfy a number of other <code>Constraint</code>s.
-* <code>Constraints</code>: a set of facilities to build <code>Constraint</code>s.
+* <code>Constraints</code>: factory methods for an expression language of <code>Constraint</code>s.
 [[Image:PredicatePackage.jpg|center|Predicate and Constraints packages]]
-==== Building Predicates ====
-Similarly to <code>gDoc</code> trees, <code>gDoc</code> predicates may be built with classic constructor-based idioms and/or else with ''predicate generators'', a collection of factory methods in <code>Predicates</code> class and <code>Constraints</code> class which can be statically imported and then composed into a pseudo literal syntax for gDoc predicates. We concentrate here on predicate generators as the preferred way to build <code>gDoc</code> predicates. See the code documentation for the constructors available in predicate and constraint classes.
+=== Building Predicates ===
+Similarly to <code>gDoc</code> trees, <code>gDoc</code> predicates may be built with classic constructor-based idioms and/or else with ''predicate generators'', a collection of factory methods in the <code>Predicates</code> class and <code>Constraints</code> classes which can be statically imported and then composed into a pseudo expression language for <code>gDoc</code> predicates. We concentrate here on predicate generators as the preferred way to build <code>gDoc</code> predicates. See the code documentation for the constructors available in predicate and constraint classes.
 Consider this first example:
@@ Line 506: / Line 509: @@
 Predicate p = tree(
-                  one("a",any),
+                one("a",any()),
-                  one("b",text(either(is("abc"),is("efg")))),
+                one("b",text(either(is("abc"),is("efg")))),
-                  atleast("c",bool(is(true))),
+                atleast("c",bool(is(true))),
-                  opt("d",tree()),
+                opt("d",tree()),
-                  many("e",date(future())),,
+                many("e",date(future())),
-                  one("f", uri(matches("^http.*"))),
+                one("f", uri(matches("^http.*"))),
-                  many("g", num(all(less(5),more(10))),
+                many("g", num(all(less(5),more(10)))),
-                  one("h", text());
+                one("h", text()),
-                  one("j",text(not(is("somestring")))),
+                one("j",text(not(is("somestring")))),
-                  one("k",tree(id("12345"))));
+                one("k",id("12345",tree())),
+                only("l", num()));
 </source>
 Here, the predicate characterises trees with:
-* a single <code>a</code>-edge that ends in any type of node, inner node or leaf, not characterised further;
+* a single <code>a</code>-edge that ends in any type of node, inner node or leaf. (<code>any()</code> is a generator of <code>AnyPredicate</code>s);
-* one or more <code>b</code>-edges that end in leaves whose values are either one of two strings;
+* a single <code>b</code>-edge that ends in a leaf whose value is either one of two strings;
-* a single <code>c</code>-edge that ends in a leaf with a boolean value of <code>true</code>;
+* one or more <code>c</code>-edges that end in leaves with a boolean value of <code>true</code>;
-* zero or one <code>d</code>-edges that end in inner nodes, not characterised further;
+* zero or one <code>d</code>-edges that end in inner nodes;
 * zero or more <code>e</code>-edges that end in leaves whose values are dates in the future;
 * a single <code>f</code>-edge that ends in a leaf whose value is an absolute http URI;
@@ Line 530: / Line 534: @@
 * a single <code>j</code>-edge that ends in a leaf whose value differs from a given string;
 * a single <code>k</code>-edge that ends in an an inner node with an identifier of <code>12345</code>;
+* zero or more <code>l</code>-edge that ''all'' end in leaves with numeric values;
-Clearly, predicates can nest recursively to match the structure of trees. In addition, all predicates refer to labels by plain strings, yet these can be qualified names where both namespace and local parts can be matched by regular expressions. For example, the following predicate:
+'''note''': predicates can nest recursively to match the structure of trees.
+'''note''': the <code>Edge</code>-predicates above use plain strings to match edge labels, but they may more generally use qualified names with regular expressions on both namespace and local parts. For example, the following predicate:
 <source lang="java5">
@@ Line 539: / Line 546: @@
 </source>
-characterise trees that have one or more edges whose labels begin with <code>part</code> and whose targets contain edges with no more than one edge with in a namespace that ends with <code>acme.org</code> and  with a numeric value.
+:characterises trees that have one or more edges whose labels begin with <code>part</code> and whose targets contain edges with no more than one edge with in a namespace that ends with <code>acme.org</code> and  with a numeric value.
 For a full list of available predicate and constraint generators, see the code documentation of the <code>Predicates</code> and <code>Constraints</code> classes.
-==== Matching and Pruning ====
+=== Matching and Pruning ===
-A <code>gDoc</code> predicate can be projected over a <code>gDoc</code> tree using the methods <code>match()</code> and <code>prune()</code> common to all <code>Predicate</code>s. The first indicates whether tree satisfies the predicate, the second prunes it of all the nodes that remain uncharacterised by the predicate.
+A <code>gDoc</code> predicate can be projected over a <code>gDoc</code> tree using the methods <code>match()</code> and <code>prune()</code> common to all <code>Predicate</code>s. The first indicates whether tree satisfies the predicate, the second prunes it of all the nodes that are not matched by the predicate.
 Consider this simple example:
@@ Line 590: / Line 597: @@
 '''note''': <code>match()</code> does never fail, as a <code>gDoc</code> tree either satisfies the predicate or it does not. In contrast, <code>prune</code> fails whenever the tree does not match the predicate. In other words, <code>prune</code> subsumes <code>match</code> and reacts with a failure to mismatches.
-==== Serialising and Deserialising Predicates ====
+'''note''': <code>Edge</code> predicates are applied in order and each predicate can match any edge that has not been matched by previous predicates.
-The predicate and constraint classes are ready for JAXB binding to XML (i.e. contain appropriate JAXB annotations). In addition, the <code>Predicates</code> class encapsulates a JAXB context and exposes <code>javax.xml.bind.Marshaller</code>s and <code>javax.xml.bind.Unmarshaller</code>s ready for client use (cf. <code>getMarshaller()</code>, <code>getUnmarshaller</code>).
+In the projection above, all the constraints applied to the data of interest, i.e. parts of the trees that were ''not'' to be pruned. Often, however, the requirement is to characterise parts of tree while retaining others. For this, some predicates at the edges can be marked as ''conditions'', as shown in the following example:
-For example, a client that needs works with character streams may operate as follows:
+<source lang="java5" highlight="17">
+Date d = new Date();
-<source lang="java5">
+GDoc doc = gdoc(
-import static org.gcube.contentmanagement.contentmanager.stubs.model.predicates.Predicates.*;
+         e("a",-1),
+         e("a",1),
-//serialise predicate
+         e("a",2),
-Predicate p1 = ...;
+         e("b","..."),
-Writer w = ...
+         e("b",n(
-getMarshaller().marshal(p,w);
+             e("b1","..."))),
+             e("c",n(
-//deserialise predicate
+                  e("c1",d),
-Reader r =...
+                  e("c2","..."))),
-Predicate  p2 = (Predicate) getUnmarshaller().unmarshal(r);
+          e("d","..."));
-</source>
-'''note''': clients who use the Content Management [[#Client_Libraries|libraries]] do not explicitly need to worry about conversion to and from DOM representations of predicates. The libraries perform conversions on their behalf.
-== Interfaces ==
-As overviewed [[#Service_Design|above]], the interface of the Content Manager is distributed across three stateful port-types:
-* the <code>Factory</code> port-type, which is the interface of a single WS-Resource;
-* the <code>ReadManager</code> port-type, which is the read-only interface of many, collection-specific WS-Resources;
-* the <code>WriteManager</code> port-type, which is the write-only interface of many, collection-specific WS-Resources.
-In this Section, we discuss in more detail the operations of the port-types and the Resource Properties of the corresponding WS-Resources. For clarity, we show and comment the signatures of operations in terms of the underlying Java implementation, which mirror the WSDL definitions. We point directly to the WSDL for the definition of auxiliary data structures for input and outputs (e.g. values of Resource Properties or records of ResultSets).
-'''note''': The service offers [[#Client_Libraries|client-side libraries]] which operate at a higher level of abstraction than its public interface. The operations discussed below are of interest to clients that choose to bypass those facilities.
-=== Factory ===
-The <code>Factory</code> resource exposes two operations, both which are intended for clients that wish to create Collection Managers (<code>ReadManager</code>s and/or <code>WriteManager</code>s). The operations are ''not'' for generic use, in that clients are expected to target specifically the plugin to which the <code>Factory</code> resource ought to delegate the creation of Collection Managers. The operations, their inputs, and their outputs are defined in the [[Content Manager's Factory PortType WSDL|WSDL]] of the port-type.
-The first operation is synchronous, in that clients block waiting for the creation of the Collection Managers:
-* <code>public EprList create(CMSCreateParameters parameters) throws GCUBEFault</code>
-: the operation creates Collection Managers and returns a list of references to their endpoints, indexed by the name of the corresponding port-type. The creation of resource is paremetrised by:
-:*  the name of the plugin to which the service ought to dispatch the request, the ''target plugin''.
-:*  the directive to the service to broadcast the request for state replication, in line with [[OCMA:_The_Open_Content_Management_Architecture#Activation_Records| OCMA patterns]].
-:*  an arbitrary DOM payload of parameters specific to the target plugin.
-: these may vary from plugin to plugin, but typically will contain sufficient information to directly or indirectly identify one or more collections and the repository that hosts them.
-The second operation is instead asynchronous and it is indicated when target plugins declare long creation times in their documentation:
-* <code>String createAsync(CMSCreateParameters parameters) throws GCUBEFault</code>
-: the operation takes the same input as the synchronous version but returns immediately with the locator of a [[GCube_ResultSet_(gRS)| ResultSet]] which will be populated by the service only when the Collection Managers are created by the target plugin. Specifically, the ResultSet will contain a single <code>CMSCreateOutcome</code>, which captures the outcome of the operation (see WSDL definition). This can be a success or else a failure. In the first case, clients obtain the <code>EprList</code> described above. In the second case, they obtain the stack trace of the fault that occurred during the creation of the Collection Managers.
-FInally, the <code>Factory</code> resource publishes a multi-valued Resource Property <code>Plugin</code>, which contains a <code>PluginDescription</code> for each of the plugins hosted by the running instance (see WSDL definition).
-=== Read Managers ===
-A <code>ReadManager</code> resource exposes three read-only operations over the content of the bound collection. The operations are intended for generic use, in that clients are not required to know the repository that hosts the collection or the service plugin that mediates access to that repository. The operations, their inputs, and their outputs are defined in the [[Content Manager's ReadManager PortType WSDL|WSDL]] of the port-type.
-* <code>AnyHolder getByID(GetByIDParams params) throws GCUBEFault</code>
-: the operation returns (the DOM representation of) a <code>gDoc</code> tree. Invocations are parameterised by:
-:* the identifier of the tree root;
-:* [optional] a tree predicate with respect to which the tree should be pruned before it is returned.
-* <code>String getByIDs(GetByIDsParams params) throws GCUBEFault</code>
-: the operation returns the locator of a ResultSet of <code>gDoc</code> trees. Invocations are parameterised by:
-:* the locator of a ResultSet of (DOM representations of) tree root identifiers;
-:* [optional] a tree predicate with respect to which the trees should be pruned before they are returned.
-:'''note''': tree predicates in input and <code>gDoc</code> trees in output are contained in an <code>AnyHolder</code> wrapper (see WSDL definition).
-* <code>String get(GetParams params) throws GCUBEFault</code>
-: the operation returns the locator of a ResultSet of <code>gDoc</code> trees. Invocations are parameterised by:
-:* [optional] a tree predicate with respect to which the trees should be pruned before they are returned.
-:* [optional] a tree predicate with respect to which the trees should be filtered before they are returned.
-:Filtering occurs before pruning and may be concern parts of the trees that are subsequently pruned.
-: '''note''': invocations that specify no parameters effectively ask for the contents of the whole collection.
-A <code>ReadManager</code> resource publishes the following Resource Properties:
-* <code>CollectionID</code>: the identifier of the bound collection, as per OCMA requirements;
-* <code>TypeID</code>: the access type of the port-type, as per OCMA requirements. The value of this property is constant: <code>gDocRead</code>;
-* <code>Plugin</code>: the name of the bound plugin;
-=== Write Managers ===
-A <code>WriteManager</code> exposes four write-only operations over the content of a bound collection. The operations are intended for generic use, in that clients are not required to know the repository that hosts the collection or the service plugin that mediates access to that repository. The operations, their inputs, and their outputs are defined in the [[Content Manager's WriteManager PortType WSDL|WSDL]] of the port-type.
-* <code>String add(AnyHolder holder) throws InvalidDocumentFault, GCUBEFault</code>
-: the operation adds a <code>gDoc</code> tree to the bound collection and returns the identifier assigned to the tree root as a result.
-: '''note''': the operation assumes that all nodes of the input tree acquire identifiers as a result of a successful addition. Depending on the plugin that serves the request, the operation may fail if the nodes already have an identifier.
-* <code>String addRS(String locator) throws GCUBEFault</code>
-: the operation adds zero or more <code>gDoc</code> trees to the bound collection and returns a locator of a ResultSet of <code>AddOutcome</code>s, one for each tree in input (see WSDL definition). In the case of successful outcome, clients obtain the identifier assigned to the tree root. In the case of failure, they obtain the stack trace of the fault that prevented the addition of the tree.
-: '''note''': again, the operation assumes that all nodes of the input tree acquire identifiers as a result of a successful addition. Depending on the plugin that serves the request, the operation may fail if the nodes already have an identifier.
-* <code>VOID update(AnyHolder holder) throws UnknownDocumentFault, InvalidDocumentFault, GCUBEFault</code>
-: the operation updates a <code>gDoc</code> tree in the bound collection (the ''target tree''). Invocations are parameterised by ''delta trees'', <code>gDoc</code> trees that capture all the updates to be applied to target trees, including addition or removal of attributes, changes to attribute values, addition and removal of nodes, and changes to leaf values. In more detail, a delta tree is comprised of nodes marked with an attribute <code>http://gcube-system.org/namespaces/contentmanagement/gdoc:status</code> and includes:
-:* all the nodes of the target tree which have changed, with the corresponding identifiers and with a <code>status</code> of <code>MODIFIED</code> or <code>DELETED</code>;
-:: '''note''': attributes of the target tree that have been removed occur in the delta tree with a value of <code>_null</code>;
-:: '''note''': <code>DELETED</code> nodes are either leaves with a value of <code>_null_</code> or inner nodes with no children.
-:: '''note''': nodes that have no identifiers cannot be referred to in the delta tree, hence cannot be updated.
-:* all the nodes that should be added to the target tree, without identifiers and with a <code>status</code> of <code>NEW</code>.
-:: '''note''': <code>NEW</code> nodes are leaves or inner nodes with <code>NEW</code> descendants.
-:: '''note''': depending on the bound plugin, <code>NEW</code> nodes may be assigned identifiers upon being added to the target tree.
-:As an example, consider the following <code>gDoc</code> tree, where <code>status</code> attributes are depicted as node markers:
-[[Image:DeltaTree.jpg| center | A Delta Tree]]
-:Here, the delta tree captures a set of updates to the descendants of a <code>gDoc</code> tree:
-:* the root has acquired or changed an attribute <code>x</code>;
-:* node <code>2</code> has lost its child <code>5</code> and all its descendants;
-:* node <code>3</code> has acquired a child and its descendants;
-:* node <code>4</code> has lost the attribute <code>z</code> and the child <code>6</code>, while the value of its child leaf <code>7</code> has changed.
-: '''note''': all the nodes of the target tree that have not changed do not occur in the delta tree.
-* <code>String updateRS(String locator) throws GCUBEFault</code>
-: the operation takes the locator of a ResultSet with delta trees for zero or more target <code>gDoc</code> trees and returns the locator of a ResultSet of <code>UpdateFailure</code>s (see WSDL definition), one for each delta tree that could not be processed into an update of the corresponding target tree. In particular, clients obtain the identifier of the root of the target tree and the stacktrace of the error that occurred during its update.
-:'''note''': delta trees in input are contained in <code>AnyHolder</code> wrappers (see WSDL definition).
-A <code>WriteManager</code> resource publishes the following Resource Properties:
-* <code>CollectionID</code>: the identifier of the bound collection, as per OCMA requirements;
-* <code>TypeID</code>: the access type of the port-type, as per OCMA requirements. The value of this property is constant: <code>gDocWrite</code>;
-* <code>Plugin</code>: the name of the bound plugin;
-== Plugins ==
-...coming soon...
-= Client Libraries =
-The Content Manager service is developed alongside a number of client libraries that simplify the task of interacting with the service.
-The [[#Stub_Distribution|Stub Distribution]] of the service is the first and foremost client library. Its design goal is to offer abstractions over the content model and interface of the service. In this sense, the distribution acts both as a client library and and a service-side library for plugin developments. In particular, it is a dependency of the service as well as a dependency of service clients.
-The [[#Content_Management_Library|Content Management Library]] (CML) builds upon the stub distribution to support the ''gCube document model'', a concrete document model made of [[Canonical_gDoc_Forms|canonical forms]] for metadata, annotations, document parts, and alternative representations.
-== Stub Distribution ==
-The distribution includes:
-* the API for <code>gDoc</code> trees;
-* the API for <code>gDoc</code> tree predicates;
-* the stubs of the service automatically generated from the WSDL definition of its port-types;
-* the ''high-level calls'', a set of abstractions over the service stubs;
-* a Java protocol handler and associated facilities for deriving and resolving ''content URIs'', i.e. resolvable URIs to arbitrary nodes of <code>gDoc</code> trees.
-[[Image:StubDistro.jpg|The Stub Distribution]]
-We have [[#Content_Model|previously]] presented most of the APIs for <code>gDoc</code> trees and tree predicates. We concentrate here on high-level calls and content URIs, completing the presentation of the tree and tree predicate APIs in the process.
-=== High-Level Calls ===
-High-level calls are Java objects that model single-step or multi-step interactions with the Content Management service.
-The objects encapsulate stub-based interactions behind local object-oriented interfaces that offer transparencies over the  [[#Interfaces|remote interfaces]] of the service port-types.
-The local interfaces are based on language features that are not found in the service stubs, including high-level models of inputs and outputs, method overloading, parametric types, asynchronous callbacks.
-Behind these abstractions, the call objects engage in optimised and best-effort interactions with the WS-Resources of the services; in particular, they can hide from clients the complexity of resource discovery while keeping visible the remote nature of the interactions and the possibility of their failure.
-High-level calls are defined in the package <code>org.gcube.contentmanagement.contentmanager.stubs.calls</code>. The main components are depicted below:
-[[Image:HLCalls.jpg|High-Level Calls]]
-* <code>BaseCall</code>: the base class for all high-level calls.
-* <code>FactoryCall</code>: a <code>BaseCall</code> that represents calls to the code>Factory</code> resource of the service.
-* <code>FactoryParams</code>: used in <code>FactoryCall</code> to model the input of operations to the code>Factory</code> resource of the service.
-* <code>FactoryConsumer</code>: used in <code>FactoryCall</code> to callback invokers of the asynchronous operation of the code>Factory</code> resource of the service.
-* <code>ManagerCall</code>: an abstract extension of <code>BaseCall</code> for calls to the Collection Managers of the service.
-* <code>ReadManagerCall</code>: a <code>ManagerCall</code> that represents calls to <core>ReadManager</code> resources of the service.
-* <code>WriteManagerCall</code>: a <code>ManagerCall</code> that represents calls to <core>WriteManager</code> resources of the service.
-* <code>MappingRegistry</code>: a central registry of type mappings for I/O.
-* <code>Constants</code>: a collection of service-specific constants.
-* <code>Utils</code>: a collection of utilities for I/O conversions.
-In what follows, we exemplify the use of <code>FactoryCall</code>s, <code>ReadManagerCall</code>s, and <code>WriteManagerCalls</code>.
-==== Factory Calls ====
+Predicate p = tree(
+          many("a",num(more(0))),
+          cond(atleast("b",tree())),
+          one("c", tree(
+                          one("c1",date()))));
-A <code>FactoryCall</code> is created in a a scope:
+GDoc pruned = gdoc(
+              e("a",1),
+              e("a",2),
+              e("c",n(
+                   e("c1",d))));
-<source lang="java5" highlight="4">
+assert pruned.equals(p.prune(doc));
-//some scope
-GCUBEScope scope = .....
-FactoryCall call = new FactoryCall(scope);
 </source>
-In a secure infrastructure, the call may also be created with a security manager:
+Here the predicate includes a condition on <code>b</code>-edges. At prune time, the condition is used to match the tree but it does not imply that matching edges ought to be preserved.
-<source lang="java5" highlight="7">
+'''note''': condition predicates do not alter the semantics of <code>match()</code>, only <code>prune()</code>.
-//some scope
-GCUBEScope scope = .....
-//some security manager = ....
+Another common requirement for pruning projections is to preserve all the children of a given node, as  long as some children satisfy some constraints. As an example, consider the following document:
-GCUBESecurityManager manager = ....
-FactoryCall call = new FactoryCall(scope,manager);
+<source lang="java5">
+GDoc doc = gdoc(
+             e("a",n(
+                 e("b",-1),
+                 e("c","..."),
+                 e("d","..."),
+                 e("e","..."))),
+              e("a",n(
+                 e("b","notanumber"),
+                 e("c","..."),
+                 e("d","..."),
+                  e("e","..."))));
 </source>
-The call may then be issued, i.e. used to create CollectionManagers. In line with the operations of the remote port-type, this can be done synchronously or asynchronously.
+One may wish to prune this document so as to retain the <code>a</code> nodes whose <code>b</code> child contains a number.
-The synchronous invocation requires the preparation of <code>FactoryParameters</code>;
+One way to achieve this is to use regular expressions and <code>AnyType</code> to preserve all the children of <code>a</code> nodes that do not need to be explicitly characterised, as shown below:
-<source lang="java5" highlight="11">
+<source lang="java5" highlight="3">
-FactoryParameters params = new FactoryParameters() ;
+p = tree(many("a",
-params.setPlugin("..somepluginname...");
+           tree(
-params.setBroadcast(false);
+             one("b",num()),tail())));
-//the DOM serialisation of plugin-specific creation parameters
-org.w3c.dom.Element payload = ...
-params.setPayload(payload)
+GDoc pruned = gdoc(
+             e("a",n(
+                 e("b",-1),
+                 e("c","..."),
+                 e("d","..."),
+                 e("e","..."))));
+p.prune(doc);
-//issue the call
+assertEquals(pruned, doc);
-List<EprPair> eprs =  call.create(params);
-//process the response
-for (EprPair pair : eprs)
-   .... pair.getPorttype() ... pair.getEpr() ...
 </source>
-{|style="border-collapse: separate; border-spacing: 10; border-width: 2px; border-style: solid;"
+Here, <code>Predicates.tail()</code> is a factory method for the common <code>Edge</code> predicate <code>many(".*",any())</code>, which achieves the desired result.
-|
-'''note''': typically, plugin will offer object bindings for the payloads that they support. The payload input to the <code>create()</code> method will then be obtained by serialising the bound objects.
-|}
-The asynchronous invocation requires the additional preparation of a <code>FactoryConsumer</code>:
+Finally, pruning projections can include ''cut predicates'' that discard ''all'' the children of matching inner nodes, which is useful to reduce the amount of data that returned by <code>prune()</code> when the properties of matching nodes are not required for further processing (e.g. when matching nodes need only to be counted).  The following example illustrates:
-<source lang="java5" highlight="17">
+<source lang="java5" highlight="2">
-//prepare as above
+p = tree(many("a",
-FactoryParameters params = .....
+           cut(
+              tree(one("b",num())))));
-//creates consumer
+GDoc pruned = gdoc(e("a",n()));
-FactoryConsumer consumer = new FactoryConsumer {
+p.prune(doc);
-     protected void onCompletion(List<EprPair> eprs) {
-             .... process pairs as above
-     };
-     protected void onFailure(Exception e) {
-             ... handle failure
-     };
-};
-//issue the call
+assertEquals(pruned, doc);
-call.createASync(params,consumer);
 </source>
-In both interactions above, the <code>FactoryCall</code> will attempt to discover <code>Factory</code> WS-Resources that host the plugin named in the parameters. It will then try to interact with each resource in turn, until one responds successfully or else indicates that continuing will be to no avail (by returning a <code>GCUBEUnretrievableFault</code>).
+Here, <code>Predicates.cut()</code> is a factory method that generates a cut predicate from a standard predicate on inner nodes.
-{|style="border-collapse: separate; border-spacing: 10; border-width: 2px; border-style: solid;"
+=== Serialising and Deserialising Predicates ===
-|
-'''note''': clients can obtain and customise the query that underlies the strategy (cf. <code>getQuery()</code>) and, if needed, reset it to its default (<code>resetQuery()</code>).
-'''note''': while call objects are often created anew for individual calls to the remote port-type, clients can use the same object for multiple calls (though this is unlikely for <code>FactoryCall</code>s). When this is the case, the calls occur in the same, initially configured scope and the second call 'sticks' to the resource used by the first. The best-effort strategy is intentionally limited to the first invocation only.
+The predicate and constraint classes are ready for JAXB bindings to XML (i.e. contain appropriate JAXB annotations). In addition, the <code>Predicates</code> class encapsulates a JAXB context and exposes <code>javax.xml.bind.Marshaller</code>s and <code>javax.xml.bind.Unmarshaller</code>s ready for client use (cf. <code>getMarshaller()</code>, <code>getUnmarshaller</code>).
-|}
-Clients who know and wish to target a specific <code>Factory</code> resource, can disable the best-effort strategy by configuring the call with a reference to its endpoint:
+For example, a client that needs works with character streams may operate as follows:
-<source lang="java5" highlight="4">
+<source lang="java5">
-//a reference to the endpoint of a Content Manager RI
+import static org.gcube.contentmanagement.contentmanager.stubs.model.predicates.Predicates.*;
-EndpointReferenceType epr = ...
-call.setEndpointReferenceType(epr);
+//serialise predicate
+Predicate p1 = ...;
+Writer w = ...
+getMarshaller().marshal(p,w);
-//alternatively:
+//deserialise predicate
-call.setEndpoint("... somehostname ...",".. someport ..");
+Reader r =...
+Predicate  p2 = (Predicate) getUnmarshaller().unmarshal(r);
 </source>
-==== ReadManager Calls ====
+'''note''': clients who use the client libraries (such as the [[Content_Manager:_Stub_Distribution|Stub Distribution Library]] and the [[gCube_Document_Library|gCube Document Library]]) do not explicitly need to worry about conversion to and from DOM representations of predicates. The libraries perform conversions on their behalf.
-A <code>ReadManagerCall</code> gives high-level write access to the content of a given collection, as allowed by a <code>ReadManager</code> resource bound to that collection.
+The following is an example of predicate serialisation (namespaces are omitted for simplicity):
-It follows the same patterns already illustrated for <code>FactoryCall</code>s. In particular, it is created in a scope and, optionally, with a security manager.
-<source lang="java5" highlight="4,9">
-//some scope
-GCUBEScope scope = .....
-ReadManagerCall call = new ReadManagerCall(scope);
+[[Image:PredicateXML.jpg|center|Sample Predicate XML Representation]]
-//some security manager = ....
-GCUBESecurityManager manager = ....
-ReadManagerCall secureCall = new ReadManagerCall(scope,manager);
+The full schemas of <code>gDoc</code> tree predicates and constraints is available [[gDoc Predicate Schemas|here]].
-</source>
-As a further option, it may be crated with the identifier of the target collection:
+= Interfaces =
-<source lang="java5" highlight="4,9">
+As overviewed [[#Architecture|above]], the interface of the Content Manager is distributed across three stateful port-types:
-//some scope
-GCUBEScope scope = .....
-ReadManagerCall call = new ReadManagerCall("... some collection identifier ...",scope);
+* the <code>Factory</code> port-type, which is the interface of a single WS-Resource;
+* the <code>ReadManager</code> port-type, which is the read-only interface of many, collection-specific WS-Resources;
+* the <code>WriteManager</code> port-type, which is the write-only interface of many, collection-specific WS-Resources.
-//some security manager = ....
-GCUBESecurityManager manager = ....
-ReadManagerCall secureCall = new ReadManagerCall("... some collection identifier ...",scope,manager);
+[[Image:InterfaceOverview.jpg|center|CM Port-Types Overview]]
-</source>
-{|style="border-collapse: separate; border-spacing: 10; border-width: 2px; border-style: solid;"
-|
-'''note''': the collection identifier may also be set after call construction (cf. <code>setCollectionID(String)</code>).
-|}
-The call object may be configured as a <code>FactoryCall</code>, i.e. setting reference to resource endpoint for targeted interactions (cf. <code>setEndpointReference(EndpointReference)</code>), or else relying on implicit discovery and best-effort strategy. In the latter case, the query that underlie the strategy can be customised and reset (cf. <code>getQuery()</code>,<code>resetQuery()</code>).
+In this Section, we discuss in more detail the operations of the port-types and the Resource Properties of the corresponding WS-Resources. For clarity, we show and comment the signatures of operations in terms of their Java implementations, which mirror the WSDL definitions. We point directly to the WSDL for the precise definition of input and output structures (particularly for values of Resource Properties and records of ResultSets).
-The call object may then be used to retrieve <code>gDoc</code> trees from the target collection. To this end, its operations may be classified in two groups: the those that return single trees and those that return multiple trees. The first class includes lookup operations while the second class includes both lookup and query operations based on tree predicates. Multi-valued operations are execute asynchronously at the service, based on the [[GCube_ResultSet_(gRS)|ResultSet]] mechanism.
+'''note''': The service is associated with client-side libraries (such as the [[Content_Manager_Library|Content Manager Library]] (CML) and the [[gCube_Document_Library|gCube Document Library]]) (gDL) which operate at a higher level of abstraction than its public interface. The operations discussed below are of interest to clients that cannot afford those dependencies.
-The following example illustrates the use single-valued lookups:
+== Factory ==
-<source lang="java5" highlight="2,8">
+The <code>Factory</code> resource exposes two operations, both which are intended for clients that wish to create collection managers, i.e. <code>ReadManager</code> and/or <code>WriteManager</code> resources. The operations are ''not'' for generic use, in that clients are expected to target a specific plugin to which the <code>Factory</code> ought to delegate the creation of the collection managers. The operations, their inputs, and their outputs are defined in the [[Content Manager's Factory PortType WSDL|WSDL]] of the port-type and we present them here informally.
-//synchronous: return one gDoc tree
-GDoc doc1 = call.get("... tree root identifier ...");
-//some tree predicate to use for pruning
+With the first operation, clients block waiting for the creation of the collection managers:
-Predicate projection = ....
-//synchronous: prune and return one gDoc tree
+* <code>public CollectionReferences create(CMSCreateParameters parameters) throws GCUBEFault</code>
-GDoc doc2 = call.get("... tree root identifier ...",projection);
+: the operation creates managers for one of more collections, depending on the creation parameters,  and returns a list of <code>CollectionReference</code>, each of which groups references to the endpoints of the managers created for a given collection. The operation is parameterised by:
-</source>
+:*  the name of the plugin to which the service ought to dispatch the request, the ''target plugin''.
+:*  the directive to the service to broadcast the request for state replication, in line with [[OCMA:_The_Open_Content_Management_Architecture#Activation_Records| OCMA patterns]].
+:*  an arbitrary payload of parameters specific to the target plugin. These may vary from plugin to plugin, but typically will contain sufficient information to directly or indirectly identify one or more collections and the repository that hosts them.
-Here, <code>get(String)</code> and <code>get(String,Predicate)</code> bind the output tree to the object model of <code>gDoc</code> [[#gDoc_API|tree API]].
-We note that there are semantically equivalent operations that return DOM bindings, so as to raise no further parsing costs if a binding other than to the <code>gDoc</code> tree API is required upstream (cf. <code>getAsElement(String)</code> and <code>getAsElement(String,Predicate)</code>).
+[[Image:createop.jpg|center|The <code>create</code> operation of the Factory resource]]
-Muti-valued lookups may be exemplified as follows:
-<source lang="java5" highlight="5,8">
+The second operation is instead asynchronous, and it is indicated when target plugins declare long creation times in their documentation:
-//a locator to a ResultSet of tree root identifiers, produced using standard ResultSet production idioms
-RSLocator identifiers = ....
-//asynchronous: returns a locator to a remote ResultSet of gDoc trees with given identifiers
+* <code>String createAsync(CMSCreateParameters parameters) throws GCUBEFault</code>
-RSLocator locator1 = call.get(identifiers);
+: the operation takes the same input as its synchronous version but returns immediately with the locator of a [[GCube_ResultSet_(gRS)| ResultSet]]. This ResultSet will be populated by the <code>Factory</code/> only when the creation of the collection managers completes. Specifically, the ResultSet will contain a single <code>CMSCreateOutcome</code>, which captures the outcome of the operation. This can be a success or else a failure. In the first case, clients obtain the <code>CollectionReferences</code> described above. In the second case, they obtain the fault that occurred during the creation of the managers.
-//asynchronous: returns a locator to a remote ResultSet of gDoc trees with given identifiers, pruned by a tree predicate
-RSLocator locator2 = call.get(identifiers,predicate);
-</source>
-The ResultSets returned by the lookups contain XML representations of <code>gDoc</code> trees. Standard ResultSet consumption idioms may then be used to extract the XML representations and bind them to object models of choice. The distribution supports more transparent idioms, however:
-<source lang="java5" highlight="4,7">
+[[Image:createasyncop.jpg|center|The <code>createAsync</code> operation of the Factory resource]]
-//a locator to a ResultSet of gDoc trees.
-RSLocator locator = ....
-GDocRSCollection docs = new GDocRSCollection(locator);
-//use standard
-for (GDoc doc : doc)
-  ...process document...
-</source>
+Whether synchronous or asynchronous, a request that comes with a broadcasting directive induces the <code>Factory</code> to publish a record of its activation in the system.
+An [[OCMA:_The_Open_Content_Management_Architecture#Activation_Records|activation record]] is used by other instances of the service as a replication mechanism, i.e. to self-stage as if the request had been made directly against them. In particular, all service instances subscribe with services of the Information System to receive notifications about the creation of activation records, as illustrated in the figure below:
-Here, <code>GDocRSCollection</code> is a collection of <code>gDoc</code> trees which is backed by the ResultSet identified by the locator. The collection is 'lazily' assembled, in that it does not allow direct access to its elements, but can only be iterated over with standard language idioms, as shown. The iteration subsumes XML bindings to the native object model and hides binding failures in the process.  Clients that wish to process failures can do so asynchronously with respect to the iteration, by previously registering a <code>FaultListener</code> with the collection (e.g. at construction time):
-<source lang="java5" highlight="4-11">
+[[Image:replication.jpg|center|The replication of collection managers across service instances]]
-//a locator to a ResultSet of gDoc trees.
-RSLocator locator = ....
-FaultListener listener = new FaultListener() {
- @Override
+An example of activation record is the following:
- void onFault(String unparsedResult, Throwable failure) {...process failure...}
-}
-GDocRSCollection docs = new GDocRSCollection(locator, listener);
-for (GDoc doc : doc)
-  ...process document...
-</source>
+[[Image:cmactivationrecord.jpg|center|An Activation Record of the service]]
-{|style="border-collapse: separate; border-spacing: 10; border-width: 2px; border-style: solid;"
-|
-'''note''': a <code>GDocRSCollection</code> may be iterated over an arbitrary number of times, as expected. Due to its lazy nature, however, the iteration may unconventionally fail at the start. To reduce this chance, the collection eagerly creates a first iterator at construction time. If this succeeds, the first iteration is guaranteed to start correctly. Later iterations may still fail, however, as further iterators are created on demand.
-|}
-Clients which are not well-served by the asynchronous delivery of failures can instead opt for a <code>GDocRSIterator</code>, which again offers binding transparencies but delivers failures synchronously:
+The replication mechanism of the service implies that each instance ought to recognise and ignore the notifications of the activation records that it previously published. This raises a requirement of persistence on the <code>Factory</code> of the instance, which stores the identifiers of its own activation records.
-<source lang="java5" highlight="4,6,9,12-14">
+FInally, the <code>Factory</code> resource publishes a multi-valued Resource Property <code>Plugin</code>, which contains a <code>PluginDescription</code> for each of the plugins hosted by the running instance (see WSDL definition).
-//a locator to a ResultSet of gDoc trees.
-RSLocator locator = ....
-GDocRSIterator it = new GDocRSCollection(iterator);
-while (it.hasNext) {
+[[Image:factoryRPs.jpg|center|The Resource Properties of Factory resource]]
-    ...
-  try {
-    GDoc doc = it.next();
-    ....process document...
-  }
- catch (Throwable failure) {
-        ...handle failure...
-  }
- ...
-}
-</source>
-{|style="border-collapse: separate; border-spacing: 10; border-width: 2px; border-style: solid;"
+== Read Managers ==
-|
-'''note''': <code>GDocRSCollection</code> and <code>GDocRSIterator</code> are specialisation of more generic facilities: <code>RSCollection&lt;T&gt;</code> and <code>RSIterator&lt;T&gt;</code>, respectively, where <code>T</code> can be specialised to bindings other than <code>GDoc</code> (e.g. JAXB bindings), and in fact to results other than <code>gDoc</code> trees. For this, clients must define a suitable implementation of <code>ResultParser&lt;T&gt;</code>. For more details, see the code documentation.
-|}
+A <code>ReadManager</code> resource exposes read-only operations over the content of the bound collection. The operations are intended for generic use, in that clients are not required to know the repository that hosts the collection or the service plugin that mediates access to that repository. The operations, their inputs, and their outputs are defined in the [[Content Manager's ReadManager PortType WSDL|WSDL]] of the port-type.
-Finally, we give an example of queries for <code>gDoc</code> trees:
-<source lang="java5" highlight="2,8,11">
+* <code>AnyHolder getByID(GetByIDParams params) throws UnknownDocumentFault,GCUBEFault</code>
-//asynchronous: return a locator to a remote ResultSet of many gDoc trees pruned by a tree predicate
+: the operation returns a <code>gDoc</code> tree. Invocations are parameterised by:
-RSLocator locator3 = call.get(projection);
+:* the identifier of the tree root;
+:* [optional] a tree predicate with respect to which the tree should be pruned before it is returned.
-//some tree predicate to use for filtering
-Predicate filter = ....
-//asynchronous: return a locator to a remote ResultSet of many pruned gDoc trees that satisfy a given filter
-RSLocator locator4= call.get(projection,filter);
-//asynchronous: return a locator to a remote ResultSet of all the gDoc trees in the collection
+[[Image:getbyidop.jpg|center|The <code>getByID</code> operation of ReadManager resources]]
-RSLocator locator5 = call.get();
-</source>
-Again, the ResultSets returned by the queries can be consumed with standard ResultSet consumption idioms.
-==== WriteManager Calls ====
-A <code>WriteManagerCall</code> gives high-level write access to the content of a given collection, as allowed by a <code>WriteManager</code> resource bound to that collection.
+* <code>String getByIDs(GetByIDsParams params) throws GCUBEFault</code>
-It follows the same patterns already seen for <code>FactoryCall</code>s. In particular, it is created in a scope and, optionally, with a security manager:
+: the operation returns the locator of a ResultSet of <code>gDoc</code> trees. Invocations are parameterised by:
+:* the locator of a ResultSet of tree root identifiers;
+:* [optional] a tree predicate with respect to which the trees should be pruned before they are returned.
+:'''note''': tree predicates in input and <code>gDoc</code> trees in output are contained in an <code>AnyHolder</code> wrapper (see WSDL definition).
-<source lang="java5" highlight="4,9">
-//some scope
-GCUBEScope scope = .....
-WriteManagerCall call = new WriteManagerCall(scope);
-//some security manager = ....
+[[Image:getbyidsop.jpg|center|The <code>getByIDs</code> operation of ReadManager resources]]
-GCUBESecurityManager manager = ....
-WriteManagerCall secureCall = new WriteManagerCall(scope,manager);
-</source>
-As a further option, it may be crated with the identifier of the target collection:
-<source lang="java5" highlight="4,9">
+* <code>String get(GetParams params) throws GCUBEFault</code>
-//some scope
+: the operation returns the locator of a ResultSet of <code>gDoc</code> trees. Invocations are parameterised by:
-GCUBEScope scope = .....
+:* [optional] a tree predicate with respect to which the trees should be pruned before they are returned.
+: '''note''': invocations that specify no predicate effectively ask for the contents of the whole collection.
-WriteManagerCall call = new WriteManagerCall("... some collection identifier ...",scope);
-//some security manager = ....
-GCUBESecurityManager manager = ....
-ReadManagerCall secureCall = new WriteManagerCall("... some collection identifier ...",scope,manager);
+[[Image:getop.jpg|center|The <code>get</code> operation of ReadManager resources]]
-</source>
-{|style="border-collapse: separate; border-spacing: 10; border-width: 2px; border-style: solid;"
-|
-'''note''': the collection identifier may also be set after call construction (cf. <code>setCollectionID(String)</code>).
-|}
-The call may be configured as a <code>FactoryCall</code>, i.e. setting reference to resource endpoint for targeted interactions (cf. <code>setEndpointReference(EndpointReference)</code>). or else relying on implicit discovery and best-effort strategy; in the latter case, the query that underlie the strategy can be customised (cf. <code>getQuery()</code>,<code>resetQuery()</code> ).
+* <code>AnyHolder getNode(Path path) throws UnknownPathFault,GCUBEFault</code>
+: the operation returns the node of a <code>gDoc</code>. Invocations are parameterised by:
+:*  the path of node identifiers that connects the node to the root of the tree.
+:'''note''': returned nodes are contained in an <code>AnyHolder</code> wrapper (see WSDL definition).
-The call may then be used to add or update individual or multiple <code>gDoc</code> trees into the target collection. Additions and updates can be applied to individual trees as well as in bulk. In the latter case, the changes are applied by the service asynchronously, based on the [[GCube_ResultSet_(gRS)|ResultSet]] mechanism.
-Additions operations may be illustrated as follows:
+[[Image:getnodeop.jpg|center|The <code>getNode</code> operation of ReadManager resources]]
-<source lang="java5" highlight="5,11">
-//A gDoc tree without identifiers
-GDoc doc = ....
-//synchronous: adds a gDoc tree and receives the identifier assigned to its root
-String rootID = call.add(doc);
-//a locator to a ResultSet of gDoc trees without identifiers.
+* <code>String getNodes(String path) throws GCUBEFault</code>
-RSLocator locator1 = ....
+: the operation returns the locator of a ResultSet of <code>gDoc</code> tree nodes. Invocations are parameterised by:
+:* the locator of a ResultSet of paths of node identifiers that connect the returned nodes to the roots of the corresponding trees.
-//asynchronous: adds many gDoc trees and receives a locator to a remote ResultSet of AddOutcome objects (see WSDL)
-RSLocator locator2 = call.add(locator);
-</source>
-Here, <code>add(Gdoc)</code> requires a binding of the input tree to the object model of <code>gDoc</code> [[#gDoc_API|tree API]].
-There is semantically equivalent operation that takes DOM bindings, so as to raise no further serialisation costs if a binding other than to the <code>gDoc</code> tree API is already used upstream (cf. <code>getAsElement(String)</code> and <code>add(Element)</code>).
-The ResultSet returned by the second method may be consumed with standard ResultSet idioms but, as already shown for [[#ReadManager_Calls|ReadManager calls]], <code>RSCollection&lt;T&gt;</code> and <code>RSIterator&lt;T&gt;</code> are more convenient options. In particular, the stub distribution offers a <code>AddOutcomeParser</code> to use in conjunction with  <code>RSCollection&lt;AddOutcome&gt;</code> or <code>RSIterator&lt;AddOutcome&gt;</code>:
+[[Image:getnodesop.jpg|center|The <code>getNodes</code> operation of ReadManager resources]]
-<source lang="java5" highlight="5,12">
-//A locator to ResultSet of AddOutcomes
-RSLocator locator = ...
-//may also set a FaultListener, if required
-RSCollection<AddOutcome> outcomes = new RSCollection<AddOutcome>(locator, new AddOutcomeParser());
-for (AddOutcome outcome : outcomes)
+A <code>ReadManager</code> resource publishes the following Resource Properties:
-   ...process outcomes...
-//or, alternatively..
+* <code>CollectionID</code>: the identifier of the bound collection, as per OCMA requirements;
+* <code>TypeID</code>: the access type of the port-type, as per OCMA requirements. The value of this property is constant: <code>gDocRead</code>;
+* <code>Plugin</code>: the name of the bound plugin;
+* <code>Cardinality</code>: an estimate of the number of documents in the bound collection;
+* <code>LastUpdate</code>: an estimate of the time in which the bound collection was last updated;
-RSIterator<AddOutcome> it = new RSIterator<AddOutcome>(locator, new AddOutcomeParser());
-while (it.hasNext() {
+[[Image:readmanagerps.jpg|center|The Resource Properties of ReadManager resources]]
-   ...
-   try {
-      AddOutcome outcome = it.next();
-      ... process outcome...
-   }
-   catch(Throwable failure) {
-     ...handle failure...
-  }
-}
-</source>
-{|style="border-collapse: separate; border-spacing: 10; border-width: 2px; border-style: solid;"
+== Write Managers ==
-|
-'''note''': since the <code>gDoc</code> trees in input are still to be assigned identifiers, they can only be associated to successes or failures in output based on their order within the ResultSet.
-|}
-The next example illustrates update operations, which rely on the notion of 'delta document' discussed [[#Write_Managers|above]]:
+A <code>WriteManager</code> exposes four write-only operations over the content of a bound collection. The operations are intended for generic use, in that clients are not required to know the repository that hosts the collection or the service plugin that mediates access to that repository. The operations, their inputs, and their outputs are defined in the [[Content Manager's WriteManager PortType WSDL|WSDL]] of the port-type.
-<source lang="java5" highlight="5,11">
+* <code>String add(AnyHolder holder) throws InvalidDocumentFault, GCUBEFault</code>
-//A delta tree
+: the operation adds a <code>gDoc</code> tree to the bound collection and returns the identifier assigned to the tree root as a result.
-GDoc delta = ....
+: '''note''': the operation assumes that all nodes of the input tree acquire identifiers as a result of a successful addition. Depending on the plugin that serves the request, the operation may fail if the nodes already have an identifier.
-//synchronous: updates the document with a delta tree
-call.update(delta);
-//a locator to a ResultSet of DOM representations of delta trees.
+[[Image:Addop.jpg|center|The <code>add</code> operation of WriteManager resources]]
-RSLocator deltas = ...
-//asynchronous: updates zero or more documents with corresponding delta trees and receives a locator to a ResultSet of UpdateFailure objects (see WSDL)
-RSLocator locator3 = call.update(deltas);
-</source>
-As for the <code>add</code> operations, consuming the ResultSet can conveniently rely on appropriate type specialisation of <code>RSCollection&lt;T&gt;</code> and <code>RSIterator&lt;T&gt;</code>. In particular, the stub distribution offers a <code>UpdateFailureParser</code> to use in conjunction with  <code>RSCollection&lt;UpdateFailure&gt;</code> or <code>RSIterator&lt;UpdateFailure&gt;</code>:
+* <code>String addRS(String locator) throws GCUBEFault</code>
+: the operation adds zero or more <code>gDoc</code> trees to the bound collection and returns a locator of a ResultSet of <code>AddOutcome</code>s, one for each tree in input (see WSDL definition). In the case of successful outcome, clients obtain the identifier assigned to the tree root. In the case of failure, they obtain the stack trace of the fault that prevented the addition of the tree.
+: '''note''': again, the operation assumes that all nodes of the input tree acquire identifiers as a result of a successful addition. Depending on the plugin that serves the request, the operation may fail if the nodes already have an identifier.
-<source lang="java5" highlight="5,12">
-//A locator to ResultSet of UpdateFailures
-RSLocator locator = ...
-//may also set a FaultListener, if required
+[[Image:AddRSop.jpg|center|The <code>addRS</code> operation of WriteManager resources]]
-RSCollection<UpdateFailure> failures = new RSCollection<UpdateFailure>(locator, new UpdateFailureParser());
-for (UpdateFailure failure : failures)
-   ...process failure...
-//or, alternatively..
+* <code>VOID update(AnyHolder holder) throws UnknownDocumentFault, InvalidDocumentFault, GCUBEFault</code>
+: the operation updates a <code>gDoc</code> tree in the bound collection (the ''target tree''). Invocations are parameterised by ''delta trees'', <code>gDoc</code> trees that capture all the updates to be applied to target trees, including addition or removal of attributes, changes to attribute values, addition and removal of nodes, and changes to leaf values. In more detail, a delta tree is comprised of nodes marked with an attribute <code>http://gcube-system.org/namespaces/contentmanagement/gdoc:status</code> and includes:
+:* all the nodes of the target tree which have changed, with the corresponding identifiers and with a <code>status</code> of <code>MODIFIED</code> or <code>DELETED</code>;
+:: '''note''': attributes of the target tree that have been removed occur in the delta tree with a value of <code>_null</code>;
+:: '''note''': <code>DELETED</code> nodes are either leaves with a value of <code>_null_</code> or inner nodes with no children.
+:: '''note''': nodes that have no identifiers cannot be referred to in the delta tree, hence cannot be updated.
+:* all the nodes that should be added to the target tree, without identifiers and with a <code>status</code> of <code>NEW</code>.
+:: '''note''': <code>NEW</code> nodes are leaves or inner nodes with <code>NEW</code> descendants.
+:: '''note''': depending on the bound plugin, <code>NEW</code> nodes may be assigned identifiers upon being added to the target tree.
-RSIterator<UpdateFailure> it = new RSIterator<UpdateFailure>(locator, new UpdateFailureParser());
-while (it.hasNext() {
+[[Image:Updateop.jpg|center|The <code>update</code> operation of WriteManager resources]]
-   ...
-   try {
-      UpdateFailure failure = it.next();
-      ... process failure...
-   }
-   catch(Throwable f) {
-     ...handle iteration failure...
-  }
-}
-</source>
-A difficulty in issuing updates is to produce the corresponding delta documents. To this end, the <code>gDoc</code> [[#gDoc_API|tree API]] offers a method <code>delta(GDoc)</code> on its <code>GDoc</code> class for root nodes. The method takes the root of a second <code>gDoc</code> tree and computes the delta document between the two trees in the assumption that the second tree represents the evolution of the first under update (i.e. its future version).
+:As an example, consider the following <code>gDoc</code> tree, where <code>status</code> attributes are depicted as node markers:
-Accordingly, the API supports the following 'clone-change-compare' model of update at the client-side:
-* the client clones trees;
+[[Image:DeltaTree.jpg| center | A Delta Tree]]
-* the client updates the clones;
-* the client computes and uses delta documents between the original trees and the evolved clones;
-The following example illustrates the model:
-<source lang="java5" highlight="5,10">
+:Here, the delta tree captures a set of updates to the descendants of a <code>gDoc</code> tree:
-//original tree, as obtained from the service, directly or indirectly.
-GDoc doc = .....
-//use copy-constructor to clone the tree
+:* the root has acquired or changed an attribute <code>x</code>;
-GDoc clone = new GDoc(doc);
+:* node <code>2</code> has lost its child <code>5</code> and all its descendants;
+:* node <code>3</code> has acquired a child and its descendants;
+:* node <code>4</code> has lost the attribute <code>z</code> and the child <code>6</code>, while the value of its child leaf <code>7</code> has changed.
-... update the clone ...
+: '''note''': all the nodes of the target tree that have not changed do not occur in the delta tree.
-//compute delta document
+* <code>String updateRS(String locator) throws GCUBEFault</code>
-GDoc delta = doc.delta(clone);
+: the operation takes the locator of a ResultSet with delta trees for zero or more target <code>gDoc</code> trees and returns the locator of a ResultSet of <code>UpdateFailure</code>s (see WSDL definition), one for each delta tree that could not be processed into an update of the corresponding target tree. In particular, clients obtain the identifier of the root of the target tree and the stacktrace of the error that occurred during its update.
+:'''note''': delta trees in input are contained in <code>AnyHolder</code> wrappers (see WSDL definition).
-... use delta in update operations ...
-</source>
+[[Image:UpdateRS.jpg|center|The <code>updateRS</code> operation of WriteManager resources]]
-{|style="border-collapse: separate; border-spacing: 10; border-width: 2px; border-style: solid;"
-|
-'''note''': <code>delta()</code> leaves the original tree and its evolved clone untouched.
-'''note''': the model does ''not'' require that the clone is updated using the <code>gDoc</code> tree API. The clone may be updated under any object model, including a JAXB-annotated class with mutator methods. When the delta document is needed, it may be unbound from such model and bound to the object model of the <code>gDoc</code> tree API, so as to pass it to the <code>delta()</code> method and, from there, to the updated operation of the call object.
+A <code>WriteManager</code> resource publishes the following Resource Properties:
-|}
-=== Content URIs ===
-<code>ReadManagerCall</code>s allows clients to easily lookup <code>gDoc</code> trees if they know the collections that contain them. Lookups thus need context, and this context complicates the dissemination of content within the system. In particular, we cannot exchange root identifiers as ''references'' to the trees, as these alone are sufficient for neither identification nor lookup. The problem worsens if we wish to exchange references an arbitrary tree node, as in this case the context increases to include not only the collection that contains the tree but also all the path that connects the root of the tree to the node. To solve this problem, the stub distribution defines a scheme for URIs that point to arbitrary nodes of <code>gDoc</code> trees, encapsulating all the information required to resolve the trees and access the nodes.
-A ''content URI'' is a URI of this form:
-<code>cms://</code>id(0)<code>/</code>id(1)<code>/</code>...<code>/</code>id(N)
-where:
-* <code>cms</code> is the scheme of the URI;
-* ''id(0)'' is the identifier of a collection;
-* ''id(1)'' is the identifier of the root a <code>gDoc</code> tree in the collection;
-* ''id(i)'' is the identifier of  a child of the node identified by ''id(i-1)'', where ''i''>1.
-Overall, a content URI identifies a node of a <code>gDoc</code> tree in some collection, and service clients can disseminate it as a compact reference to it.
-Of course, of our URIs should be resolvable, i.e. should have the semantics of URLs. The stubs distribution provides two means to resolve content URIs. The first is via the static method <code>get(URI)</code> of the <code>ReadManagerCall</code>:
-<source lang="java5" highlight="8">
+* <code>CollectionID</code>: the identifier of the bound collection, as per OCMA requirements;
-//A content URI
+* <code>TypeID</code>: the access type of the port-type, as per OCMA requirements. The value of this property is constant: <code>gDocWrite</code>;
-URI uri = ....
+* <code>Plugin</code>: the name of the bound plugin;
+* <code>Cardinality</code>: an estimate of the number of documents in the bound collection;
-//Scope of resolution
+* <code>LastUpdate</code>: an estimate of the time in which the bound collection was last updated;
-GCUBEScope scope = ...
-//resolve
-Node node = ReadManagerCall.get(uri,scope);
-</source>
-Essentially, the method extracts the collection identifier from the URI in input and uses it to create a <code>ReadManagerCall</code> in the input scope. It then extracts the document identifier from the URI and passes it to the [[#ReadManager Calls|method <code>get(String)</code>]] of the call object to resolve the corresponding <code>gDoc</code> tree. It then navigates the tree along the node identifiers in the URI and returns the node with the last identifier.
-The second method of resolution is stream-based and relies on standard Java APIs for URL resolution:
-<source lang="java5" highlight="12,13">
-import static org.gcube.contentmanagement.contentmanager.stubs.model.protocol.URIs.*;
-import static org.gcube.contentmanagement.contentmanager.stubs.model.trees.Bindings.*;
-...
-//A content URI
-URI uri = ....
-//Scope of resolution
-GCUBEScope scope = ...
-//resolve
-URLConnection connection = connection(uri,scope);
-Node node = fromXML(connection.getInputStream());
-</source>
-Here, we work with a standard <code>URLConnection</code>, though we obtain it from a utility of the <code>URIs</code> class. The utility converts the URI to a URL, obtains from it a <code>URLConnection</code> and configures it with the target scope (cf. <code>URIs.connection(URI,scope)</code>). We then obtain a stream from the connection and use the binding facilities in [[#Binding Trees|<code>Bindings</code>]] to parse the stream into a node (other bindings are of course possible).
-{|style="border-collapse: separate; border-spacing: 10; border-width: 2px; border-style: solid;"
-|
-'''note''': URL-based resolution requires the registration of a protocol handler for the <code>cms</code> scheme. The handler is part of the stub distribution and is registered automatically when the <code>URIs</code> class is first loaded, e.g. when invoking the method <code>connection()</code> above. Manual registrations are also possible (cf. <code>org.gcube.contentmanagement.contentmanager.stubs.model.protocol.cms.Handler.activateProtocol()</code>).
-|}
-As for the generation of content URIs for given nodes of <code>gDoc</code> trees, clients may invoke the method <code>uri()</code> of the <code>Node</code> class of the <code>gDoc</code> [[#gDoc API|tree API]]:
-<source lang="java5" highlight="5">
-//a node of a gDoc tree
-Node n = ...
-//Its content URI
-URI uri = node.uri()
-</source>
-{|style="border-collapse: separate; border-spacing: 10; border-width: 2px; border-style: solid;"
-|
-'''note''': <code>uri()</code> fails if the root of the tree has no collection identifier, or if any of the nodes on the path that connects it to the root have no identifiers.
-|}
-Besides the method <code>connection()</code> discussed above, the class <code>URIs</code> offers a number facilities to create and manipulate content URLs, including:
-* <code>make(String, String*)</code> : creates a content URI from a collection identifier and a number of object identifiers.
-: the method is invoked by the method <code>uri()</code> just discussed, but clients may also invoke it directly if they obtain the components of the URI through other means.
-* <code>collectionID(URI)</code>: returns the identifier of the collection ''in'' a content URI;
-* <code>documentID(URI)</code>: returns the identifier of the document ''in'' a content URI;
-* <code>nodeID(URI)</code>: returns the identifier of the node referred to ''by'' the content URI;
-* <code>nodeIDs(URI)</code>: returns the all the node identifiers ''in'' a content URI;
-* <code>parentURI(URI)</code>: returns the content URI of the parent of the node referred to ''by'' the content URI;
-* <code>documentURI(URI)</code>: returns the content URI of the document ''in'' the content URI;
-* <code>predicate(URI)</code>: returns a <code>gDoc</code> [[#Tree Predicates|tree predicate]] for the existence of the path comprised of the identifiers ''in'' a content URI;
-== Content Management Library ==
+[[Image:WriteManagerRPs.jpg|center|The Resource Properties of WriteManager resources]]