Ideas on derivation

bulk/repo-access-api.adoc

@@ -49,7 +49,24 @@ We enforce conformance to{fn-org223}
 * No unresolvable containment/annotation ids
 * No unresolvable parent ids
 
-For now, we do not support paging, as paging tree nodes is non-trivial{fn-org204}
+For now, we do not support paging, as paging tree nodes is non-trivial.{fn-org204}
+
+== Deleted nodes
+We don't have an explicit delete API.{fn-org221}
+
+We don't support [[orphan, orphans]]_orphans_ for now.
+They are immediately deleted.{fn-org219}
+
+Deleted nodes don't exist anymore in the repository from the client's point of view.
+They might still exist in other contexts (e.g. another branch), or physically within the repository for internal reasons (e.g. storage optimization, concurrent editing support).
+A deleted node MUST NOT appear in any responses according to this API.{fn-org220}
+
+A repository MAY consider the deleted node's id to be _unused_, and thus allow to re-use it.
+A repository also MAY disallow re-using previously deleted node ids.
+
+== Error cases for all SerializationChunks
+=== Node with same id sent more than once
+
 
 [[apis]]
 == APIs
@@ -71,15 +88,17 @@ None.
 The partitions are sent as complete nodes.{fn-org202}
 Does NOT include <<{m3}.adoc#Language, Languages>> or partition children.
 
+==== Error cases
+None.
+
 [[createPartitions, createPartitions]]
 === createPartitions: Create new partitions
 Creates new partitions in the repository.{fn-org216}
 
 Each sent node is its own partition.
 Thus, we cannot send the contents (i.e. (indirect) annotations/containments) of a partition; We can send them in a later <<store>> call.
 We also MUST NOT mention any annotation/containment node ids in the partition nodes, as they cannot be part of the same request, and we don't allow moving nodes in this operation.
-
-We MAY send properties and references #TODO correct?#{fn-org225}
+We MAY send properties and references{fn-org225}
 
 Each partition node id MUST NOT exist in the repository, and the sending client MUST use node ids allocated to it via <<ids>>.
 
@@ -90,6 +109,11 @@ Each partition node id MUST NOT exist in the repository, and the sending client
 ==== Result
 #TODO#
 
+==== Error cases
+===== Partition node id already exists
+===== Partition node id not reserved for this client
+===== Partition node lists contained or annotated nodes
+
 [[deletePartitions, deletePartitions]]
 === deletePartitions: Delete partitions and all their contents
 Deletes all mentioned partitions, including all (transitive) annotations and children.
@@ -105,6 +129,10 @@ All (transitive) annotations and children become <<orphan, orphans>>.
 ==== Result
 #TODO#
 
+==== Error cases
+===== Node with that id is not a partition
+===== Node with that id does not exist
+===== Invalid node id
 
 [[retrieve, retrieve]]
 === retrieve: Get nodes from repository
@@ -137,6 +165,11 @@ We need to omit the parameter if we don't want to limit the depth.
 {chunk} containing all nodes according to `nodeIds` and `depthLimit` parameters.
 Does NOT include the definition of <<{serialization}.adoc#UsedLanguage, UsedLanguages>>, only their <<{serialization}.adoc#MetaPointer, MetaPointers>>.
 
+==== Error cases
+===== Node with requested id does not exist
+===== Invalid depthLimit
+===== Invalid node id
+
 ==== Example request
 [source, httprequest]
 ----
@@ -189,21 +222,6 @@ Also, we assume no knowledge of the metamodel.
 
 #TODO do we support changes to classifier? What about changes in metapointer.version? Migration use cases?#{fn-org69}
 
-
-[[orphan]]
-.Orphans
-An _orphan_ node is a node present in the repository that is not mentioned in any other node as containment or annotation, and is not a partition.{fn-org219}
-We cannot create orphans explicitly.
-A node becomes an orphan if it already exists in the repository, but the node's id is removed from its parent containment/annotation,
-and the node is not moved to another parent.
-
-#TODO: What about references to orphans? Do they resolve?#
-
-A repository MAY immediately <<delete>> orphans, or keep them in a <<{trash}.adoc, trash>>.
-
-The repository MUST NOT update/change references to orphan nodes.
-Rationale: We in general do support unresolved or unresolvable references.
-
 .How to handle unknown ids?
 
 * If requested by this client via <<ids>>: Create new node
@@ -226,46 +244,13 @@ Rationale: We in general do support unresolved or unresolvable references.
 // include::partitions.json[]
 // ----
 
+==== Error cases
+===== Node id mentioned as annotation/child in more than one parent
+===== Move would create loop in tree
+===== Parent / child / annotation node id unknown
+===== Parent doesn't match child/annotation
+===== New node id not reserved for this client
 
-[[delete, delete]]
-=== delete: Delete nodes from repository
-Deletes nodes from the repository.{fn-org221}
-
-Deleted nodes don't exist anymore in the repository from the client's point of view.
-They might still exist in other contexts (e.g. another branch), or physically within the repository for internal reasons (e.g. storage optimization, concurrent editing support).
-After this call succeeds, the deleted nodes MUST NOT appear in any responses according to this API.{fn-org220}
-
-If we delete a node, we implicitly remove it from its parent's containment/annotation.
-
-(Transitively) contained/annotation nodes of deleted nodes that are not explicitly mentioned in the call are deleted.
-
-After this call succeeds, a repository MAY consider the deleted node's id to be _unused_, and thus allow to re-use it.
-A repository also MAY disallow re-using previously deleted node ids.
-
-The whole call fails, without any changes to the repository, if any of the provided node ids does not exist in the repository, or any of the provided node ids is a <<{m3}.adoc#predefined-builtins-keys, built-in id>>.
-
-==== Parameters
-[[delete.nodeIds]]
-`nodeIds`:: List of node ids we want to delete from the repository.
-
-==== Result
-#TODO#
-
-==== Example request
-[source, httprequest]
-----
-DELETE /bulk/delete?nodes=["first-node-id","13123123","c2Vjb25kIG5vZGUgaWQ"]
-----
-
-[NOTE]
-====
-link:https://www.rfc-editor.org/rfc/rfc9110.html#name-delete[RFC 9110: HTTP Semantics] states about the request body of a DELETE method:
-
-> An origin server SHOULD NOT rely on private agreements to receive content, since participants in HTTP communication are often unaware of intermediaries along the request chain.
-
-Thus, this example sends the node ids via URL query.
-
-====
 
 [[ids, ids]]
 === ids: Get available ids
@@ -298,6 +283,9 @@ It MAY return less than `count` ids.
 ==== Result
 List of ids guaranteed to be free.
 
+==== Error cases
+None.
+
 ==== Example request
 [source, httprequest]
 ----
@@ -321,7 +309,7 @@ create:: <<createPartitions>> for partitions, <<store, store call>> that sends a
 
 update:: <<store, store call>> that sends a node (both partitions and other nodes) with an _existing id_, including all its features (both updated and unchanged).
 
-delete:: <<deletePartitions>> for partitions (including all descendants), <<delete>> for other nodes
+delete:: <<deletePartitions>> for partitions (including all descendants), for others <<store>> of the parent node without mentioning the deleted node.
 
 move:: Assume we want to move node `N` from its current parent `S` to its new parent `T`.
 +

derived/completeness.adoc

@@ -0,0 +1,175 @@
+= Completeness Scenario
+
+"complete" means "a processor has finished all its work _up to the point in time at which we asked it_"
+
+Same as in https://github.com/LionWeb-io/specification/issues/248#issuecomment-2079730360[#248]
+
+== Assumptions
+
+* processors use the same APIs as other clients to retrieve their base models (and any other nodes they might need).
+
+* There is a central authority `derivationBackend` within a repository that handles requests for derivations.
+
+* `derivationBackend` has a special API to ask processors for a _complete_ derivation.
+(Note that this only contains the derived nodes from this processor -- `derivationBackend` aggregates the results of all processors that contribute to the same derivation).
+
+NOTE: Red activity lines for repository are updates. +
+Yellow boxes are the incoming, unprocessed changes.
+
+[plantuml,scenario,svg]
+----
+!pragma teoz true
+
+actor generator as gen
+participant derivationBackend as db
+participant ScopeProcessor as scope<<delta>>
+participant ValidationProcessor as val<<bulk>>
+participant DomainValidator as dom<<delta>>
+database repository as repo
+
+autonumber 1
+
+repo <<-]: <font color=darkcyan>change1</font>
+activate repo #red
+autonumber 1
+repo ->> scope: <font color=darkcyan>change1</font>
+activate scope
+autonumber 1
+scope -> scope: queue
+note over scope: <b>inbox:</b>\n<font color=darkcyan>change1</font>
+[-[hidden]>scope
+deactivate scope
+
+autonumber 1
+repo ->> dom: <font color=darkcyan>change1</font>
+deactivate repo
+activate dom
+autonumber 1
+dom -> dom: queue
+note over dom: <b>inbox:</b>\n<font color=darkcyan>change1</font>
+[-[hidden]>dom
+deactivate dom
+
+autonumber 1
+repo <<-]: <font color=coral>change2</font>
+activate repo #red
+autonumber 1
+repo ->> scope: <font color=coral>change2</font>
+note over scope: <b>inbox:</b>\n<font color=darkcyan>change1</font>\n<font color=coral>change2</font>
+autonumber 1
+repo ->> dom: <font color=coral>change2</font>
+note over dom: <b>inbox:</b>\n<font color=darkcyan>change1</font>\n<font color=coral>change2</font>
+deactivate repo
+
+autonumber 1
+repo <<-]: <font color=deeppink>change3</font>
+activate repo #red
+autonumber 1
+repo ->> scope: <font color=deeppink>change3</font>
+note over scope: <b>inbox:</b>\n<font color=darkcyan>change1</font>\n<font color=coral>change2</font>\n<font color=deeppink>change3</font>
+autonumber 1
+repo ->> dom: <font color=deeppink>change3</font>
+note over dom: <b>inbox:</b>\n<font color=darkcyan>change1</font>\n<font color=coral>change2</font>\n<font color=deeppink>change3</font>
+deactivate repo
+
+== global state: aa ==
+
+autonumber 2
+
+gen -> db: get validation\nderivation
+activate db
+  autonumber 3
+  db ->> scope
+  activate scope
+
+  autonumber 3
+  db ->> val
+  activate val
+
+  autonumber 3
+  db ->> dom
+  activate dom
+
+  autonumber 13
+  & dom -> repo ++: get persisted derived model
+  return
+
+  autonumber 4
+  scope -> scope: process <font color=darkcyan>change1</font>
+  note over scope: <b>inbox:</b>\n<font color=coral>change2</font>\n<font color=deeppink>change3</font>
+  autonumber 10
+  &  val -> repo++: get original model
+  return
+
+
+  autonumber 5
+  scope -> scope: process <font color=coral>change2</font>
+  note over scope: <b>inbox:</b>\n<font color=deeppink>change3</font>
+
+
+  autonumber 14
+  & dom -> dom: process <font color=darkcyan>change1</font>\nno change
+  note over dom: <b>inbox:</b>\n<font color=coral>change2</font>\n<font color=deeppink>change3</font>
+  autonumber 15
+  dom -> dom ++: process <font color=coral>change2</font>
+  autonumber stop
+    dom -> repo ++ #red: update persisted derived model
+    return
+
+== global state: ab ==
+
+  return
+  note over dom: <b>inbox:</b>\n<font color=deeppink>change3</font>
+
+  autonumber 12
+  val -> val: calculate
+  autonumber 12
+  val -->> db: validations
+  deactivate val
+
+
+  autonumber 6
+  repo <<-]: <font color=goldenrod>newChange</font>
+  activate repo #red
+  autonumber 7
+  repo ->> scope: <font color=goldenrod>newChange</font>
+  note over scope: <b>inbox:</b>\n<font color=deeppink>change3</font>\n<b>delayed inbox:</b>\n<font color=goldenrod>newChange</font>
+  autonumber 16
+  repo ->> dom: <font color=goldenrod>newChange</font>
+  note over dom: <b>inbox:</b>\n<font color=deeppink>change3</font>\n<b>delayed inbox:</b>\n<font color=goldenrod>newChange</font>
+  deactivate repo
+
+== global state: bb ==
+
+  autonumber 7
+  scope -> scope: process <font color=deeppink>change3</font>
+  note over scope: <b>inbox:</b>\n<i>empty</i>\n<b>delayed inbox:</b>\n<font color=goldenrod>newChange</font>
+  autonumber 8
+  scope -->> db: validations
+  deactivate scope
+  note over scope: <b>inbox:</b>\n<font color=goldenrod>newChange</font>
+
+  autonumber 17
+  dom -> dom: process <font color=deeppink>change3</font>\nno change
+  note over dom: <b>inbox:</b>\n<i>empty</i>\n<b>delayed inbox:</b>\n<font color=goldenrod>newChange</font>
+  autonumber 18
+  dom -->> db: validations
+  deactivate dom
+  note over dom: <b>inbox:</b>\n<font color=goldenrod>newChange</font>
+
+autonumber 20
+gen <-- db
+deactivate db
+
+activate scope
+autonumber 9
+scope -> scope: process <font color=goldenrod>newChange</font>
+[-[hidden]>scope
+deactivate scope
+
+activate dom
+autonumber 19
+dom -> dom: process <font color=goldenrod>newChange</font>\nno change
+[-[hidden]>dom
+deactivate dom
+----