Skip to main content

· 5 min read
Herwig Mannaert

The use of transactions in software systems is quite a fundamental and non-trivial issue. Every one knows the obvious example of retrieving money from a cash machine. Either your account is debited and you receive the money, or none of the above happens. In case the machine fails to dispense the money, the transaction requires a rollback of the debiting. However, it is not always that straightforward. Though the use of transactions providing automated rollbacks is often desirable to end-users and system analysts, it may well be unnecessary complicated or even plainly impossible to implement for the software developer.

In order to enable scaling and to avoid deadlocks, transaction processing systems typically deal with large numbers of small things, and strive to come in, do the work, and get out. This means that it is in general not feasible to use technical transactions to provide end-to-end transactional integrity for customer transactions. For instance, guaranteeing the customer transaction of a money transfer is in general not implemented through a technical transaction encompassing the debiting of the source bank account and the crediting of the destination bank account, which may reside in another bank across the globe. It is typically guaranteed by taking the transfer request as fast as possible into a secure database, followed by a number of sequential processing steps or individual transactions. This flow may even require human intervention to make sure that the customer transaction, i.e., the transfer request stored in the secure database, is executed properly.

The use of end-to-end technical transactions would often cause lots of problems in the real world. For instance, defining a technical transaction around the reservation of a flight ticket, a rental car, and a hotel room, could seem desirable to indulge the customer, but would soon lead to the simultaneous locking of all reservation systems around the world. Avoiding such deadlocks is closely related to the NS theorem Separation of States. This theorem states that after every task or action, the result state should be stored in a corresponding and appropriate data entity. This will enable the independent execution, and therefore evolution, of the implementation of the processing task, and the error handling of the task. Besides better evolvability, this strategy improves the tracing and handling of errors, and reduces the chances of having locked resources.

Another often overlooked issue in transaction management is the complexity of the rollback. While a previous write operation in the same database could be compensated by a relatively simple rollback, developers would have to implement dedicated compensating transactions in case the various steps imply changes across different systems. These compensating actions are not always straightforward or even possible. For instance, in case one of the steps has triggered a physical action, like shipping a product or opening a switch or water tap, it is simply impossible to rollback. Unless mopping the floor is part of the compensating transaction.

These issues are also quite relevant when client systems invoke an API (Application Programming Interface) of a target system. Examples of such client systems are business process management systems invoking services from underlying transactional systems, or user interface front-ends developed upon the service APIs provided by back-end systems. The client systems often desires a customer transaction encompassing several service calls on the target system. This requires client and target system to agree on one or multiple scenarios, that define the responsibilities and behavior at both ends. We discuss here the various possibilities.

  • The client system defines the transaction and the target system is expected to honor the transaction. By invoking some start-transaction interface, the client system expects the target system to be able to rollback everything that is being processed until the stop-transaction interface is called. This is a very tough, cumbersome, and nearly impossible requirement to fulfil for the target system.

  • The client system attempts to both manage and honor the transaction. Within the scope of a desired transaction, the client system keeps track of the various changes it incurs in the target system, and will rollback and/or compensate the various changes that have already been performed in case something fails.

  • The target system provides an explicit aggregated service API for certain customer transactions required by the client system. In this case, the target system attempts to manage and honor the transactional integrity for a specific set of customer transactions, thereby providing appropriate rollbacks and/or compensating actions for these specified transactions.

  • The target system provides an asynchronous request API for the aggregated customer transaction required by the client system. The submitted request will be stored immediately in the database, and the target system can start the processing flow performing the various tasks, compensating already performed tasks if required, thereby even supporting manual interventions. The client system can choose to perform other tasks while checking regularly for an answer, or opt to wait or block until the aggregated service is fully executed. The latter option would emulate a synchronous service call at the client side.

· 2 min read
Herwig Mannaert

Decomposing business process models into state machines, it is quite common to encounter so-called hierarchical workflows, i.e. the creation in a workflow of an instance of another data element, which triggers a secondary workflow. We distinguish two possible scenarios.

  • Fire and forget: a workflow task triggers a secondary flow, which can be processed in an independent way.

  • A workflow task triggers a secondary flow, but the task in the primary flow can only be considered complete when processing is completed in the secondary flow.

In the second scenario, one should be cautious about possible dependencies and the corresponding coupling. More specifically, coupling may surface in two ways.

  • In case the primary workflow and/or task is polling the secondary flow that it has triggered, this could lead to lots of threads and coupling in time between software modules.

  • In case the secondary workflow has to report back to the primary flow upon completion, this could lead to a bidirectional dependency between the two workflows. This could result in a technical issue in case the two workflows belong to a different software component.

In order to avoid coupling and even possible locking in time, NS theory suggests that the secondary workflow needs to report back upon completion. However, to avoid bidirectional coupling between the code of both workflows, this should be done using a loosely coupled mechanism. An example of such a loosely coupled mechanism would be a service interface on the primary workflow. This would allow the primary flow to pass the URL of that service interface as a callback function to the secondary flow, avoiding tight coupling in the source base.

Based on the confirmation or completion message of the secondary workflow, the primary flow will set the corresponding status on the target data element of the primary flow. In this way, the secondary workflow does not need any knowledge of the possible states of the primary flow. In case multiple subflows need to be processed and completed before the primary flow is able to continue, it is again (a task in) the primary flow that needs to have the appropriate knowledge to combine the information provided by the various subflows.