Events

What are events and how should we treat them?

An event is a message used in asynchronous communication. Usually they represents something that has happened in a system or a domain, but they can also act as requests for change in asynchronous transactions. Events are used to communicate changes of state between different components or services in an event-driven architecture. Events can be classified into two types: fact events and command events.

A fact event is an event that describes something that has already occurred and cannot be undone. For example, “a customer placed an order” or “a payment was processed”. A fact event does not imply any action or expectation from the receiver. It is simply a notification of what happened.

A command event is an event that requests or instructs something to happen. For example, “create a new account” or “cancel an order”. A command event implies an action or expectation from the receiver. It is usually sent by a source that has the authority or responsibility to initiate the action.

The difference between a fact event and a command event is mainly in the intention and semantics of the message. A fact event informs about the past, while a command event directs the future. A fact event can be published to multiple subscribers, while a command event should be sent to a specific target, acting as an API call to the target. A fact event should be idempotent and immutable, while a command event may have side effects and dependencies.

Events are together with gRPC and REST interfaces considered be the APIs of an application and should be documented and treated as contracts.

Events are usually not distributed directly between services but are usually published to an event bus that enables a publish and subscribe distribution method. This enables event distributions to become one-to-many and decouples the publishers from consumers. Popular event buses are Apache Kafka, NATS and RabbitMQ to mention a few.

Events are the key component to scalable and decoupled Event Driven Services (:mailbox_with_no_mail: Event Driven Services). They can be used to define the bounded context, scope the responsibility and borders of a service, to in turn give listeners the power to stack up and add functionality in a contract based and scalable way.

Fact event examples

Fact event can further be broken down to notification events and state transfer events.

A Fact event - Notification

A notification event is a minimal intentless event that informs about an occurrence and its related entities. For example, customer 911000100 created order 49. This can be useful when you are not allowed to put certain details in an event, for example PII due to GDPR. Consumers can then call-back using gRPC or REST to get details. You can read more here (🛍️ Notification events and strong coupling)

{
    "type": "com.netigate.responseservice.answer.created",
    "source": "com.netigate.responseservice",
    "id": "A234-1234-1234",
    "time": "2018-04-05T17:31:00Z",
    "data": {
        "personaId": "1234-1234-A234",
        "answerId": "PJ3C-VI41-SM3O"
    }
}

A Fact event - state full transfer

A state transfer event is a more detailed event that either contains the whole object or its changes, so that consumers can update their cache or act accordingly. For example, customer 911000100 changed her billing address from 84 to 48 Baker Street. You can read more here ( :flags: CQRS and state in Event Driven Services)

{
    "specversion": "1.0",
    "type": "com.netigate.responseservice.answer.created",
    "source": "com.netigate.responseservice",
    "id": "A234-1234-1234",
    "time": "2018-04-05T17:31:00Z",
    "data": {
        "personaId": "1234-1234-A234",
        "answer":{
            "id":"PJ3C-VI41-SM3O",
            "questionId": "KN09-C2K1-S30P",
            "surveyId": "S30P-KN09-C2K1-",
            "type": "text",
            "value": "I enjoy using this tool"
        }
    }
}

A Fact event - state delta transfer

{
    "specversion": "1.0",
    "type": "com.netigate.responseservice.answer.updated",
    "source": "com.netigate.responseservice",
    "id": "A234-1234-1234",
    "time": "2018-04-05T17:31:00Z",
    "data": {
        "personaId": "1234-1234-A234",
        "answer": {
            "id": "PJ3C-VI41-SM3O",
            "previous": {
                "value": "I enjoy using this tool"
            },
            "new": {
                "value": "I enjoy using this tool even more now"
            }
        }
    }
}

Command event examples

Command event

Command Events can be useful to trigger an action or job idempotently in the near future, with the guarantee of them running as long as the event bus has accepted the command event. Relieving oneself of implementing retry jobs. An example of this would be publishing an email sendout to a topic and letting the dispatcher pick up jobs in order. Job queue pattern.

They could also be used to rate limit influx of work by a service exposing Command Events as job API. An example would be a survey response BFF publishing events to an ingress topic of a response service. The BFF would publish incoming answers on an event bus and complete its responsibility once published, and the response service can consume events at an acceptable rate, separating load from one to another.

Example of a Command Event writing an answer to a Response Service.

{
    "specversion": "1.0",
    "type": "com.netigate.responseservice.ingress",
    "source": "com.netigate.responsebff",
    "id": "A234-1234-1234",
    "time": "2018-04-05T17:31:00Z",
    "data": {
        "personaId": "1234-1234-A234",
        "type": "CreateAnswer",
        "answer":{
            "questionId": "KN09-C2K1-S30P",
            "surveyId": "S30P-KN09-C2K1-",
            "type": "text",
            "value": "I enjoy using this tool"
        }
    }
}

Command event with reply - Saga

Command Events can also be used to orchestrate complex multistage jobs between services, or for services to implement APIs in event an driven fashion with concurrency. This is commonly done through the Saga pattern. ( :mailbox_with_no_mail: Event Driven Services | Example use cases SAGA )

When implementing a Command Event, particularly in the Saga pattern, the client might expect some sort of Notification Event as part of its success or failure. The reply event should also be defined by the service owning the command. For write requests it can contain a simple result code but can also contain detailed information in the case of a partial success.

Command Events can also be used to request data in an asynchronous way, with the reply containing the query results.

Command:

{
    "specversion": "1.0",
    "type": "com.netigate.reportservice.getreports",
    "source": "com.netigate.reportbff",
    "id": "A234-1234-1234",
    "time": "2018-04-05T17:31:00Z",
    "data": {
        "reportId": "1234-1234-A234",
        "tenantId": "contoso",
        "replyTopic": "com.netigate.reportbff.private.reply"
    }
}

Reply on client private topic:

{
    "specversion": "1.0",
    "type": "com.netigate.reportbff.private.reply",
    "source": "com.netigate.reportservice",
    "id": "A234-1234-1234",
    "time": "2018-04-05T18:31:00Z",
    "data": {
        "reportId": "1234-1234-A234",
        "tenantId": "contoso",
        "reportContent": ....
    }
}

Anatomy and implementation

Most consumers will perform a few common tasks when receiving events, understanding who sent it, when it was sent, what type and version it etc. To standardize these fields and make them known across services and implementations, it is a good practices to pick a format and re-use it for every service you write. You might not need to know how to handle an event your receive, but you should figure out if it is relevant to you.

Cloud events base

To keep things simple and not reinvent the wheel, we propose using the Cloud Event specification for defining your events. This specification is maintained by the Cloud Native Computing Foundation and ensure compatibility with a number of open source projects and 3rd party software. Further reading: CloudEvents

The specification declares event shape across REST, gRPC, Kafka and other implementations. Specification: https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/formats/json-format.md#32-examples

At Netigate, if you are using the netigate/platform.eventbus you get this transformation out of the box.

Json example

{
    "specversion" : "1.0",
    "type" : "com.example.someevent",
    "source" : "/mycontext",
    "subject": null,
    "id" : "C234-1234-1234",
    "time" : "2018-04-05T17:31:00Z",
    "datacontenttype" : "application/json",
    "data" : {
        "appinfoA" : "abc",
        "appinfoB" : 123,
        "appinfoC" : true
    }
}

Kafka example

------------------ Message -------------------

Topic Name: mytopic

------------------- key ----------------------

Key: mykey

------------------ headers -------------------

content-type: application/cloudevents+json; charset=UTF-8

------------------- value --------------------

{
    "specversion" : "1.0",
    "type" : "com.example.someevent",
    "source" : "/mycontext/subcontext",
    "id" : "1234-1234-1234",
    "time" : "2018-04-05T03:56:24Z",
    "datacontenttype" : "application/xml",

    ... further attributes omitted ...

    "data" : {
        ... application data encoded in XML ...
    }
}

-----------------------------------------------

References

Internal:

External: