Planning the technology stack for an Event Sourcing project

How to plan the infrastructure services for an event sourcing project

During my career, I had an equal number of opportunities for developing new systems and migrating legacy systems into an event sourcing solution. In my opinion, the latter is a lot more complicated. You can get in touch with me if you need help with refactoring a legacy system to use an event sourcing pattern and also to migrate your existing data.

Event store database

Friendly advice: Don't use message brokers such as Kafka as your source of persisted truth. Message brokers are precisely what their name implies, brokers, even if the message stream is cached or logged, the messages should be persisted elsewhere.

Read model database

Also, note that cost is a significant factor for these decisions. Most cloud service providers will charge you for both incoming and outgoing traffic, not just for storage and computing. Thus, hosting multiple databases on separate instances will affect your traffic fees.

Application framework

• The ability to resolve concrete implementations of my dependencies.
• The ability to easily swap all of my concrete implementations with new ones.
• An Anti-corruption layer between incoming requests and the Model.
• Some form of Identity and Authentication for the clients and Users.
• Ability to segregate configurable options using environment variables.

API access

Local development environment

I'm not focussing this series of articles on deployment strategies or even distributed systems. I am not implementing a micro-services approach, but although the system may feel like it is a monolith, in reality, we can deploy every back-end service on separate machines. You can run this system on a Kubernetes Cluster if so needed by merely following this article.

1. I created a new remote Git repository on GitHub.
2. I've set up my local directories for the project.

mkdir -p ~/code/keithmifsud/php-event-sourcing-demo
cd ~/code/keithmifsud/php-event-sourcing-demo

3. I installed Laravel in a directory named code.

composer create-project --prefer-dist laravel/laravel code

It is probably best if you install Laravel from within Docker, but since I'm using an Ubuntu host, it doesn't matter.

4. I initialised Git and added the remote repository as its origin.

cd code
git init
git remote add origin git@github.com:keithmifsud/php-event-sourcing-demo.git

5. I updated the project description in the composer.json file, deleted the readme.md file provided by Laravel and create a new and initial README.md file to describe the purpose of this repository. You may also wish to add the license file.
6. Once done, I committed my changes and pushed them to the remote origin.

git add .
git commit -m "#GH-1 Initial commit containing the Laravel framework."
git push -u origin master

Docker containers for Event Sourcing

version: '3'
services:

  # PHP Runtime Service
  es_demo_php_runtime:
    build:
      context: .
      dockerfile: ./docker/PHP_Dockerfile
    image: digitalocean.com/php
    container_name: es_demo_php_runtime
    restart: unless-stopped
    tty: true
    environment:
      SERVICE_NAME: es_demo_php_runtime
      SERVICE_TAGS: dev
    working_dir: /var/www
    volumes:
      - ./:/var/www
      - ./docker/configuration/php/local.ini:/usr/local/etc/php/conf.d/local.ini
    networks:
      - es-demo-network

  # Nginx Service
  es_demo_nginx_webserver:
    image: nginx:alpine
    container_name: es_demo_nginx_webserver
    restart: unless-stopped
    tty: true
    ports:
      - "8080:80"
      - "4430:443"
    volumes:
      - ./:/var/www
      - ./docker/configuration/nginx/conf.d/:/etc/nginx/conf.d/
    networks:
      - es-demo-network

  # MySQL Service for application data
  es_demo_mysql_app_db:
    image: mysql:5.7.22
    container_name: es_demo_mysql_app_db
    restart: unless-stopped
    tty: true
    ports:
      - "3315:3306"
    environment:
      MYSQL_DATABASE: es_demo_application_db
      MYSQL_ROOT_PASSWORD: secret
      SERVICE_TAGS: dev
      SERVICE_NAME: es_demo_mysql_app_db
    volumes:
      - ./docker/storage/app-data:/var/lib/mysql/
      - ./docker/configuration/mysql/my.cnf:/etc/mysql/my.cnf
    networks:
      - es-demo-network

  # MySQL Service for event store data
  es_demo_mysql_event_store_db:
    image: mysql:5.7.22
    container_name: es_demo_mysql_event_store_db
    restart: unless-stopped
    tty: true
    ports:
      - "3316:3306"
    environment:
      MYSQL_DATABASE: es_demo_event_store_db
      MYSQL_ROOT_PASSWORD: secret
      SERVICE_TAGS: dev
      SERVICE_NAME: es_demo_mysql_event_store_db
    volumes:
      - ./docker/storage/event-store-data:/var/lib/mysql/
      - ./docker/configuration/mysql/my.cnf:/etc/mysql/my.cnf
    networks:
      - es-demo-network

  es_demo_mongo_read_model_db:
    image: mongo:3.4.22-xenial
    container_name: es_demo_mongo_read_model_db
    restart: unless-stopped
    tty: true
    ports:
      - "8081:8081"
    environment:
      MONGO_INITDB_DATABASE: es_demo_read_model_db
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: secret
      SERVICE_NAME: es_demo_mongo_read_model_db
      SERVICE_TAGS: dev
    volumes:
      - ./docker/storage/read-model-data:/data/db
    networks:
      - es-demo-network

# Docker Networks
networks:
  es-demo-network:
    driver: bridge

# Volumes
volumes:
  app-data:
    driver: local
  event-store-data:
    driver: local
  read-model-data:
    driver: local

mkdir -p ./docker/storage/app-data

mkdir ./docker/storage/event-store-data

mkdir ./docker/storage/read-model-data

Running the application

docker-compose up -d

docker-compose exec es_demo_php_runtime php ./vendor/bin/phpunit

Code structure

docker-compose exec es_demo_php_runtime php artisan app:name ESDemo\\Application

Whether you use Eloquent or not in a different architecture, such as Domain-Driven Design (DDD), believe you me, DBAL models are not your business models. When you use a DBAL (a Database Abstraction Layer) to enforce business rules, you will end up querying the database for every check, even if it should already exist in memory.

My Directory structure for enterprise systems in PHP

code
|__ app(... application stuff (like Laravel's things)
|__ src
|   |
|   |
|   |___Infrastructure (Concrete implementations of things)
|   |
|   |___Models
|      |
|      |___Listing
|      |    |        
|      |    |___Commands
|      |    | 
|      |    |___Domain
|      |    |
|      |    |___Events
|      |    |
|      |    |___Exceptions
|      |    |
|      |    |___Handlers
|      |    |
|      |    |___Listeners
|      |    |
|      |    |___Repositories
|      |    |
|      |    |___Specifications
|      |
|      |___Listings (Read model, as detailed below)
|          |___ ... 

1. Commands

2. Domain

3. Events

4. Exceptions

5. Handlers

6. Listeners

7. Repositories

8. Specifications

code
|__ ... application stuff (like Laravel's things)
|__ src
|   |
|   |
|   |___Infrastructure (Concrete implementations of things)
|   |
|   |___Model
|      |
|      |___Listing (Write model, as detailed above)
|      |    |___ ...
|      |
|      |___Listings
|           |
|           |___Exceptions
|           | 
|           |___Projectors
|           |
|           |___ReadModels
|           |
|           |___Repositories

1. Exceptions

2. Projectors

3. Read Models

4. Repositories

PSR-4 Namespacing

    "autoload": {
        "psr-4": {
            "ESDemo\\Application\\": "app/",
            "ESDemo\\": "src/"
        }
        ....
    },

Please note that I'm not using a Container for composer, therefore, I cannot run composer dump-autoload inside Docker. I run this from my host. However, you may wish to either add a Container for Composer or even keep Composer installed in your PHP Runtime container. I discourage pushing Composer to a production server and instead build the Docker Image during my CD process.

Testing

mkdir ./tests/Integration

        <testsuite name="Integration">
            <directory suffix="Test.php">./tests/Integration</directory>
        </testsuite>

    "autoload-dev": {
        "psr-4": {
            "ESDemo\\Tests\\": "tests/"
        }
    },

composer dump-autoload

docker-compose exec es_demo_php_runtime ./vendor/bin/phpunit

Until next time