Event-Driven mechanisms

Just like in event-driven programming, which usually revolves around user clicking events, automated tasks could be driven by events, as long as the events and executed tasks share a common framework. For this reason, I personally recommend using a Message Queue framework still, and simply proxy your event cues through the creation of Message Queue items on the events. Depending on how you setup your metronome, events can be processed almost instantly even through a Message Queue. Albeit, ours does include a bit of latency for the peer-to-peer concepts behind our automation infrastructure.

Reason being that most of the time, when automating enterprise systems, you’ll have to deal with disparate types of systems. Windows, Unix, and their different flavors, and even, loaded environments can pose a compatibility challenge, and because one needs to rely on the shared common libraries, it can become quite cumbersome to port internal coding libraries across different environments and even languages. Event-driven programming across different systems is quite cumbersome. Even if the common denominator is a REST-style API served over an HTTP port, each process requires its webserver, which is an additional exploit vector for free.

A Message Queue, for being accessible through simple database queries, remains accessible in most environments with minimal additional dependencies. And, if you really want to get rid of database dependencies in your enterprise programming projects, then you can also abstract the Message Queue system through a REST API, or using a pre-built library like RabbitMQ, which is already ported to many different languages and doesn’t depend on a database as per say (but it does depend on extra network services). Or if you want to live on the wild side, a self-sharding data cluster service like ElasticCache.

Also, event-driven execution doesn’t cleanly resolve chained-dependencies situations, making those a lot harder to implement, monitor and tune, without a messaging queue in place.

Therefore, event-driven programming for automation is not exactly the right tool. We’ll see how we implement such event-driven Bots on the SSH file reception servers, reacting to new files arriving for processing. But it is a grizzly task to integrate such event detection.

Automated tasks are more procedural in their nature, and it becomes very hard to manage “events” on a network. The normal way of doing the later is to translate the event-message-bus on a message queue, like Microsoft does internally to run the user experience in Windows, most of the time, we’ll implement a message queue in the backend of an application stack to process event-driven functionality, so really, we’re back to square 1 in terms of coding lines and associated risks.

In an automation context, events to be considered are normally file events as well (a received file, or a disk full for examples). These types of events are, as of today, still badly integrated in the different OSes and application stacks. For example, it’d be easy to program file-events under Samba, but a lot harder under SSH. And the other negative aspect of handling file-events is the exploit vector we’re opening, reacting to incoming files which could be manipulated by hackers. You’ll see how our Message Queue-driven approach also resolves security risks associated with automation.

Another good example of a thorny event-driven situation is the typical JavaScript UI event cascading that we need to program in order to properly chain events. If any of the event’s result turns out to be errored out, the rest of the chain gets short-circuited and special exit handling must be programmed at each step to properly generate an audit trail for debugging. Forgetting to save this audit trail, or not catching an error properly can generate grievous discrepancies in an automated system where events must be chained.

But, the point of this book being to implement a system which yourself can build on, and access through SQL, securely with peace of mind, our preferred approach will be the home-grown relational Message Q to store & distribute event-driven messages.