Disabling queues when testing
To handle bursty message rates and different processing times in modules, each module has a message queue at the input. This is the default way of operating and also in most cases the best way to deal with common scenarios. There are different ways you can tweak how these queues operate, but we leave that for another time. If you want to know more about message queues, take a look here. For now we will focus on one use case: debugging.
If you have a Flow where some modules take much longer times to process messages, e.g. modules communicating with external systems or code modules running complex logic, messages may queue up at the input of the ‘slow’ modules. This is usually not a problem, but in some cases it may make debugging difficult.
Let’s take a look at an example:
Here we start by getting 100 tags from a Modbus PLC every second. It will come out as an array with the 100 values and associated names and other metadata. Then we want to write each tag into a separate row in a SQL database. The next step is then to loop over the tags array, add a timestamp to each tag and map the data to the column names in the database.
The problem you may end up with when trying to debug a Flow like this is that the ‘Loop over tags’ operation is very fast, while writing each tag to the database most likely is much slower. Hence you will end up with a lot of messages in the input queue on the database module. It will still work as expected, as long as the database can write these 100 rows in one second. But if some of the messages fail when being processed by the modules in front of the database module it may be tricky to find which one is causing the problem, because when looking in the debug window you will most likely see ~100 messages from the Timestamp module, followed by ~100 messages from the Property Mapper module and then a few messages from the database module. It is therefore difficult to follow the sequence of messages being processed.
For scenarios like this it can be useful to turn off message queueing while testing. You can turn off queues for the whole Flow in the Flow Settings (panel on the right-hand side of the Flow Studio), or for individual modules on the General tab in the module settings. Make sure to turn on queues again before you deploy the Flow, since this is in almost all cases the best setup. With queues disabled modules will only accept one message at a time. Previous modules need to wait until the next module has finished processing a message (back-pressure). This will allow you to follow the sequence of operations applied to each message and find out which message that causes problems.
The debug window will only show a limited number of messages, so if you are processing large arrays the messages of interest may disappear before you could check them. Then a final tweak is to temporarily add a Delay module before the module of interest, to slow down the whole processing and allow you to monitor the messages as they are processed.
