Memory Buffer Documentation

Version: 2.0.0

Retrieved: 2025-10-09 15:16:02

Memory Buffer

Used for making sure that a critical operation succeeds. For example a call to an external service that requires internet connectivity.

The module buffer messages and process one by one until the message is discarded due to many attempts or until success is reported. The actual processing is done by the module that the buffer module sends to. So the module that do the actual processing needs to connect back to the buffer module and report the status of the attempt.

The buffer is not blocking so new messages can always be added until the Max Buffer setting is reached.

Settings

Name	Requirements	Purpose	Default
Max Buffer	Number 1 to Int.Max	The maximum number of messages allowed in the buffer	100
Max Retries	Number -1 to Int.Max	The number of retries before discarding a message. -1 = Infinite retries	5
Back off Algorithm Enabled	Bool	If true the module will automatically back off to avoid flooding	true
Retry Delay	Number 0 to Int.Max	The retry delay in ms between failed attempts	1000
Message Delay	Number 0 to Int.Max	The delay in ms between successful queue processing	0
Show Buffer Info	Bool	If true the the output will include info about the buffer	false
Success Property	String 1 - 64 in length	The property to flag success/failure with	memorybuffer.success

Buffer or Result?

Since the buffer will queue new messages and process other as a result they need to be separated. This is done by using the Success Property on the message and setting it to true/false. You will have to add that property by yourself, this is easily done with the property mapper module.

Example Successful Processing

At least two messages will enter the buffer module to have a successful operation. The first message is the one to be queued and processed, the next message is the result that is sent back from the next module in the flow.

# Settings:
Using default settings

# 1: First incoming message (can be anything):
{
    'data':{
        'temp':465.3,
        'timestamp': '2019-08-10 16:23:44'
    }
}

// The message is sent to the next module for processing and the buffer awaits the result before processing a new message

# 2: Second incoming message (should have the result):
{
    // Some object
    ,
    'memorybuffer':{
        'success':true
    }
}

# 3: Now the message is removed and the next message in the queue (if any) will be processed.

Example Failed Processing

This example show the process when the next module reports a failure. The module will pass a message to the next module 3 times (first try and 2 retries). Then the message will be discarded and the next message in the queue will be processed.

Note that we enabled Show Buffer Info so that we can see some information about the queue (just for debugging).

# Settings:
Max Retries = 2
Retry Delay = 1000
Back off Algorithm Enabled = false
Show Buffer Info = true

# IN 1: First incoming message (can be anything):
{
    'data':{
        'temp':465.3,
        'timestamp': '2019-08-10 16:23:44'
    }
}

# OUT 1: First outgoing message:
{
    'data':{
        'temp':465.3,
        'timestamp': '2019-08-10 16:23:44'
    },
    'memorybuffer':{
        'count': 0,
        'tries': 1,
        'retryDelay': 0,
        'delay': 0
    }
}

# IN 2: Second incoming message (should have the result):
{
    // Some object
    ,
    'memorybuffer':{
        'success':false
    }
}

# OUT 2: Second outgoing message:
{
    'data':{
        'temp':465.3,
        'timestamp': '2019-08-10 16:23:44'
    },
    'memorybuffer':{
        'count': 0,
        'tries': 2,
        'retryDelay': 1000,
        'delay': 0
    }
}

# IN 3: Third incoming message (should have the result):
{
    // Some object
    ,
    'memorybuffer':{
        'success':false
    }
}

# OUT 3: Third outgoing message:
{
    'data':{
        'temp':465.3,
        'timestamp': '2019-08-10 16:23:44'
    },
    'memorybuffer':{
        'count': 0,
        'tries': 3,
        'retryDelay': 1000,
        'delay': 0
    }
}

# IN 4: Second incoming message (should have the result):
{
    // Some object
    ,
    'memorybuffer':{
        'success':false
    }
}

# Now the message is removed (since max retries was reached) and the next message in the queue (if any) will be processed.

Search Documentation

Other Modules