Combine Crosser, InfluxDB and Grafana to build a modern Historian

A lot has changed in recent years, there are new requirements for capturing and analysing process data from machines and there’s also been a tremendous technology evolution. In this blog we will discuss how you can build a modern historian by combining best-of-breed tools in a flexible and efficient way.

What is a Historian?

Historians have been used for a long time to store operational data in the manufacturing and processing industries. The primary use cases have been to visualize data to spot trends and to analyze the data with algorithms to automatically identify issues.

Typical features of a historian:

• Collect data (typically time series sensor data or machine parameters)
• Store data in a database
• Visualise stored data
• Access stored data with preprocessing, e.g. windowing, interpolation and aggregation

A historian may need to store huge amounts of data, both due to the number of sensor time series but also because data may be kept for very long times (several years).

Why do we need a ‘modern’ historian?

Traditional historians are often tightly integrated into other systems on the factory floor, such as DCS and SCADA systems. This limits their use to whatever is needed/supported by these systems. In today’s more and more digitised factory floors both the data sources that may need to be stored in a historian are more heterogeneous and the tools that need access to the data are more diverse, e.g. using machine learning to analyze data to find anomalies. Hence it is advantageous to have a historian that is more loosely coupled with the other systems and more openly available for new data sources as well as new analysis tools.

In addition, technology has evolved which opens up for new ways of implementing solutions to traditional problems. By combining best-of-breed solutions for each sub-function we can build a flexible, cost effective and open historian.

Use Crosser, InfluxDB and Grafana to build a modern historian

Crosser offers a best-of-breed streaming analytics solution that can collect data from a diversity of sources, normalize the data as well as analyze the data with different algorithms and/or machine learning models. By combining this technology with the InfluxDB time series database and the Grafana visualization tool you get a modern and cost-effective historian.

Benefits

• Collect and normalize data from multiple sources by combining pre-built modules into flows using the Crosser graphical FlowStudio tool.
• Store data efficiently in the InfluxDB time series database.
• Visualize data on dashboards created with the powerful Grafana tool.
• Analyze historical data using custom algorithms or machine learning models running on the Crosser edge node.
• Manage the configuration of any number of historians from a central location using the Crosser Cloud service.
• Flexible deployment options that can evolve over time to match your requirements.

Deployment options

The easiest way to deploy a historian with the above technologies is to use Docker containers, where one container hosts the Crosser node and another container the InfluxDB and Grafana tools. In the most basic setup both these containers are deployed on the same host. Crosser provides pre-built deployment templates to quickly setup such a system. You will have a new modern historian up and running in minutes! Deploy it wherever you need to store, visualise and/or analyse data, whether you need a local UI next to the machine or want to analyse data from your whole factory floor.

Using these building blocks it is easy to implement other architectures to adapt to your specific needs. For example, if you already have Crosser Nodes deployed next to your machines, or if you need to scale your pre-processing capacity, use multiple Crosser Nodes feeding a single instance of InfluxDB/Grafana.

If you need to process your historical data, just add another Crosser Node next to your InfluxDB/Grafana container.

With this kind of setup you can also easily build a cloud based historian by moving the InfluxDB/Grafana container to the cloud while keeping the Crosser Nodes for data collection and preprocessing close the data sources on-premise.

Crosser Streaming Analytics

The Crosser Node is a generic streaming analytics engine that can be configured for different types of data collection and processing by using the FlowStudio tool in Crosser Cloud. Using drag’n drop you combine pre-built modules into flows to define the sequence of operations you want to perform with your data. Crosser offers a comprehensive library of modules for collecting data from different sources, process/analyse data and deliver results both to different on-premise and cloud systems, as well of course to the InfluxDB database.

In a historian setup you would use one flow to capture and pre-process data before storing it in InfluxDB. The data to be captured is defined by this flow. Using the schema less design of InfluxDB there is no need to configure the database. New data points can be added at any time by just updating the flow capturing data.

If you want to analyse the stored data you just add another flow that uses the InfluxDB as the source. The analysis can be as simple as some aggregation and condition logic, but can also use your custom algorithms or machine learning models. The results can either be written back to InfluxDB for presentation on your dashboards, or be sent to any system or service you want to use by utilising the extensive library of output modules provided by Crosser.

Summary

By combining Crosser’s advanced streaming analytics solution with the InfluxDB time series database and the Grafana visualisation tool you get a flexible, open and cost-effective historian. Collect data from any data source, preprocess new data before storing it and analyse historical data with the same flow-based processing engine. Manage one or several historians from a central place in Crosser Cloud. Scale and live with your new historians for a long time using the flexible deployment options to match your needs and requirements at any time.

Learn more about combining Crosser, InfluxDB and Grafana (Video) → 

Understand how to transform IoT data (Video) →