The digitization of industrial plants & machines requires reliable ways to transfer data from many different sources to central cloud platforms. Especially in distributed systems with edge devices and sensors, there is the challenge of consolidating heterogeneous data sources efficiently and securely. One possible approach is to use MQTT in combination with an IoT gateway to connect to Microsoft Azure. This article discusses a technically proven architecture for transmitting industrial sensor data via MQTT to the Azure cloud, presents relevant components and highlights challenges and solutions during implementation.

In modern production environments, computers & IoT devices are often used on machines that function as edge devices. These are directly connected to sensors and control systems and consolidate the data at the point where it is collected. Here, initial preprocessing steps can be carried out, such as averaging or filtering measurement values. The data is processed using common industrial protocols such as Modbus or MQTT made available to the outside world. OPC UA or other fieldbuses are also possible, but MQTT is particularly common due to its simplicity and flexibility and was therefore chosen as the messaging protocol for this architecture.

The edge devices send their data via Ethernet to a central gateway — in many cases an industrial embedded device such as a robust Linux-based wireless embedded computer gateway. This device is an IoT gateway platform specifically designed for demanding edge computing applications in industrial and mobile environments. It acts as an interface between the machine/device level and the cloud.

Since the data in our model is already pre-processed, no direct sensor integration is required on the gateway. This simplifies the architecture and enables a clearly defined separation between edge and cloud levels.

MQTT (Message Queuing Telemetry Transport) is a lightweight, open publish-subscribe protocol specifically designed for use in resource-limited environments and unstable network connections. It is particularly suitable for IoT and industrial applications, as it has minimal overhead and can reliably transfer small data packets. The data source, the so-called publisher, sends its data to a central broker, which distributes the messages specifically to subscribing clients (subscribers). By using topic hierarchies and wildcards, data streams can be flexibly subscribed and structured.

This architecture enables decoupled communication and high scalability. Thanks to its simplicity, flexibility, and broad support from industrial gateways and cloud services, MQTT has established itself as the de facto standard for transmitting sensor data — particularly in modern edge-to-cloud scenarios.

Machine data is typically transferred to the cloud not directly from the sensor or edge device, but via a multi-stage data pipeline. It is structured in such a way that it takes modularity, fault tolerance and clear responsibilities into account.

The architecture is divided into three layers — Edge, Gateway and Cloud. The most important components of the edge and gateway layers that pave the way to the cloud are described below:

1. Modbus MQTT conversion (Edge Layer)

Many edge devices use Modbus as a communication channel instead of MQTT. In order to be able to tailor the entire data transfer to MQTT, each Modbus source must be coordinated with an MQTT topic. The conversion consists of the following steps:

• Connect to Modbus servers
• Reading registers according to configuration
• Conversion of 16-bit raw data — depending on byte order, scaling and interpretation — into general data types such as int64, float, bool, string or even JSON
• Publication of converted values on MQTT topics
• Since Modbus itself only has 16-bit registers and does not specify a fixed byte order (Big/Little Endian) nor standardized data formats, a variety of configurations are created, some of which are proprietary. The converter brings these heterogeneous Modbus data sources into a uniform, structured MQTT interface — and thus lays the foundation for consistent further processing in the cloud.

2. MQTT broker (gateway)

The connected MQTT messages are sent on gateway centralized and received by the MQTT broker. This one on Eclipse Mosquitto based service is the central component of the architecture. By decoupling via the publish/subscribe model, new data sources or customers can be easily integrated without changing the existing architecture. that industrial IoT gateway offers sufficient performance to process multiple parallel data streams. In addition to processing MQTT messages, the broker also performs tasks such as authentication via username/password or support for encrypted connections (TLS).

Edge devices publish their data periodically or on an event-based basis on predefined topics. The intervals and data can be very heterogeneous. For example, GPS data is sent at high frequency, whereas sensor data with slow dynamics, such as charging a battery, is only transmitted every 60 seconds. In addition, time-independent signals are also processed in the case of edge-dependent state events.

3rd MQTT Azure connector (gateway)

In order to be able to bridge the MQTT ecosystem and the cloud, an additional service was introduced — the MQTT Azure connector. He subscribes to relevant topics on the MQTT broker and sends the incoming messages using the Azure IoT Hub SDK to the cloud. This SDK authenticates and manages the gateway as an “IoT device” in Azure and allows both telemetry data to be sent and configurations received via the so-called digital twin.

For more demanding scenarios, Azure IoT Edge can also be operated directly on the gateway instead of the SDK. This is a container runtime environment that was developed specifically for IoT devices. Especially with limited storage conditions — as on some embedded gateways — this approach offers decisive advantages:

• Module-based architecture: Docker containers can be orchestrated and versioned via JSON configs
• Local intelligence: AI models can be run directly at the edge
• Message buffering: Temporary caching in case of network outages
• Monitoring and Management: Storage, CPU usage, and network metrics available
• Container lifecycle management: with integrated garbage collector for resource optimization

‍

The implementation of such data transmission is technically demanding. There are a number of hurdles to overcome during implementation:

1. Map high-level data types in Modbus

Modbus only transmits simple 16-bit register values, which makes it difficult to map more complex data types such as GPS coordinates, strings, or status bits. These must be distributed across multiple registers and correctly interpreted — including byte order, scaling factors, and bit masks. Configurable parser logic is necessary to correctly read out the data and pass it on in MQTT format.

2. Memory optimization on embedded devices

The gateway used has limited memory, with the system area being strictly separated from the user data area. To ensure stability, Docker Containers must be specifically stored in data storage and operated with lean images (such as Alpine). Logging and temporary data storage should also be configured to save resources.

3. Testing without real devices

Since the edge devices used were not consistently available, a local test setup with simulated data in Docker was used. Simulating different data sources can be very complex. Nevertheless, this enabled realistic development and early debugging of the entire MQTT route. It was only after a successful laboratory test that the solution was used in the customer's real environment and successfully tested.

4. Gateway selection and platform decision

When choosing the gateway, the decision was made in favour of an open, Linux-based platform, as it offers significantly more flexibility and computing power. Own services such as MQTT Broker or Azure Connector can be freely deployed, which was essential for individual requirements.

Alternatively, gateways with preconfigured features, integrated user interface, and VPN functionality were also considered. These systems are quick to use, but are significantly more limited when it comes to running your own applications or operating containers. In addition, they are usually less robust and only suitable for mobile or industrial scenarios to a limited extent.

Despite the higher configuration effort, the chosen platform offered the necessary openness and future security to implement a modular, adaptable solution.

‍

Typical fields of application for the architecture shown are primarily in industrial manufacturing, logistics and infrastructure systems. A central application scenario is the condition monitoring of machines and systems (condition monitoring). Physical variables such as temperature, pressure, vibration or power consumption are continuously recorded, preprocessed locally on the edge device and then transferred to the cloud via MQTT. In this way, changes in status can be identified at an early stage and well-founded decisions can be made.

Based on this, the architecture also enables predictive maintenance, i.e. predictive maintenance. Historical measurement data, combined with real-time information, serve as the basis for models that predict the optimal maintenance time. This makes it possible to avoid failures, optimize maintenance intervals and plan service deployments in a more targeted manner.

Another area of application is quality monitoring in production. Here, process data such as cycle times, speeds or temperatures are recorded and analyzed in order to draw conclusions about product quality. Deviations from the normal range can be identified in real time, which allows an immediate reaction in the production process and contributes to optimization in the long term.

The combination of MQTT, robust edge gateway and Microsoft Azure provides a powerful, scalable and future-proof architecture for transferring industrial sensor data to the cloud. Practical experience shows that a clean separation between edge, gateway and cloud levels as well as the use of open standards such as MQTT not only enable technical stability, but also long-term expandability. At the same time, implementation requires a deep understanding of the underlying protocols, embedded systems, and cloud services — particularly in data modeling, resource optimization, and security.

In the future, such architectures will continue to gain in importance: With the increasing spread of AI models at the edge, growing amounts of data and increasing requirements for real-time capability and reliability, seamless integration between OT and IT worlds will become a decisive success factor for smart industrial applications. Anyone who invests in scalable, standards-based IoT infrastructures at an early stage lays the foundation for data-driven business models. We support you from the recording of requirements, through implementation, to further operation and support, and feel free to contact us if you have any questions.