Changes for page MQTT Forward Instruction
Last modified by Kilight Cao on 2024/08/31 16:26
Summary
-
Page properties (1 modified, 0 added, 0 removed)
Details
- Page properties
-
- Content
-
... ... @@ -1,13 +1,18 @@ 1 - =INTRODUCTION=1 +**~ Contents：** 2 2 3 +{{toc/}} 4 + 5 + 6 += 1. Introduction = 7 + 3 3 Dragino LoRa/LoRaWAN gateway support MQTT forwarding. It can forward the sensor data from LoRa network to MQTT server , and vice verse. 4 4 5 -== Support Devices == 10 +== 1.1 Support Devices == 6 6 7 7 This MQTT forward instruction is for below devices: 8 8 9 9 * Firmware Version > LG02_LG08-5.3.1580178039 [[Firmware Download>>url:http://www.dragino.com/downloads/index.php?dir=LoRa_Gateway/LPS8/Firmware/Release/]] 10 -* LG01N, OLG01N (Warning: LG01-P LG01-S use another instruction: [[MQTT for LG01-P/LG01S>>url:https://wiki.dragino.com/index.php/Through_MQTT_to_upload_data]]) 15 +* LG01N, OLG01N ((% class="mark" %)**Warning**(%%): LG01-P LG01-S use another instruction: [[MQTT for LG01-P/LG01S>>url:https://wiki.dragino.com/index.php/Through_MQTT_to_upload_data]]) 11 11 * LG02, OLG02 12 12 * LG308, DLOS8 13 13 * LPS8 ... ... @@ -15,65 +15,79 @@ 15 15 * MS14 series if installed with the same firmware. (in this case, the MQTT forward will work , but no LoRa support) 16 16 17 17 18 - \\23 += 2. Firmware Change Log for MQTT feature = 19 19 20 -= Firmware Change Log for MQTT feature = 21 - 25 +((( 22 22 This instruction is wrote start from LG02_LG08-5.3.1580178039. Below is related change log since this version of firmware. 27 +))) 23 23 24 24 * LG02_LG08-5.3.1580178039 25 25 ** Initiate version 26 26 27 27 28 - \\33 += 3. MQTT forward operating principle = 29 29 30 -= MQTTforwardoperatingprinciple =35 +== 3.1 Network Structure == 31 31 32 -== Network Structure == 33 - 34 34 Below shows the network structure for MQTT forwarding. 35 35 36 36 * For Uplink: The sensor sends data to LoRa Gateway via LoRa wireless, The gateway will process these data and forward to remote MQTT Broker via Internet. 37 37 * For Downlink: The gateway subscribe a topic in the MQTT broker, when there is update on the topic, the gateway will know and broadcast the data to Local LoRa network, 38 38 39 -[[ ~[~[image:https://wiki.dragino.com/images/thumb/4/45/MQTT_Forward_1.png/600px-MQTT_Forward_1.png~|~|height="348" width="600"~]~]>>url:https://wiki.dragino.com/index.php/File:MQTT_Forward_1.png]]42 +[[image:https://wiki.dragino.com/images/thumb/4/45/MQTT_Forward_1.png/600px-MQTT_Forward_1.png||height="348" width="600"]] 40 40 41 41 General MQTT structure 42 42 43 -== How sensor data is forwarded == 44 44 47 +== 3.2 How sensor data is forwarded == 48 + 45 45 In this MQTT forward feature, the key point is how the gateway process the sensor data. 46 46 47 -=== Upstream === 48 48 52 +=== 3.2.1 Upstream === 53 + 49 49 Assume there are two sensor nodes, their ID are Node1 ID: 6734 , Node2 ID: 7456. In the remote MQTT broker there are two topics: Topic1: /channel/765800, Topic2: /channel/367860. We can set up in the gateway to map Node1 to Topic1 and Node2 to Topic2. So when there is a sensor data from Node1, the gateway will forward the data to Topic1, when there is sensor data from Node2, the gateway will forward to Topic2. 50 50 56 +((( 51 51 The data flow works as below diagram. 58 +))) 52 52 53 53 [[~[~[image:https://wiki.dragino.com/images/thumb/2/2a/MQTT_Forward_2.png/600px-MQTT_Forward_2.png~|~|height="355" width="600"~]~]>>url:https://wiki.dragino.com/index.php/File:MQTT_Forward_2.png]] 54 54 62 +((( 55 55 Upstream path 64 +))) 56 56 66 +((( 57 57 Note: The sensor data can base or LoRa or other method, as long as there are data on the file /var/iot/channels. /span> 68 +))) 58 58 59 -=== Downstream === 70 +((( 71 + 72 +))) 60 60 74 +=== 3.2.2 Downstream === 75 + 61 61 The gateway subscribes to a topic of the remote MQTT broker topic. When there is some one publish a value on this topic. The gateway will get it and broadcast to local LoRa Network. 62 62 78 +((( 63 63 Below are the data flow for downstream. 80 +))) 64 64 65 -[[ ~[~[image:https://wiki.dragino.com/images/thumb/3/3a/MQTT_Forward_3.png/600px-MQTT_Forward_3.png~|~|height="368" width="600"~]~]>>url:https://wiki.dragino.com/index.php/File:MQTT_Forward_3.png]]82 +[[image:https://wiki.dragino.com/images/thumb/3/3a/MQTT_Forward_3.png/600px-MQTT_Forward_3.png||height="368" width="600"]] 66 66 67 67 Downstream path 68 68 69 69 70 -== Macro Definition == 87 +== 3.3 Macro Definition == 71 71 72 72 The MQTT publish command use Macro settings to generate flexible upstream payload for MQTT publish. 73 73 74 -Currently the -t (topic) and -m (message) support Macros. 91 +((( 92 +Currently the (% class="mark" %)**-t (topic)**(%%) and (% class="mark" %)**-m (message)**(%%) support Macros. 93 +))) 75 75 76 -=== -t topic macro === 95 +=== 3.3.1 -t topic macro === 77 77 78 78 * CHANNEL: Remote Channel ID 79 79 * CLIENTID: Client ID , Same as -i ... ... @@ -81,7 +81,7 @@ 81 81 * USERNAME: User ID (-u) 82 82 * HOSTNAME: Device Hostname 83 83 84 -=== -m message macro === 103 +=== 3.3.2 -m message macro === 85 85 86 86 * HOSTNAME: Device Hostname 87 87 * CHANNEL: Remote Channel ID ... ... @@ -89,9 +89,9 @@ 89 89 * META: Completely sensor data with time stamp and rssi 90 90 * JSON: Convert META to json format. 91 91 92 -=== Example for Macro === 111 +=== 3.3.3 Example for Macro === 93 93 94 -[[ ~[~[image:https://wiki.dragino.com/images/thumb/c/c7/MQTT_Command_9.png/600px-MQTT_Command_9.png~|~|height="385" width="600"~]~]>>url:https://wiki.dragino.com/index.php/File:MQTT_Command_9.png]]113 +[[image:https://wiki.dragino.com/images/thumb/c/c7/MQTT_Command_9.png/600px-MQTT_Command_9.png||height="385" width="600"]] 95 95 96 96 MQTT Publish configure 97 97 ... ... @@ -102,151 +102,156 @@ 102 102 103 103 When there is a LoRa sensor arrive. it will be store at the /var/iot/channels as below: 104 104 105 -[[ ~[~[image:https://wiki.dragino.com/images/thumb/c/c4/MQTT_Command_20.png/600px-MQTT_Command_20.png~|~|height="325" width="600"~]~]>>url:https://wiki.dragino.com/index.php/File:MQTT_Command_20.png]]124 +[[image:https://wiki.dragino.com/images/thumb/c/c4/MQTT_Command_20.png/600px-MQTT_Command_20.png||height="325" width="600"]] 106 106 107 107 Sensor Data 108 108 109 -According to above macro. Gateway will publish field1=22.0&field2=49.0 to topic: dragino-1b7060/78901/data, where 78901 is the remote channel for this node ID. 128 +((( 129 +According to above macro. Gateway will publish (% class="mark" %)**field1=22.0&field2=49.0**(%%) to topic: (% class="mark" %)**dragino-1b7060/78901/data**(%%), where 78901 is the remote channel for this node ID. 130 +))) 110 110 111 111 112 - \\133 +== 3.4 Modify the MQTT to support more options == 113 113 114 - 115 -== Modify the MQTT to support more options == 116 - 117 117 The MQTT Client Utility used in Dragino is **mosquitto_pub** and **mosquitto_sub**. User can add more options to the mqtt commands. User can check the valid options by command mosquitto_pub ~-~-help. as below: 118 118 119 -{{{root@dragino-1ec39c:~# mosquitto_pub --help 137 +(% class="box" %) 138 +((( 139 +root@dragino-1ec39c:~~# mosquitto_pub ~-~-help 120 120 mosquitto_pub is a simple mqtt client that will publish a message on a single topic and exit. 121 121 mosquitto_pub version 1.6.4 running on libmosquitto 1.6.4. 122 - 142 + 123 123 Usage: mosquitto_pub {[-h host] [-p port] [-u username] [-P password] -t topic | -L URL} 124 124 {-f file | -l | -n | -m message} 125 - [-c] [-k keepalive] [-q qos] [-r] [--repeat N] [--repeat-delay time] 145 + [-c] [-k keepalive] [-q qos] [-r] [~-~-repeat N] [~-~-repeat-delay time] 126 126 [-A bind_address] 127 127 [-i id] [-I id_prefix] 128 - [-d] [--quiet] 148 + [-d] [~-~-quiet] 129 129 [-M max_inflight] 130 130 [-u username [-P password]] 131 - [--will-topic [--will-payload payload] [--will-qos qos] [--will-retain]] 132 - [{--cafile file | --capath dir} [--cert file] [--key file] 133 - [--ciphers ciphers] [--insecure] 134 - [--tls-alpn protocol] 135 - [--tls-engine engine] [--keyform keyform] [--tls-engine-kpass-sha1]] 136 - [--psk hex-key --psk-identity identity [--ciphers ciphers]] 137 - [--proxy socks-url] 138 - [--property command identifier value] 151 + [~-~-will-topic [~-~-will-payload payload] [~-~-will-qos qos] [~-~-will-retain]] 152 + [{~-~-cafile file | ~-~-capath dir} [~-~-cert file] [~-~-key file] 153 + [~-~-ciphers ciphers] [~-~-insecure] 154 + [~-~-tls-alpn protocol] 155 + [~-~-tls-engine engine] [~-~-keyform keyform] [~-~-tls-engine-kpass-sha1]] 156 + [~-~-psk hex-key ~-~-psk-identity identity [~-~-ciphers ciphers]] 157 + [~-~-proxy socks-url] 158 + [~-~-property command identifier value] 139 139 [-D command identifier value] 140 - mosquitto_pub --help 141 - 142 - 160 + mosquitto_pub ~-~-help 161 + 162 +-A : bind the outgoing socket to this host/ip address. Use to control which interface 143 143 the client communicates over. 144 - 145 - 146 - 147 - 148 - 149 - 164 +-d : enable debug messages. 165 +-D : Define MQTT v5 properties. See the documentation for more details. 166 +-f : send the contents of a file as the message. 167 +-h : mqtt host to connect to. Defaults to localhost. 168 +-i : id to use for this client. Defaults to mosquitto_pub_ appended with the process id. 169 +-I : define the client id as id_prefix appended with the process id. Useful for when the 150 150 broker is using the clientid_prefixes option. 151 - 152 - 153 - mqtt(s)://[username[:password]@]host[:port]/topic 154 - 155 - 156 - 157 - 158 - 159 - 160 - 161 - 162 - 163 - 164 - 165 - 171 +-k : keep alive in seconds for this client. Defaults to 60. 172 +-L : specify user, password, hostname, port and topic as a URL in the form: 173 + mqtt(s):~/~/[username[:password]@]host[:port]/topic 174 +-l : read messages from stdin, sending a separate message for each line. 175 +-m : message payload to send. 176 +-M : the maximum inflight messages for QoS 1/2.. 177 +-n : send a null (zero length) message. 178 +-p : network port to connect to. Defaults to 1883 for plain MQTT and 8883 for MQTT over TLS. 179 +-P : provide a password 180 +-q : quality of service level to use for all messages. Defaults to 0. 181 +-r : message should be retained. 182 +-s : read message from stdin, sending the entire input as a message. 183 +-t : mqtt topic to publish to. 184 +-u : provide a username 185 +-V : specify the version of the MQTT protocol to use when connecting. 166 166 Can be mqttv5, mqttv311 or mqttv31. Defaults to mqttv311. 167 - 168 - 169 - 170 - 171 - 187 +~-~-help : display this message. 188 +~-~-repeat : if publish mode is -f, -m, or -s, then repeat the publish N times. 189 +~-~-repeat-delay : if using ~-~-repeat, wait time seconds between publishes. Defaults to 0. 190 +~-~-quiet : (% class="mark" %)don't print error messages. 191 +~-~-will-payload : payload for the client Will, which is sent by the broker in case of 172 172 unexpected disconnection. If not given and will-topic is set, a zero 173 173 length message will be sent. 174 - 175 - 176 - 177 - 194 +~-~-will-qos : QoS level for the client Will. 195 +~-~-will-retain : if given, make the client Will retained. 196 +~-~-will-topic : the topic on which to publish the client Will. 197 +~-~-cafile : path to a file containing trusted CA certificates to enable encrypted 178 178 communication. 179 - 199 +~-~-capath : path to a directory containing trusted CA certificates to enable encrypted 180 180 communication. 181 - 182 - 183 - 184 - 185 - 201 +~-~-cert : client certificate for authentication, if required by server. 202 +~-~-key : client private key for authentication, if required by server. 203 +~-~-keyform : keyfile type, can be either "pem" or "engine". 204 +~-~-ciphers : openssl compatible list of TLS ciphers to support. 205 +~-~-tls-version : TLS protocol version, can be one of tlsv1.3 tlsv1.2 or tlsv1.1. 186 186 Defaults to tlsv1.2 if available. 187 - 207 +~-~-insecure : do not check that the server certificate hostname matches the remote 188 188 hostname. Using this option means that you cannot be sure that the 189 189 remote host is the server you wish to connect to and so is insecure. 190 190 Do not use this option in a production environment. 191 - 192 - 193 - 194 - 195 - 196 - socks5h://[username[:password]@]hostname[:port] 211 +~-~-tls-engine : If set, enables the use of a TLS engine device. 212 +~-~-tls-engine-kpass-sha1 : SHA1 of the key password to be used with the selected SSL engine. 213 +~-~-psk : pre-shared-key in hexadecimal (no leading 0x) to enable TLS-PSK mode. 214 +~-~-psk-identity : client identity string for TLS-PSK mode. 215 +~-~-proxy : SOCKS5 proxy URL of the form: 216 + socks5h:~/~/[username[:password]@]hostname[:port] 197 197 Only "none" and "username" authentication is supported. 198 - 199 -See https://mosquitto.org/ for more information.}}} 218 + 219 +See https:~/~/mosquitto.org/ for more information. 220 +))) 200 200 201 201 202 202 and modify the /usr/bin/mqtt_process.sh script, the location to change the command option is below lines: 203 203 204 -{{{# Call MQTT Publish command 205 - 206 -# 1. Case with User, Password and Client ID present (e.g. Azure) 225 +(% class="box" %) 226 +((( 227 +# Call MQTT Publish command 228 + 229 +# 1. Case with User, Password and Client ID present (e.g. Azure) 207 207 if [ ! -z "$pass" ] && [ ! -z "$user" ] && [ ! -z "$clientID" ]; then 208 - 209 - 210 - 231 +case="1" 232 +mosquitto_pub $D -h $server -p $port -q $pub_qos -i $clientID -t $pub_topic -u $user -P "$pass" $C $cafile $PUB_FLAG "$mqtt_data" 233 + 211 211 # 2. Case with Certificate, Key and ClientID present (e.g. AWS) 212 212 elif [ ! -z "$certfile" ] && [ ! -z "$key" ] && [ ! -z "$clientID" ]; then 213 - 214 - 215 - 236 +case="2" 237 +mosquitto_pub $D -h $server -p $port -q $pub_qos -i $clientID -t $pub_topic ~-~-cert $cert ~-~-key $key $C $cafile $PUB_FLAG "$mqtt_data" 238 + 216 216 # 3. Case with no User, Certificate or ClientID present 217 217 elif [ -z "$user" ] && [ -z "$certfile" ] && [ -z "$clientID" ]; then 218 - 219 - 220 - 241 +case="3" 242 +mosquitto_pub $D -h $server -p $port -q $pub_qos -t $pub_topic $PUB_FLAG "$mqtt_data" 243 + 221 221 # 4. Case with no User, Certificate, but with ClientID present 222 222 elif [ -z "$user" ] && [ -z "$certfile" ] && [ ! -z "$clientID" ]; then 223 - 224 - 225 - 246 +case="4" 247 +mosquitto_pub $D -h $server -p $port -q $pub_qos -i $clientID -t $pub_topic $PUB_FLAG "$mqtt_data" 248 + 226 226 # 5. Case with User and ClientID present, but no Password and no Certificate present 227 227 elif [ -z "$pass" ] && [ -z "$certfile" ] && [ ! -z "$user" ] && [ ! -z "$clientID" ]; then 228 - 229 - 230 - 251 +case="5" 252 +mosquitto_pub $D -h $server -p $port -q $pub_qos -i $clientID -t $pub_topic -u $user $PUB_FLAG "$mqtt_data" 253 + 231 231 # 6. Case with User and Password present, but no ClientID and no Certificate present 232 232 elif [ ! -z "$user" ] && [ ! -z "$pass" ] && [ -z "$clientID" ] && [ -z "$certfile" ]; then 233 - 234 - 235 - 256 +case="6" 257 +mosquitto_pub $D -h $server -p $port -q $pub_qos -t $pub_topic -u $user -P "$pass" $PUB_FLAG "$mqtt_data" 258 + 236 236 # 0. Else - invalid parameters, just log 237 237 else 238 - case="Invalid parameters" 239 - logger "[IoT.MQTT]:Invalid Parameters - mosquitto_pub not called." 240 -fi}}} 261 +case="Invalid parameters" 262 +logger "[IoT.MQTT]:Invalid Parameters - mosquitto_pub not called." 263 +fi 264 +))) 241 241 242 242 243 -= Example to communicate to a simple MQTT server = 267 += 4. Example to communicate to a simple MQTT server = 244 244 245 -== Overview == 269 +== 4.1 Overview == 246 246 247 247 This section is an example to show how to set up LG01-N to communicate with a MQTT server. The MQTT server is a simple utility set up in a local PC. Note: User can set up same server via [[this instruction>>url:http://www.steves-internet-guide.com/install-mosquitto-broker/]]. 248 248 249 -== Simulate via MQTT.fx utility == 273 +== 4.2 Simulate via MQTT.fx utility == 250 250 251 251 The [[MQTT.fx>>url:http://mqttfx.jensd.de/index.php/download]] is a MQTT client tool. We can use this to simulate a MQTT connection to our MQTT broker first to make sure the MQTT broker works. This will also help us understand how it works. 252 252 ... ... @@ -269,7 +269,7 @@ 269 269 Downstream: Subscribe a topic to get downstream 270 270 271 271 272 -== Simulate via Dragino Command Line == 296 +== 4.3 Simulate via Dragino Command Line == 273 273 274 274 For first try of MQTT connection, simulate via command line is recommend, there are many servers / connection type for MQTT. They are using different connection parameters. Simulating the connection via command line will help us rapidly connect to server and debug. 275 275 ... ... @@ -297,7 +297,7 @@ 297 297 298 298 mosquitto_sub 299 299 300 -== Configure Dragino UI for MQTT connection == 324 +== 4.4 Configure Dragino UI for MQTT connection == 301 301 302 302 This chapter are step by step to show to configure the Dragino Menu for MQTT auto connection. 303 303 ... ... @@ -313,7 +313,7 @@ 313 313 314 314 forward to MQTT 315 315 316 -=== Configure the MQTT Client for Upstream === 340 +=== 4.4.1 Configure the MQTT Client for Upstream === 317 317 318 318 Below screenshot is same as the publish command: 319 319 ... ... @@ -347,7 +347,7 @@ 347 347 MQTT Publish 348 348 349 349 350 -=== Configure the MQTT Client for Downstream === 374 +=== 4.4.2 Configure the MQTT Client for Downstream === 351 351 352 352 Below screen shot equal to this subscribe command: 353 353 ... ... @@ -373,13 +373,12 @@ 373 373 downstream simulation 374 374 375 375 376 -\\ 377 377 378 -== Add LoRa support to communicate with remote sensor == 401 +== 4.5 Add LoRa support to communicate with remote sensor == 379 379 380 380 In above section, we have configured the UI to support MQTT upstream and downstream. We can simulate via Linux command. In this section, we will guide how to communicate with remote LoRa End Node for upstream and downstream. 381 381 382 -=== Use LoRa Raw protocol for communication ~-~- For LG01/LG02 === 405 +=== 4.5.1 Use LoRa Raw protocol for communication ~-~- For LG01/LG02 === 383 383 384 384 We can use [[LoRa Shield>>url:http://www.dragino.com/products/lora/item/102-lora-shield.html]] to send LoRa Raw data to Gateway and receive data from gateway. 385 385 ... ... @@ -409,7 +409,7 @@ 409 409 Downstream Data Flow 410 410 411 411 412 -=== Use LoRaWAN Protocol for communication ~-~- For LG308/LPS8/DLOS8 === 435 +=== 4.5.2 Use LoRaWAN Protocol for communication ~-~- For LG308/LPS8/DLOS8 === 413 413 414 414 Since firmware LG02_LG08~-~-build-v5.3.1585192026-20200326-1109, Dragino LoRaWAN gateways support the communication to LoRaWAN ABP end node locally without the need of LoRaWAN server. This feature allow us to integrate MQTT in the gateway to support LoRaWAN to MQTT forwarding or visa verse. 415 415 ... ... @@ -416,7 +416,8 @@ 416 416 When use test this feature, please use the version higher then : LG02_LG08~-~-build-v5.4.1593400722-20200629-1120, in this version, the upload format is changed and readable, which is easier for integration. 417 417 418 418 419 -\\Video Instruction:[[https:~~/~~/youtu.be/qJTY441-t90>>url:https://youtu.be/qJTY441-t90]] 442 + 443 +Video Instruction:[[https:~~/~~/youtu.be/qJTY441-t90>>url:https://youtu.be/qJTY441-t90]] 420 420 421 421 422 422 Step 1: Refer [[Communicate with ABP End Node>>url:https://wiki.dragino.com/index.php/Communication_with_ABP_End_Node]] to know how to set up LG308 to work with LoRaWAN End node. ... ... @@ -472,7 +472,7 @@ 472 472 Notice: The text use for Downstream must meet the requirement from [[LG308 Downstream Payload>>url:https://wiki.dragino.com/index.php/Communication_with_ABP_End_Node#Downstream]] 473 473 474 474 475 -= Example For Different MQTT Servers = 499 += 5. Example For Different MQTT Servers = 476 476 477 477 |=((( 478 478 [[~[~[image:https://wiki.dragino.com/images/thumb/7/73/ThingSpeak1.png/200px-ThingSpeak1.png~|~|alt="ThingSpeak1.png" height="98" width="200"~]~]>>url:https://wiki.dragino.com/index.php/File:ThingSpeak1.png]] ... ... @@ -497,12 +497,11 @@ 497 497 [[AWS Examples>>url:https://wiki.dragino.com/index.php/MQTT_Forward_to_Amazon_AWS-IOT]] 498 498 ))) 499 499 524 += 6. How to Debug = 500 500 501 -= How to Debug = 502 - 503 503 User can login the gateway's console and run **logread -f**. It will shows the output when there is packet arrive.User can see if it is correct. 504 504 505 -= How to ask for Support = 528 += 7. How to ask for Support = 506 506 507 507 If a user still not have trouble making it works. please send a mail to support@dragino.com with the below info: 508 508