Using Ecowitt weather sensors often means using their WiFi gateway to process sensors updates. It offers a web interface that allows you to configure several weather services like Weather Underground, Weathercloud and others. But what if you have other plans with your data, like exporting it to your own database?
There are several ways to go about this. My preferred option is to communicate directly with the gateway and retrieve the current sensor data from it. For this purpose, the device offers a local API. Unlike, for example, the Davis WeatherLink Live, Ecowitt does not document an option to retrieve the data via an HTTP request with a neat JSON response. The API documentation requires that you send commands using specifically formatted TCP packets to the device. In other words, the gateway has its own communication protocol that requires a more low level approach than your regular API interaction using HTTP clients.
An HTTP solution?
However, before I get any further into this, I want to mention that there is currently a way to avoid all this and get the data from the gateway via HTTP request resulting in a JSON response. The gateway web interface itself uses an endpoint called /get_livedata_info, which is used by the Live Data view to show the current sensor states.
The local IP address of my GW1100 gateway is 192.168.1.5. When I fire a GET request at http://192.168.1.5/get_livedata_info, I receive the following JSON object:
{
"common_list": [
{
"id": "0x02",
"val": "6.3",
"unit": "C"
},
{
"id": "0x07",
"val": "88%"
},
{
"id": "3",
"val": "6.3",
"unit": "C"
},
{
"id": "0x03",
"val": "4.5",
"unit": "C"
},
{
"id": "0x04",
"val": "3.9",
"unit": "C"
},
{
"id": "0x0B",
"val": "3.2 m/s"
},
{
"id": "0x0C",
"val": "4.1 m/s"
},
{
"id": "0x19",
"val": "6.6 m/s"
},
{
"id": "0x15",
"val": "13.40 w/m2"
},
{
"id": "0x17",
"val": "0"
},
{
"id": "0x0A",
"val": "207"
}
],
"rain": [
{
"id": "0x0D",
"val": "0.8 mm"
},
{
"id": "0x0E",
"val": "0.0 mm/Hr"
},
{
"id": "0x10",
"val": "1.3 mm"
},
{
"id": "0x11",
"val": "3.6 mm"
},
{
"id": "0x12",
"val": "1.3 mm"
},
{
"id": "0x13",
"val": "3.6 mm",
"battery": "0"
}
],
"wh25": [
{
"intemp": "15.2",
"unit": "C",
"inhumi": "55%",
"abs": "1012.4 hPa",
"rel": "1012.4 hPa"
}
],
"ch_aisle": [
{
"channel": "7",
"name": "",
"battery": "0",
"temp": "14.4",
"unit": "C",
"humidity": "58%"
}
]
}In my current firmware version (GW1100A_V2.1.3), this object contains 4 arrays with data which are directly related to 4 components of the live view in the gateway web interface.
The key ‘common_list’ contains the data contained in the top tiles. Data like Outdoor temperature, Wind speed, Solar radiation and such. The second array is ‘wh25’. It contains the sensor data from the gateway itself. Indoor temperature, humidity and pressure. The array ‘rain’ contains all rain data. Then there is ‘ch_aisle’. I cannot be sure what this array look like at any time, but right now it contains the temperature/humidity data from my sensor at custom channel 7.
The only question is how to interpret the id’s as they appear in the common_list and the rain data. We need the API documentation for this. If we look at the id’s for the rain data, they are all defined in the documentation, starting on page 7.
The same applies to all other single byte identifiers that are received from the gateway. They all have a corresponding definition in the documentation.
Using this HTTP endpoint could be the simpler solution for many. However, the downside is that this option is not documented. The problem with using undocumented solutions is that you have no guarantee the developers at Ecowitt might not introduce a change in the next firmware update that will break it. If you are OK with that risk, using the get_livedata_info endpoint could be the way to go.
The documented solution
If you prefer using the solution contained in the official API documentation, things just got slightly more complex. I’ve written some code in Java in order to communicate with my GW1100. There are some public repositories out there that do the same in Python, NodeJS or other languages. Below I will share snippets of my code in order to illustrate parts of the process.
To get straight to the point: getting a response from the gateway requires you to send a command to port 45000 of the device. This command is encoded in a sequence of bytes in a very specific format. Page 4 of the documentation describes this format. It contains the following elements:
- A fixed header of 2 bytes
- The command being sent, which is a single byte
- The size, excluding the fixed header
- An optional payload
- A checksum, which is the sum of all the bytes sent, excluding the fixed header
The fixed header
The fixed header should be prefixed to any byte sequence sent to the gateway. Its value is documented as 0xFFFF. In my code this boils down to this simple 2 byte array:
byte[] fixedHeader = new byte[]{(byte) 0xff, (byte) 0xff};
The command
The gateway can process quite a few commands. They are defined on page 4 and 5 of the documentation. Many of the commands will likely not be relevant for your situation. In my case, I am only interested in a single command: reading the live data. The name of this command is CMD_GW1000_LIVEDATA, and it is represented by byte 39, or (in its hexadecimal representation) 0x27. In Java:
byte command = (byte) 0x27;
The size and payload
The size is the number of bytes sent, excluding the header. Since the command, the size and the checksum are all always 1 byte each, the only variable we deal with in the payload length. Requesting the live data from the gateway does not require a payload, so its length is 0 and our size is 3. Should you be sending a command that contains payload, the size is calculated like this (assuming that payload is an array of bytes):
byte size = (byte) (3 + payload.length);
Note that in this case we are casting a plain number to a byte in stead of its hexadecimal equivalent.
The checksum
The checksum is obtained by aggregating the byte values of the command, the size and any optional payload bytes. When requesting the live data, we are sending 0x27 or byte 39. Our size is 3, so we are sending byte 0x03 or simply 3. This makes our checksum 39 + 3 = 42 or 0x2A. A simple function calculating the checksum is this:
private static int calcChecksum(byte[] body) {
int checksum = 0;
for (byte b : body) {
checksum += b;
}
checksum = checksum % 256;
return checksum;
}
Note the modulus 256 near the end. The size of the checksum must be 1 byte. Should our int value exceed 255, we would need a second byte to represent that number (since a single byte can only represent 2^8 = 256 values, 0 to 255). In that case, we only send the least significant byte.
When joining all the above, this results in a request body containing the following 5 bytes: 0xFF 0xFF 0x27 0x03 0x2A.
Or, in decimal representation: 255 255 39 3 42.
In Java, I use the java.nio.channels.AsynchronousSocketChannel class to connect with the gateway and send and receive data. Below is a partial class containing a connect function which receives an IP address (the IP address of the gateway) and a port (which should be 45000) to connect to.
import java.nio.channels.AsynchronousSocketChannel;
...
private AsynchronousSocketChannel client;
public void startConnection(String ip, int port) throws IOException, ExecutionException, InterruptedException {
client = AsynchronousSocketChannel.open();
InetSocketAddress hostAddress = new InetSocketAddress(ip, port);
client.connect(hostAddress).get();
}
...A Java method to send the actual command and wait for a response could look like this:
public byte[] sendCommand(byte[] msg) throws TimeoutException, ExecutionException, InterruptedException {
client.write(ByteBuffer.wrap(msg));
ByteBuffer bb = ByteBuffer.allocate(1024);
client.read(bb).get(10, TimeUnit.of(ChronoUnit.SECONDS));
return Arrays.copyOfRange(bb.array(), 0, bb.position());
}In this method, the client.write() method is called, passing the byte array to it. After sending the data, a ByteBuffer is created with a size of 1024, which is more than enough to contain any response we will be getting from the gateway. Then, client.read() will read any response into the ByteBuffer. The read() method returns a Future. A timeout value of 10 seconds is passed to the get() method. Should the Future not resolve within that time (that is, should we get no response), a TimeoutException is thrown. If a response is received within the 10 seconds, the resulting bytes are copied to an array with a size fitting the length of the response.
Decoding the response
If all goes well, the sendMessage() method just returned a tidy byte array containing all our live data. Now we need to decode it. A typical response in hexadecimal format looks like this:
0xFF 0xFF 0x27 0x00 0x45 0x01 0x00 0x9B 0x06 0x37 0x08 0x27 0xA7 0x09 0x27 0xA7 0x02 0x00 0x35 0x07 0x58 0x0A 0x00 0xEA 0x0B 0x00 0x0B 0x0C 0x00 0x0F 0x15 0x00 0x00 0x00 0x00 0x16 0x00 0x00 0x17 0x00 0x20 0x00 0x9A 0x28 0x3A 0x19 0x00 0x24 0x0E 0x00 0x00 0x10 0x00 0x19 0x11 0x00 0x30 0x12 0x00 0x00 0x00 0x19 0x13 0x00 0x00 0x00 0x30 0x0D 0x00 0x00 0x3B
As the documentation states on page 26, the first 5 bytes and the last byte do not contain any actual live data. The first 2 bytes are recognizable as the fixed header, the third byte is the command that was sent and the other 2 bytes represent the size. The last byte in the array is the checksum (0x3B).
The bytes in between contain the relevant data. Each byte marking the data type is followed by a number of bytes representing the corresponding value. For example, the 6th byte is 0x01. On pages 7 – 10 the meaning of each byte is defined as well as the byte count of the value. 0x01 is Indoor temperature.
The byte count is 2. That means that the 7th and 8th byte contain the value for the indoor temperature. These bytes are 0x00 and 0x9B. Converting that value results in 155, or 15,5 degrees Celsius. The 9th byte is 0x06, which resolves to Indoor humidity with a byte count of 1. That means that the humidity percentage is represented by 0x37 or 55. This procedure of taking the marker byte and then the number of bytes associated with that data type continues until we reach the final byte of the array (the checksum byte).
In order to process all possible values, we need to make our code aware of the definitions (marker byte and byte count) for each type. For this purpose, I created a simple record called ApiDefinition.
public record ApiDefinition(byte marker, String name, int length) {}I hard coded all the definitions into my decoder class. Since some readers may want to use it, here it the complete static block initializing a map of definitions.
public static final Map<Byte, ApiDefinition> API_DEFINITIONS;
static {
List<ApiDefinition> apiDefs = new ArrayList<>();
apiDefs.add(new ApiDefinition((byte) 0x01, "INTEMP", 2));
apiDefs.add(new ApiDefinition((byte) 0x02, "OUTTEMP", 2));
apiDefs.add(new ApiDefinition((byte) 0x03, "DEWPOINT", 2));
apiDefs.add(new ApiDefinition((byte) 0x04, "WINDCHILL", 2));
apiDefs.add(new ApiDefinition((byte) 0x05, "HEATINDEX", 2));
apiDefs.add(new ApiDefinition((byte) 0x06, "INHUMI", 1));
apiDefs.add(new ApiDefinition((byte) 0x07, "OUTHUMI", 1));
apiDefs.add(new ApiDefinition((byte) 0x08, "ABSBARO", 2));
apiDefs.add(new ApiDefinition((byte) 0x09, "RELBARO", 2));
apiDefs.add(new ApiDefinition((byte) 0x0A, "WINDDIRECTION", 2));
apiDefs.add(new ApiDefinition((byte) 0x0B, "WINDSPEED", 2));
apiDefs.add(new ApiDefinition((byte) 0x0C, "GUSTSPEED", 2));
apiDefs.add(new ApiDefinition((byte) 0x0D, "RAINEVENT", 2));
apiDefs.add(new ApiDefinition((byte) 0x0E, "RAINRATE", 2));
apiDefs.add(new ApiDefinition((byte) 0x0F, "RAINHOUR", 2));
apiDefs.add(new ApiDefinition((byte) 0x10, "RAINDAY", 2));
apiDefs.add(new ApiDefinition((byte) 0x11, "RAINWEEK", 2));
apiDefs.add(new ApiDefinition((byte) 0x12, "RAINMONTH", 4));
apiDefs.add(new ApiDefinition((byte) 0x13, "RAINYEAR", 4));
apiDefs.add(new ApiDefinition((byte) 0x14, "RAINTOTALS", 4));
apiDefs.add(new ApiDefinition((byte) 0x15, "LIGHT", 4));
apiDefs.add(new ApiDefinition((byte) 0x16, "UV", 2));
apiDefs.add(new ApiDefinition((byte) 0x17, "UVI", 1));
apiDefs.add(new ApiDefinition((byte) 0x18, "TIME", 6));
apiDefs.add(new ApiDefinition((byte) 0x19, "DAILYWINDMAX", 2));
apiDefs.add(new ApiDefinition((byte) 0x1A, "TEMP1", 2));
apiDefs.add(new ApiDefinition((byte) 0x1B, "TEMP2", 2));
apiDefs.add(new ApiDefinition((byte) 0x1C, "TEMP3", 2));
apiDefs.add(new ApiDefinition((byte) 0x1D, "TEMP4", 2));
apiDefs.add(new ApiDefinition((byte) 0x1E, "TEMP5", 2));
apiDefs.add(new ApiDefinition((byte) 0x1F, "TEMP6", 2));
apiDefs.add(new ApiDefinition((byte) 0x20, "TEMP7", 2));
apiDefs.add(new ApiDefinition((byte) 0x21, "TEMP8", 2));
apiDefs.add(new ApiDefinition((byte) 0x22, "HUMI1", 1));
apiDefs.add(new ApiDefinition((byte) 0x23, "HUMI2", 1));
apiDefs.add(new ApiDefinition((byte) 0x24, "HUMI3", 1));
apiDefs.add(new ApiDefinition((byte) 0x25, "HUMI4", 1));
apiDefs.add(new ApiDefinition((byte) 0x26, "HUMI5", 1));
apiDefs.add(new ApiDefinition((byte) 0x27, "HUMI6", 1));
apiDefs.add(new ApiDefinition((byte) 0x28, "HUMI7", 1));
apiDefs.add(new ApiDefinition((byte) 0x29, "HUMI8", 1));
apiDefs.add(new ApiDefinition((byte) 0x2A, "PM25_CH1", 12));
apiDefs.add(new ApiDefinition((byte) 0x2B, "SOILTEMP1", 2));
apiDefs.add(new ApiDefinition((byte) 0x2C, "SOILMOISTURE1", 1));
apiDefs.add(new ApiDefinition((byte) 0x2D, "SOILTEMP2", 2));
apiDefs.add(new ApiDefinition((byte) 0x2E, "SOILMOISTURE2", 1));
apiDefs.add(new ApiDefinition((byte) 0x2F, "SOILTEMP3", 2));
apiDefs.add(new ApiDefinition((byte) 0x30, "SOILMOISTURE3", 1));
apiDefs.add(new ApiDefinition((byte) 0x31, "SOILTEMP4", 2));
apiDefs.add(new ApiDefinition((byte) 0x32, "SOILMOISTURE4", 1));
apiDefs.add(new ApiDefinition((byte) 0x33, "SOILTEMP5", 2));
apiDefs.add(new ApiDefinition((byte) 0x34, "SOILMOISTURE5", 1));
apiDefs.add(new ApiDefinition((byte) 0x35, "SOILTEMP6", 2));
apiDefs.add(new ApiDefinition((byte) 0x36, "SOILMOISTURE6", 1));
apiDefs.add(new ApiDefinition((byte) 0x37, "SOILTEMP7", 2));
apiDefs.add(new ApiDefinition((byte) 0x38, "SOILMOISTURE7", 1));
apiDefs.add(new ApiDefinition((byte) 0x39, "SOILTEMP8", 2));
apiDefs.add(new ApiDefinition((byte) 0x3A, "SOILMOISTURE8", 1));
apiDefs.add(new ApiDefinition((byte) 0x3B, "SOILTEMP9", 2));
apiDefs.add(new ApiDefinition((byte) 0x3C, "SOILMOISTURE9", 1));
apiDefs.add(new ApiDefinition((byte) 0x3D, "SOILTEMP10", 2));
apiDefs.add(new ApiDefinition((byte) 0x3E, "SOILMOISTURE10", 1));
apiDefs.add(new ApiDefinition((byte) 0x3F, "SOILTEMP11", 2));
apiDefs.add(new ApiDefinition((byte) 0x40, "SOILMOISTURE11", 1));
apiDefs.add(new ApiDefinition((byte) 0x41, "SOILTEMP12", 2));
apiDefs.add(new ApiDefinition((byte) 0x42, "SOILMOISTURE12", 1));
apiDefs.add(new ApiDefinition((byte) 0x43, "SOILTEMP13", 2));
apiDefs.add(new ApiDefinition((byte) 0x44, "SOILMOISTURE13", 1));
apiDefs.add(new ApiDefinition((byte) 0x45, "SOILTEMP14", 2));
apiDefs.add(new ApiDefinition((byte) 0x46, "SOILMOISTURE14", 1));
apiDefs.add(new ApiDefinition((byte) 0x47, "SOILTEMP15", 2));
apiDefs.add(new ApiDefinition((byte) 0x48, "SOILMOISTURE15", 1));
apiDefs.add(new ApiDefinition((byte) 0x49, "SOILTEMP16", 2));
apiDefs.add(new ApiDefinition((byte) 0x4A, "SOILMOISTURE16", 1));
apiDefs.add(new ApiDefinition((byte) 0x4C, "LOWBAT", 16));
apiDefs.add(new ApiDefinition((byte) 0x4D, "PM25_24HAVG1", 2));
apiDefs.add(new ApiDefinition((byte) 0x4E, "PM25_24HAVG2", 2));
apiDefs.add(new ApiDefinition((byte) 0x4F, "PM25_24HAVG3", 2));
apiDefs.add(new ApiDefinition((byte) 0x50, "PM25_24HAVG4", 2));
apiDefs.add(new ApiDefinition((byte) 0x51, "PM25_CH2", 2));
apiDefs.add(new ApiDefinition((byte) 0x52, "PM25_CH3", 2));
apiDefs.add(new ApiDefinition((byte) 0x53, "PM25_CH4", 2));
apiDefs.add(new ApiDefinition((byte) 0x58, "LEAK_CH1", 1));
apiDefs.add(new ApiDefinition((byte) 0x59, "LEAK_CH2", 1));
apiDefs.add(new ApiDefinition((byte) 0x5A, "LEAK_CH3", 1));
apiDefs.add(new ApiDefinition((byte) 0x5B, "LEAK_CH4", 1));
apiDefs.add(new ApiDefinition((byte) 0x60, "LIGHTNING", 1));
apiDefs.add(new ApiDefinition((byte) 0x61, "LIGHTNING_TIME", 4));
apiDefs.add(new ApiDefinition((byte) 0x62, "LIGHTNING_POWER", 4));
apiDefs.add(new ApiDefinition((byte) 0x63, "TF_USR1", 3));
apiDefs.add(new ApiDefinition((byte) 0x64, "TF_USR2", 3));
apiDefs.add(new ApiDefinition((byte) 0x65, "TF_USR3", 3));
apiDefs.add(new ApiDefinition((byte) 0x66, "TF_USR4", 3));
apiDefs.add(new ApiDefinition((byte) 0x67, "TF_USR5", 3));
apiDefs.add(new ApiDefinition((byte) 0x68, "TF_USR6", 3));
apiDefs.add(new ApiDefinition((byte) 0x69, "TF_USR7", 3));
apiDefs.add(new ApiDefinition((byte) 0x6A, "TF_USR8", 3));
apiDefs.add(new ApiDefinition((byte) 0x70, "SENSOR_C02", 16));
// apiDefs.add(new ApiDefinition((byte) 0x72, "PM25_AQI", -1));
apiDefs.add(new ApiDefinition((byte) 0x72, "LEAF_WETNESS_CH1", 1));
apiDefs.add(new ApiDefinition((byte) 0x73, "LEAF_WETNESS_CH2", 1));
apiDefs.add(new ApiDefinition((byte) 0x74, "LEAF_WETNESS_CH3", 1));
apiDefs.add(new ApiDefinition((byte) 0x75, "LEAF_WETNESS_CH4", 1));
apiDefs.add(new ApiDefinition((byte) 0x76, "LEAF_WETNESS_CH5", 1));
apiDefs.add(new ApiDefinition((byte) 0x77, "LEAF_WETNESS_CH6", 1));
apiDefs.add(new ApiDefinition((byte) 0x78, "LEAF_WETNESS_CH7", 1));
apiDefs.add(new ApiDefinition((byte) 0x79, "LEAF_WETNESS_CH8", 1));
apiDefs.add(new ApiDefinition((byte) 0x80, "PIEZO_RAIN_RATE", 2));
apiDefs.add(new ApiDefinition((byte) 0x81, "PIEZO_EVENT_RAIN", 2));
apiDefs.add(new ApiDefinition((byte) 0x82, "PIEZO_HOURLY_RAIN", 2));
apiDefs.add(new ApiDefinition((byte) 0x83, "PIEZO_DAILY_RAIN", 4));
apiDefs.add(new ApiDefinition((byte) 0x84, "PIEZO_WEEKLY_RAIN", 4));
apiDefs.add(new ApiDefinition((byte) 0x85, "PIEZO_MONTHLY_RAIN", 4));
apiDefs.add(new ApiDefinition((byte) 0x86, "PIEZO_YEARLY_RAIN", 4));
apiDefs.add(new ApiDefinition((byte) 0x87, "PIEZO_GAIN_10", 2*10));
apiDefs.add(new ApiDefinition((byte) 0x88, "PIEZO_RST_RAINTIME", 3));
API_DEFINITIONS = apiDefs.stream().collect(Collectors.toMap(ApiDefinition::marker, Function.identity()));
}
The method below iterates over all relevant bytes and maps them to EcowittRreading objects.
public record EcowittReading(byte marker, String name, Integer value) {}public Collection<EcowittReading> decodeLiveData(byte[] data) {
List<EcowittReading> readings = new ArrayList<>();
for (int i = 5; i < data.length -1; i++) {
byte marker = data[i];
ApiDefinition apiDef = API_DEFINITIONS.get(data[i]);
int size = apiDef.length();
byte[] r = Arrays.copyOfRange(data, i + 1, i + 1 + size); // end index is exclusive
int value = getValue(r);
readings.add(new EcowittReading(marker, apiDef.name(), value));
i = i + size;
}
return readings;
}
private static Integer getValue(byte[] bytes) {
ByteBuffer bb = ByteBuffer.allocate(4);
bb.position(4 - bytes.length);
bb.put(bytes);
bb.position(0);
return bb.getInt();
}The method decodeLiveData() takes the array of bytes and starts a for loop at byte number 6 (at index 5). At the start of the loop, the marker byte is taken from the current index. Based on that byte, the corresponding definition is retrieved from API_DEFINITIONS. Based on the defined length, the next n bytes are taken and assigned to a separate array. The getValue() method takes these bytes and converts them to an integer value. Note that the ByteBuffer.getInt() method requires 4 bytes (the size of an integer). Whenever the array of bytes has a length of less than 4, the ByteBuffer is padded so that the size is always 4.
The loop stops just when it has reached the last byte. This is the checksum byte, and here I am not interested in that value. When the method completes, a partial content of the collection being returned looks something like this.
The values are all integer values. In several cases, we want a value with a single decimal place. The value of the first record should be 15,3 degrees and not 153.
A method like this will move the decimal place 1 digit to the left an convert our value to the desired result.
private Double addDecimalPlace(Integer number) {
return number.doubleValue() / 10.0;
}We don’t want to do this for all values. Humdity, for example, should remain a simple integer between 0 and 100.
In my current situation, I am only interested in a few temperature values. I will probably just pick those values from the list and add a decimal place. Should a larger part of the collection of readings become relevant, it might be useful to add extra information to the ApiDefinition class. Either a conversion method or simply information about the required number of decimal places if that suffices. For now, the above gives me all I need. After the values are processed they are sent to my server using an HTTP request. Other use cases will obviously call for something else.
Update February 25th 2023
Be aware of the fact that the the value 0 is represented by byte 0 (0x00). When the the value drops below 0 (relevant for temperatures below freezing), the byte value cycles back to the maximum byte value. Since the 2 bytes representing temperature have 2 ^ 16 or 65536 options, the decimal value of the max byte is 65535. So, when the temperature hits 0 degrees, byte 0 is returned. However, when the temperature drops to -0,1, byte 65535 is returned.
In order to avoid silly temperature values, the code should be made aware that very high byte values don’t represent very high temperatures, but temperatures below zero. The simplest way to account for this, is adjusting the number the getValue() function returns like this:
private static Integer getValue(byte[] bytes) {
ByteBuffer bb = ByteBuffer.allocate(4);
bb.position(4 - bytes.length);
bb.put(bytes);
bb.position(0);
int value = bb.getInt();
if (value > 30000) {
// 0 is 0, -0.1 is 65535
value = value - 65536;
}
return value;
}I made the adjusted code bold. When the retrieved value is higher than 30000, it is assumed that we are dealing with a negative number and the maximum value (65536) is subtracted. So, in the case of -0,1 degrees, the function does not return 65535, but (65535 – 65536) -1. The value 30000 is somewhat arbitrary, but is is high enough that it could never represent a positive value.