Tag Archives: EMWIN-N

GOES EMWIN-N Decoding

The previous GOES post dealt with the Low Rate Information Transmission (LRIT) data stream. There is another interesting data stream transmitted by the GOES satellite, that is the Emergency Managers Weather Information Network (EMWIN) data stream. This service streams a variety of weather related information, everything from terrestrial weather warnings and radar images, to space weather conditions. There are a variety of different dissemination techniques for the EMWIN messages. There is an internet feed available, terrestrial radio sources and of course the GOES satellite.  Even within the GOES satellite constellation there are two separate methods for transmitting the EMWIN message. Until December 14th of this year GOES-West (aka GOES-11) will use a FSK transmission approach to transmit data with very little overheard (i.e. data is streamed continuously with no FEC), essentially a 9600 baud RS-232 link. After December 14th, GOES-West will begin transmitting its data with the same scheme as GOES-East (aka GOES-13), a 17970 Symbol/Sec Offset Quadrature Phase Shift Keying (OQPSK) modulation with heavy FEC and packetized data. In addition, there is a carrier frequency change from 1690.725 MHz for EMWIN-I to 1692.7 MHz for EMWIN-N. Below you can see the spectrum for the EMWIN-I signal. For reference, the FSK has a frequency separation of 3600 Hz. Each of the spectrum plots shown below span 100 KHz bandwidth.

GOES 11 EMWIN-I FSK Spectrum

Below, for comparison, is the spectrum of the EMWIN-N spectrum from GOES-13. It’s clear that there has been some filtering of the data pulses to contain the bandwidth of this signal.

GOES 13 EMWIN-N OQPSK Spectrum

The rest of this post will detail the demodulation and decoding of the EMWIN-N signal from the GOES-13 satellite. The data packet structure is the same as the structures used by the LRIT stream, so I’ll try to skip this material and just reference anyone who’s interested to my previous post on this topic.

Decision Directed Carrier Tracking

The table below is taken from a report by the Aerospace Company. This lays out all the parameters necessary for demodulating the EMWIN-N signal. Not only does the EMWIN signal share the same packet structure as the LRIT signal, it also shares the same forward error correction parameters. The Viterbi filter parameters, the RS parameters, as well as the randomization bits are identical between LRIT and EMWIN.

EMWIN-N Signal SpecificationsI chose to go a different route to recover the carrier for this data stream as compared to the LRIT data stream. Previously, I used a Costas loop to track the carrier on the BPSK signal. The costas loop for a BPSK signal is relatively straightforward. While there are implementations of the Costas loop available for the QPSK and OQPSK signals, I chose to use a decision directed tracking method.

There are two main steps to this implementation, first the decoder must determine the carrier frequency and phase, then the decoder must track this frequency and phase in time. Even though the transmitter in this case is a geostationary satellite and should have no doppler shifting, the tracking is still important. A phase error of just pi/4 can cause the decoder to produce garbage. This amount of phase error could come from numerous sources, the carrier frequency could be slightly off, the satellite may drift around, a local clock may heat up or cool down slightly changing the sampling frequency, etc… The point being, even though things should remain aligned, they won’t, and the tracking will account for this.

To initially lock onto the signal carrier, the decoder searches both carrier frequency and phase, evaluating the error rate in the first 50 ms of data (~900 Bits) for a chosen frequency, phase pair. The frequency, phase pair which results in the lowest (ideally zero) bit errors is chosen as the initial starting point for the decoder.

The bit errors over the 2D search space are shown below. I chose to search +-200 Hz of the specified carrier, evaluating 40 different phases between -pi and pi at each frequency. The blue streaks around -155 Hz represent no detected bit errors. At this frequency, there appear to be two phases (~0 and pi radians) which result in the best decoder performance.

OQPSK Decision Directed Lock UpThe graphic below zooms in on the area outlined by the black box above. This search was performed with finer frequency and phase steps to increase the level of detail.

OQPSK Decision Directed Lock Up

The zoomed in error map makes clear that for this brief time period, there are a relatively wide range of frequencies and phases which result in no bit errors. This isn’t surprising since we haven’t processed enough data to allow for errors due to small frequency offsets to manifest themselves. There has to be some sort of trade off between initial search time and precision. This search took only a few seconds but resulted in a frequency ambiguity of a few Hz and a phase ambiguity of a few tenths of a radian. This precision is sufficient since the decoder continues to track the signal throughout the demodulation process. For curiosities sake, I did perform a search with a much longer data record, 500 ms, over a smaller search space. The result of that search is shown below. This high precision search took many minutes to complete and is really unnecessary for this application. Even if we took the best frequency, phase pair from this extended search, without continually tracking, the demodulator would accumulator too much phase error within just a few seconds.

 OQPSK Bit Error Frequency Phase Search Space

Based on the above analysis, I began the demodulation process with a carrier frequency of (1692.7e6 – 155.803) Hz, and a phase of -0.2 Radians. The actual retrieval of the bits is very straightforward. The real part is offset from the imaginary part by one half the symbol time and sampled every 1/17970 Seconds. I used an early-late timing scheme to ensure the sampling times remained correct.

I used a different process to track the phase during the demodulation than during the initial lock on procedure. I wanted to detect and correct any phase misalignment before bit errors occurred. To accomplish this, I processed the data in 90 ms batches. For each batch, I create a constellation map of the decoded symbols and recorded the average position of each symbol on the complex plane (i.e. the centroid of each symbol). Ideally, these centroid locations should be equally spaced at 45, 135, 225, and 315 degrees around the origin. Any phase error will shift the centroid locations off these ideals. I use the error between the measured centroid angles and the ideal to create a phase correction which is accounted for in the next batch. The figure below shows a constellation map from a number of different batches with the ideal centroid locations for each symbol identified.

Offset Quadrature Phase Shift Keying OQPSK Constellation Map

Tracking the phase corrections throughout the decoding process gives us some idea of how close the initial estimate of the carrier frequency was to the observed carrier frequency. The phase correction scheme used is similar to a proportional controller, any frequency offset will show up as a constant trend in the phase correction. The figure below shows the applied phase corrections, with a very obvious trend.

Clark E. Tedford

The tracking scheme applied ~400 radians of phase over ~600 seconds, giving a frequency of about 0.1 Hz (400/600/(2*pi)). Which means the initial guess was reasonably accurate. It also means that the decoder would have accumulated pi/4 radians of phase error in just 1.25 seconds if left uncorrected.

When I initially implemented the early-late timing scheme, I used a single precision number in one of my accumulators. In most of my day-to-day calculations, single precision is more than adequate (often I’m processing 12 bit ADC data). However, when used in the accumulator of the early-late algorithm it resulted in the following constellation map (same data as the previous constellation map).

Clearly a horrific decoding. Here’s what happened. This data represents a few seconds worth of collections, so millions of samples. The accumulator was keeping track of the sampling location of the next bit, this has both an integer part and a fractional part. As the integer part grew larger and larger, the ability to represent the fractional part decreased. Eventually the resolution of the fractional part grew so poor that it resulted in a complete loss of timing information. The quick fix was moving to a double precision accumulator. A better fix would have been to separate the integer and fractional part of the accumulator, thereby ensuring the resolution of the fractional component was independent of the integer size.

Once the symbols have been decoded, they must be converted to their two bit representation. I used the mapping:

+45 degrees  -> 01

+135 degrees -> 00

-135 degrees -> 10

-45 degrees  -> 11

The OQPSK allows the 17970 Symbols/Sec to represent a 35940 Bit/Sec data stream. The 35940 Bit/Sec data stream is  run through the Viterbi decoder which brings the data rate back down to 17970 Bits/Sec. The decoded bits do not represent the data, instead they have an additional layer of encoding, where a bit transition represents a “1” bit, and no transition represents a “0” bit. This is a so-called Non-Return-to-Zero Mark (NRZ-M) encoding.

After removing the NRZ-M encoding, the bit stream is in the same format as the LRIT bit stream. It is encapsulated in CADUs, VCDUs, M_PDUs, and finally CP_PDUs. Unlike the LRIT, the EMWIN-N doesn’t rely on different APID to separate data. Instead there are just two VCDU packets transmitted, a fill packet (VCID = 63) and a data packet (VCID = 13).

The data is actually relatively straightforward to retrieve. It is a continuous stream of data contained in the data payload of the CP_PDU packets. The data stream is broken up into 1116 byte packets, each with 12 bytes of consecutive zeros serving as the alignment word, followed by an 80 byte header, and 1024 byte data payload. An example header is shown below:

/PFHADT45US.TXT/PN 3    /PT 3    /CS 11343 /FD9/24/2010 8:37:34 PM

The “/PFHADT45US.TXT” string represents the file name. The “/PN 3    /PT 3” string means that this is the third packet in a file made up of 3 packets. These are the relevant field for retrieving the data from the packets.

The data contained in the payload can be written directly to the file without any intermediate steps. This allows for very easy processing of the zip, gif, and jpg files. In the files associated with this post, I’ve included a script which will search the entire bitstream for files of a specific type, and write these files to the disk.

The following shows some of the data contained in the decoded EMWIN-N data stream. Unlike the LRIT images which contain various records explaining each image, the EMWIN-N images come pretty much as is, so you’re guess about what each image represents is as good as mine. If anyone knows of a place which details the naming convention please let me know.

JPEGs

This was the first full image contained in the data set.

EMWIN-N Image

There are a number of images like the one below, each focused on a different part of the country.

EMWIN-N Image

GIFs

There were only two gif’s present in the data set, each looked like a contour map of something.

EMWIN-N Image

TXTs

The vast majority of the data packets are either text files, or zipped text files. These text files give weather reports for every part of the country as well as other miscellaneous information. The most interesting report I found was a space weather report, shown below and can be downloaded here.

EMWIN-N Space Weather Report

ZIS

The final data type transmitted through EMWIN is called ZIS, but is actually just a zip file. All the ZIS files I’ve looked at contain only text files.

M-Files and Data

I’ve organized the M-Files into three separate folders. One folder contains the code needed to demodulate the data into the OQPSK symbols. Another folder contains code to convert the symbols into the CP_PDU packets. The final folder contains the code required to retrieve individual files from the EMWIN-N data stream. I’ve included most of the binary data, but if you want to regenerate any of the other data you should only have to change a few file name declarations. The files can be downloaded here.