Tag Archives: Satellite

GOES EMWIN-N Decoding

The previous GOES post dealt with the Low Rate Information Transmission (LRIT) data stream. There is another interesting data stream transmitted by the GOES satellite, that is the Emergency Managers Weather Information Network (EMWIN) data stream. This service streams a variety of weather related information, everything from terrestrial weather warnings and radar images, to space weather conditions. There are a variety of different dissemination techniques for the EMWIN messages. There is an internet feed available, terrestrial radio sources and of course the GOES satellite.  Even within the GOES satellite constellation there are two separate methods for transmitting the EMWIN message. Until December 14th of this year GOES-West (aka GOES-11) will use a FSK transmission approach to transmit data with very little overheard (i.e. data is streamed continuously with no FEC), essentially a 9600 baud RS-232 link. After December 14th, GOES-West will begin transmitting its data with the same scheme as GOES-East (aka GOES-13), a 17970 Symbol/Sec Offset Quadrature Phase Shift Keying (OQPSK) modulation with heavy FEC and packetized data. In addition, there is a carrier frequency change from 1690.725 MHz for EMWIN-I to 1692.7 MHz for EMWIN-N. Below you can see the spectrum for the EMWIN-I signal. For reference, the FSK has a frequency separation of 3600 Hz. Each of the spectrum plots shown below span 100 KHz bandwidth.

GOES 11 EMWIN-I FSK Spectrum

Below, for comparison, is the spectrum of the EMWIN-N spectrum from GOES-13. It’s clear that there has been some filtering of the data pulses to contain the bandwidth of this signal.

GOES 13 EMWIN-N OQPSK Spectrum

The rest of this post will detail the demodulation and decoding of the EMWIN-N signal from the GOES-13 satellite. The data packet structure is the same as the structures used by the LRIT stream, so I’ll try to skip this material and just reference anyone who’s interested to my previous post on this topic.

Decision Directed Carrier Tracking

The table below is taken from a report by the Aerospace Company. This lays out all the parameters necessary for demodulating the EMWIN-N signal. Not only does the EMWIN signal share the same packet structure as the LRIT signal, it also shares the same forward error correction parameters. The Viterbi filter parameters, the RS parameters, as well as the randomization bits are identical between LRIT and EMWIN.

EMWIN-N Signal SpecificationsI chose to go a different route to recover the carrier for this data stream as compared to the LRIT data stream. Previously, I used a Costas loop to track the carrier on the BPSK signal. The costas loop for a BPSK signal is relatively straightforward. While there are implementations of the Costas loop available for the QPSK and OQPSK signals, I chose to use a decision directed tracking method.

There are two main steps to this implementation, first the decoder must determine the carrier frequency and phase, then the decoder must track this frequency and phase in time. Even though the transmitter in this case is a geostationary satellite and should have no doppler shifting, the tracking is still important. A phase error of just pi/4 can cause the decoder to produce garbage. This amount of phase error could come from numerous sources, the carrier frequency could be slightly off, the satellite may drift around, a local clock may heat up or cool down slightly changing the sampling frequency, etc… The point being, even though things should remain aligned, they won’t, and the tracking will account for this.

To initially lock onto the signal carrier, the decoder searches both carrier frequency and phase, evaluating the error rate in the first 50 ms of data (~900 Bits) for a chosen frequency, phase pair. The frequency, phase pair which results in the lowest (ideally zero) bit errors is chosen as the initial starting point for the decoder.

The bit errors over the 2D search space are shown below. I chose to search +-200 Hz of the specified carrier, evaluating 40 different phases between -pi and pi at each frequency. The blue streaks around -155 Hz represent no detected bit errors. At this frequency, there appear to be two phases (~0 and pi radians) which result in the best decoder performance.

OQPSK Decision Directed Lock UpThe graphic below zooms in on the area outlined by the black box above. This search was performed with finer frequency and phase steps to increase the level of detail.

OQPSK Decision Directed Lock Up

The zoomed in error map makes clear that for this brief time period, there are a relatively wide range of frequencies and phases which result in no bit errors. This isn’t surprising since we haven’t processed enough data to allow for errors due to small frequency offsets to manifest themselves. There has to be some sort of trade off between initial search time and precision. This search took only a few seconds but resulted in a frequency ambiguity of a few Hz and a phase ambiguity of a few tenths of a radian. This precision is sufficient since the decoder continues to track the signal throughout the demodulation process. For curiosities sake, I did perform a search with a much longer data record, 500 ms, over a smaller search space. The result of that search is shown below. This high precision search took many minutes to complete and is really unnecessary for this application. Even if we took the best frequency, phase pair from this extended search, without continually tracking, the demodulator would accumulator too much phase error within just a few seconds.

 OQPSK Bit Error Frequency Phase Search Space

Based on the above analysis, I began the demodulation process with a carrier frequency of (1692.7e6 – 155.803) Hz, and a phase of -0.2 Radians. The actual retrieval of the bits is very straightforward. The real part is offset from the imaginary part by one half the symbol time and sampled every 1/17970 Seconds. I used an early-late timing scheme to ensure the sampling times remained correct.

I used a different process to track the phase during the demodulation than during the initial lock on procedure. I wanted to detect and correct any phase misalignment before bit errors occurred. To accomplish this, I processed the data in 90 ms batches. For each batch, I create a constellation map of the decoded symbols and recorded the average position of each symbol on the complex plane (i.e. the centroid of each symbol). Ideally, these centroid locations should be equally spaced at 45, 135, 225, and 315 degrees around the origin. Any phase error will shift the centroid locations off these ideals. I use the error between the measured centroid angles and the ideal to create a phase correction which is accounted for in the next batch. The figure below shows a constellation map from a number of different batches with the ideal centroid locations for each symbol identified.

Offset Quadrature Phase Shift Keying OQPSK Constellation Map

Tracking the phase corrections throughout the decoding process gives us some idea of how close the initial estimate of the carrier frequency was to the observed carrier frequency. The phase correction scheme used is similar to a proportional controller, any frequency offset will show up as a constant trend in the phase correction. The figure below shows the applied phase corrections, with a very obvious trend.

Clark E. Tedford

The tracking scheme applied ~400 radians of phase over ~600 seconds, giving a frequency of about 0.1 Hz (400/600/(2*pi)). Which means the initial guess was reasonably accurate. It also means that the decoder would have accumulated pi/4 radians of phase error in just 1.25 seconds if left uncorrected.

When I initially implemented the early-late timing scheme, I used a single precision number in one of my accumulators. In most of my day-to-day calculations, single precision is more than adequate (often I’m processing 12 bit ADC data). However, when used in the accumulator of the early-late algorithm it resulted in the following constellation map (same data as the previous constellation map).

Clearly a horrific decoding. Here’s what happened. This data represents a few seconds worth of collections, so millions of samples. The accumulator was keeping track of the sampling location of the next bit, this has both an integer part and a fractional part. As the integer part grew larger and larger, the ability to represent the fractional part decreased. Eventually the resolution of the fractional part grew so poor that it resulted in a complete loss of timing information. The quick fix was moving to a double precision accumulator. A better fix would have been to separate the integer and fractional part of the accumulator, thereby ensuring the resolution of the fractional component was independent of the integer size.

Once the symbols have been decoded, they must be converted to their two bit representation. I used the mapping:

+45 degrees  -> 01

+135 degrees -> 00

-135 degrees -> 10

-45 degrees  -> 11

The OQPSK allows the 17970 Symbols/Sec to represent a 35940 Bit/Sec data stream. The 35940 Bit/Sec data stream is  run through the Viterbi decoder which brings the data rate back down to 17970 Bits/Sec. The decoded bits do not represent the data, instead they have an additional layer of encoding, where a bit transition represents a “1” bit, and no transition represents a “0” bit. This is a so-called Non-Return-to-Zero Mark (NRZ-M) encoding.

After removing the NRZ-M encoding, the bit stream is in the same format as the LRIT bit stream. It is encapsulated in CADUs, VCDUs, M_PDUs, and finally CP_PDUs. Unlike the LRIT, the EMWIN-N doesn’t rely on different APID to separate data. Instead there are just two VCDU packets transmitted, a fill packet (VCID = 63) and a data packet (VCID = 13).

The data is actually relatively straightforward to retrieve. It is a continuous stream of data contained in the data payload of the CP_PDU packets. The data stream is broken up into 1116 byte packets, each with 12 bytes of consecutive zeros serving as the alignment word, followed by an 80 byte header, and 1024 byte data payload. An example header is shown below:

/PFHADT45US.TXT/PN 3    /PT 3    /CS 11343 /FD9/24/2010 8:37:34 PM

The “/PFHADT45US.TXT” string represents the file name. The “/PN 3    /PT 3” string means that this is the third packet in a file made up of 3 packets. These are the relevant field for retrieving the data from the packets.

The data contained in the payload can be written directly to the file without any intermediate steps. This allows for very easy processing of the zip, gif, and jpg files. In the files associated with this post, I’ve included a script which will search the entire bitstream for files of a specific type, and write these files to the disk.

The following shows some of the data contained in the decoded EMWIN-N data stream. Unlike the LRIT images which contain various records explaining each image, the EMWIN-N images come pretty much as is, so you’re guess about what each image represents is as good as mine. If anyone knows of a place which details the naming convention please let me know.

JPEGs

This was the first full image contained in the data set.

EMWIN-N Image

There are a number of images like the one below, each focused on a different part of the country.

EMWIN-N Image

GIFs

There were only two gif’s present in the data set, each looked like a contour map of something.

EMWIN-N Image

TXTs

The vast majority of the data packets are either text files, or zipped text files. These text files give weather reports for every part of the country as well as other miscellaneous information. The most interesting report I found was a space weather report, shown below and can be downloaded here.

EMWIN-N Space Weather Report

ZIS

The final data type transmitted through EMWIN is called ZIS, but is actually just a zip file. All the ZIS files I’ve looked at contain only text files.

M-Files and Data

I’ve organized the M-Files into three separate folders. One folder contains the code needed to demodulate the data into the OQPSK symbols. Another folder contains code to convert the symbols into the CP_PDU packets. The final folder contains the code required to retrieve individual files from the EMWIN-N data stream. I’ve included most of the binary data, but if you want to regenerate any of the other data you should only have to change a few file name declarations. The files can be downloaded here.

GOES Satellite Decoding, Part 2

There are a number of different Viterbi packages freely available. For the C++ implementation of the GOES decoder, I used the package available here, which was created by Phil Karn. For the initial Matlab decoder, I relied on the Viterbi decoder in the communications toolbox. I posted the raw bit data, and I plan to post the decoded data here so anyone testing out the C++ Viterbi decoder should have enough test data to debug and get things working.

The first step in using Matlab’s Viterbi decoder is to create a Viterbi Trellis. The GOES satellite uses the NASA standard k=7, rate 1/2 encoding. The trellis for this encoding can be created in Matlab with the poly2tellis command:

trel = poly2trellis(7,[171 133]);

Here the taps on the encoding shift register are represented in Octal notation. The first set of taps is 171 (decimal 121, and hex 79), and the second is 133 (decimal 91, and hex 5B).

With the trellis created, the bitstream can be decoded with the vitdec command. There are a number of user definable parameters available with the vitdec command. The first parameter is the trace-back length for the decoder. I stuck with the convention of 5 times the trace-back length. The next option is the mode of the decoder, this options determines what state the decoder starts with. I used the ‘trunc’ option which places the decoder in the all zeros state. The final option specifies how you quantize the input data, I used binary data so I chose the hard-decision option. The entire command is given as:

decoded1 = vitdec(bit1,trel,tblen,’trunc’,’hard’);

Here tblen represents the traceback length and bit1 represents the bit stream.

There is one further complication, since this is a rate 1/2 code, each bit is represented by 2 channel symbols. There is no way to tell which BPSK bit corresponds to the first channel symbol and which corresponds to the second, to overcome this I decode two bitstreams, each offset by 1 bit. I can then re-encode the decoded data to look at the bit errors. By seeing which one of the re-encoded data streams is most similar to the original bit stream, the alignment of the channel symbols can be determined.

One point I should mention, with the Costas loop, the carrier recovery has an ambiguity of pi radians. This means that what I currently refer to as a positive could be negative in a different receiver or even a different initialization of the current receiver. This viterbi filter is transparent, that is, if the incoming bitstream is inverted, the output will decode correctly but still be inverted.

Frame Synchronization

The output of the Viterbi filter is a stream of Channel Access Data Unit (CADU) packets. These packets contain the raw imaging data and a Reed-Solomon error correcting header, all xor’d with a pseudorandom bit mask to ensure there are no long runs of ones or zeros.  The start of each packet can be identified by a 32 bit synchronization header, 0x1ACFFC1D, and is 8192 bits long, including the header.

At 32 bits, it is unlikely that this particular string of bits will occur by random, but even still, I search for the header of each packet, and once found, take the next 8192 bits from the start of the header before searching again. This ensures that if a CADU packet were to contain the synchronization header bits at a location other than the start, I would only lose two packets instead of a long stream of packets.

The frame sync stage is where I check for the correct polarity of the incoming bit stream. I look for both the synchronization header described above, and its complement. I can determine the polarity by looking at if I am finding the specified header or its complement.

De-Randomization

Once the CADU packet has been isolated, the next step is to remove the randomization. Each packet has its data payload (i.e. everything other than the synchronization header) xor’d with a pseudorandom bit mask. This randomization process is different than some other information bearing signals transmitted by GOES. The mask used on each packet is identical from transmission to transmission. Some signals (the EMWIN from GOES-11) xor their information with a free running PN bit stream. The generator polynomial for the bit stream is:

1 + x^3 + x^5 + x^7 +x^8

The first few bits of this output are:

111111110100100000001110110000001001101000001101011100001011110010001110001011

I’ve included the entire random bit string in the mFiles and data zip file at the end of this post.

After Xor’ing this Pn bit stream with the CADU, we are left with the Coded Virtual Channel Data Unit (CVCDU). The CVCDU contains the data payload along with the Reed-Solomon error correction headers.

Next Steps

We’re slowly beginning to make our way up the protocol layers. The next steps will require us to create the individual protocol data units. These must be arranged by the type of data they are transmitted (i.e. visual light image, ultraviolet light image, textual information, etc..), and then the data must be decompressed row by row to yield the final image. I’ll post the next couple steps in the decoding process in a few days.

I’ve attached the mFiles and Data files created and used in this post here. There are three mFiles, viterbiDecode, packetDecode, frameDecode. viterbitDecode takes in the raw bitstream from the last post. packetDecode takes the output of the virterbiDecode function and breaks this up into the synchronized packets. frameDecode removes the randomization. The data contains the PN bit stream, as well as the output of the Viterbi Filter and the derandomized packets.