A few years ago, for my senior design project, I worked with a group of guys to look at ways of magnetically tracking a baseball. The idea was that you could embed a small magnet in the ball and with suitably placed sensors around home plate , you could automatically detect if a pitch was a strike or a ball. The intent was not to use this in professional sports (I doubt MLB would take kindly to implanting a magnet in the ball) but rather as an inexpensive training tool for little leaguers. The project went well and we were able to create a prototype that was capable of calling a ball or strike with good fidelity.
After this project, I ended up with the magnetic sensors and I’ve always wanted to revisit this idea (tracking a permanent magnet in 3D space, not the baseball application) with the benefit of a few more years of experience as an engineer. I thought I would use this post to detail the development of a basic research platform for testing out different tracking techniques. I’ll also post some of the sensor data in case others are interested in applying their own analysis techniques.
The goal I had in designing this platform was to create a system that would allow me to easily collect data as well as test out different algorithms for determining the magnet’s position. The system used an FPGA to coordinate data collection from the sensors with data offload to the computer. Once on the computer, the raw sensor data would be displayed on screen, as well as made available to a variety of non-linear optimization routines for determining the magnet’s position relative to the sensors. The final solved-for position of the magnet would then be displayed on the screen.
The picture below shows one of the three MicroMag sensors used to measure the magnetic field. The sensor (green board) was acquired through Sparkfun, and comes with two, seven pin hundred mil headers for relatively easy prototyping. I had a separate PCB board made (yellowish board) to provide mount points as well as an RJ-45 connection for power and communication lines.
Each MicroMag sensor has the ability to detect the magnetic field in three perpendicular axis. The device sensitivity is inversely related to the time required to take a measurement. At the fastest update rate of 2 KHz/Axis, the resolution of the device is ~2 uT, at its slowest rate of ~17 Hz/Axis the resolution is ~16 nT. I typically run the device at its most sensitive setting to provide the highest SNR measurement available.
The device communicates through a slightly non-conventional SPI interface. In addition to the CE, MOSI, MISO, and SCLK typically used in an SPI port, there is also a DRDY line and a Reset line. The Reset line is pulsed high to prepare the device to perform a measurement, while the DRDY line allows the device to indicate to the host that a measurement is complete. To get a complete 3-axis reading, the host must interrogate each axis separately, each time pulsing the Reset line, transmitting an 8 bit command byte indicating which axis to poll and at what sensitivity level, wait for the device to indicate the measurement is finished, and finally clock out the 16 bit field measurement.
In this system I used an FPGA to serve as the intermediary between the computer and the sensors. The nice thing about using an FPGA vs a micro-controller is that I am free to synthesize as many SPI ports as my IO pins will allow. So where as with a micro-controller (which typically only have 1 SPI port) access to the sensor would have to be time-multiplexed, with an FPGA I can create 3 independent SPI ports and sensor communications can take place simultaneously. This feature greatly speeds up the acquisition time and was the main motivation for using an FPGA over a micro-controller.
The FPGA I used was a Spartan-3 1000K Xilinx chip mounted on a development board from Digilent Inc. The development board contains a number of nice on-board accessories, but most importantly, it contains 3, 40-pin expansion ports for connecting external hardware and logic. The board, along with two external PCB boards, is shown below.
The board attached to the expansion port along the right interfaces the FPGA development board with the individual MicroMag sensors. This board provides a separate power supply as well as the connections needed to route the RJ-45 signals to the appropriate IO pins on the header.
The board attached along the top provides a connection to an FTDI development board for their FT2232H chip. I’ve really grown to love this FTDI development board and I’ve ended up using it in a bunch of projects. It provides a really simple interface on both sides of the connection. On the FPGA or micro-controller side, you can choose from a variety of interface protocols from low speed to high speed. I really like their high speed connection, which is capable of providing a sustained throughput of well over 10 Mbytes/Sec. FTDI provides a set of drivers and an API for communicating on the host side, so there is no problem there either. My only complaint about this device, is that in the high speed mode there is only one, 8 bit data bus with no reset signal. This means you need to create your own communication protocol, and the lack of any reset signal means it can be difficult to recover from an error condition. For my communications I use a simple packet structure which contains a 16 bit address field, a 16 bit packet length field, the data payload, and a final 16 bit checksum field. In addition, on the FPGA, I am constantly monitoring the bytes transmitted independent of any protocol, when I detect a specific 128 bit pattern, I reset the USB state. This allows me to always place the USB controller in a known state by simply transmitting 8 bytes.
The logic on the FPGA consists of four main parts: a USB interface module, an SPI interface module, a FIFO for data buffering, and a finite-state machine to tie all the parts together. The FSM uses parameters set by the USB interface module to continually pull data from the sensors and construct the data packets for offload through the USB.
The final key system component is the magnet. I decided to use a rare earth magnet since I felt this would give me the strongest field in the smallest package. I briefly considered using a current carrying coil to create the magnetic field. Using a coil, I could create a very uniform, calibrated magnetic field. The big downside is that I would need cables to provide the current. I ultimately decided that I would prefer the maneuverability of a permanent magnet. In the future I may explore using small current carrying coils to provide the magnetic field. It would also be interesting to explore replacing the MicroMag sensors with small coils to sense the magnetic field. In this case I would need to replace the permanent magnet with a coil carrying AC current, since a static field would be undetectable.
The magnet I used is shown in the photo below. It’s actually two small cylindrical magnets stacked together.The biggest issue with the magnet is accurately modeling its magnetic field. I chose a magnetic dipole model to enable fast calculation of the field at arbitrary points in space. In addition, I used the simplifications suggested by this site to allow the calculation of the field with only two parameters, angle and radial distance. Using these assumptions, the field components are calculated as shown below.
I think the dipole assumption and the associated equations are probably the weakest points of the system. I am trying to measure the magnetic field relatively close to the magnet, and some of the assumptions required for the equations above may not be valid.
The only term, in the equations above, which is dependent on the magnet used is the U term. This is actually the combinations of a couple constants, as shown below.
In this equation, Mu Naught is 4*pi*10^(-7) Newtons per square Ampere, and M is the magnetic dipole moment in Ampere-square Meters. To calculate the magnetic dipole moment for this magnet, I measured the magnetic field strength at various distances along the dipole axis. I calculated the strength of the dipole moment as approximately 1.41 Ampere-square Meters. After a little bit of looking online, this seems like a reasonable value for this sort of magnet.
In order accurately locate the magnet, not only must we know the field strength, we must also know where the field was measured. To ensure the sensors were placed precisely, I created a simple plexiglass baseboard that mounted with the PCB boards pictured above. This baseboard ensured that the sensors did not slide around from measurement to measurement, and day to day.
The baseboard was nice to ensure there was no sensor movement, but I still needed to know where each sensor was located in 3D space. There were a few ways I could go about deducing this information. Since I had the plexiglass baseboard and PCB designed, I know all the dimensions used, so I could use these design files to estimate the final position of the sensors. The problem with this method, is that I gave the drill holes in the PCB and plexiglass board a wide margin to ensure I could use a variety of non-magnetic screws. This means the final position of the sensor boards could vary by hundreds of mils. Instead, what I chose to do was to assemble the sensors on the baseboard, and use photographic images of the final assembly to register the location of the sensors.
This method required registering three different images, the photograph of the final setup, the design schematic of the MicroMag Sensor, and the CAD drawing of the plexiglass baseboard. The MicroMag Schematic and the CAD drawing are nice targets for registration since they have zero rotational misalignment. The picture of the final setup is not as friendly, so the first step in the image registration process was to ensure that the image of the final setup was rotationally aligned with the CAD drawing and the MicroMag schematic. This was equivalent to ensuring the bottom edge of the PCB boards were all perfectly horizontal. The picture below shows the image of the final setup. The amount of rotation required was easily found by estimating the angle between the bottom edge of the PCB with the horizontal axis.
With all three images rotationally aligned, the only remaining step is to scale and shift each image. I chose to use the CAD drawing as the image with which to reference the other images. The registration was accomplished by identifying common points in each image (centers of mount holes, corners, edges, etc…) and then finding the global scaling and X,Y shifting that would align these common points as much as possible in the least squares sense. The nice thing about using the least squares criterion for aligning the points is that the linear algebra is relatively straightforward. The graphic below gives a quick overview of how I set up the matrices to solve the problem.
I’m able to create the Y vector, and the system matrix A from the points on the image. It’s clear that the forward problem is an over-determined problem, so in all likelihood, there will be no single set of parameters that will correctly align all points. The one thing that is a little misleading about the graphic above is the X and Y Offset labels. This registration scales the images and then shifts, so the X and Y offsets shown above may be different depending on the scaling. I’ll include the Matlab code, along with the actual images I used to perform this registration for those who are interested. Below you can see the registered sensor pod with the CAD drawing and MicroMag schematic.
For each of the sensors, MS1 measures the magnetic field in the Y-axis, MS2 measures the magnetic field in the -X-axis, and MS3 measures the magnetic field in the -Z-axis. Aligning the sensors in these axis ensures the use of a right hand coordinate system.
The only remaining step before I could begin collecting and processing meaningful data was to calibrate the sensors. The sensors needed to be calibrated in two manners. First, within each board, each of the three axis sensors needed to be calibrated to each other, I called this intra-sensor calibration. Second, each of the sensor boards needed to be calibrated to each other, I called this inter-sensor calibration. I relied on the Earth’s magnetic field for each of the calibrations. It’s interesting that the Earth’s magnetic field acts as a hinderance during normal operation, but for calibration purposes it serves wonderfully as a nice homogenous field.
To calibrate the individual axis sensors on MicroMag board I took three different sensor readings. The first reading was of the Earth’s field with the sensors oriented in their normal horizontal orientation. The second reading was after rotating the sensors to swap the Z-axis and the X-axis. The final reading was taken after a final rotation to swap the Z-axis and the Y-axis. These three readings allowed me to calculate the scalings needed to calibrate the X-axis and Y-axis to the Z-axis. As a test of the calibration, I rotated the entire setup in air while collecting data. With perfect calibration, no matter what the orientation, the overall sensed magnitude of the X,Y,Z components should not vary (i.e. the value sqrt( X^2 + Y^2 + Z^2) should be invariant under rotations). The first graphic below shows the field magnitude without any calibration.
The field magnitude is much more uniform after calibration. I tried to keep the rotational speed relatively slow since the measurement time is relatively long, but even still, there is bound to be some error due to movement that can’t be avoided.
The final calibration step is to normalize the sensor readings between boards. The graphic above makes it obvious that field strength measured by the first sensor is significantly different than the other two sensors. I set the second sensor as the reference sensor and scaled the other two sensors to match the field measured by sensor 2. The graphic below shows the fully calibrated field.
With all the sensor locations established and the readings calibrated, the remaining step is to use the measured magnetic field to locate the magnet. The problem can be framed as finding the location and orientation of a magnet whose magnetic field most closely matches the observed field. Or to put it another way, we want to find the location (X,Y,Z) and the orientation (ThetaX, ThetaY, ThetaZ) such that the difference between the measured field and the calculated field is minimized according to some norm. What this boils down to, is a non-linear optimization problem to solve for the 6 parameters describing the position and orientation of the magnet. Non-linear optimization is outside of what I deal with on a day-to-day basis so I know next to nothing about it, luckily, many other people know about it and there are many non-linear optimization solvers available. What’s needed to take advantage of these solvers is a function which takes in the parameters to be optimized and returns an error. I initially began using the Matlab environment to solve these problems but I have recently moved into a C++/QT environment which uses a solver passed in through a run-time DLL to enable rapid evaluation of various algorithms.
The first step in taking advantage of the Matlab optimizers is creating the error function. This requires creating a function which can take in the location and orientation of a magnet and return the magnetic field at the measured sensor locations. I’ve included all the M-Files needed to run the optimization routines in the link at the end of this post, but I’ll briefly go through the function which calculates the magnetic field at the sensor locations based on a given location and orientation. The following graphic shows that function.
The function takes in three parameters: Loc which contains the Cartesian coordinates of the magnet location as the first three elements of the vector, and the direction cosines as the last three elements, M which is the magnetic dipole moment, and Sensors which contains the Cartesian coordinate of the individual sensors as column vectors.
The first thing I do is separate out the components of Loc and ensure that the magnitude of the direction cosines is unity. Next, I find the radial vector (and radial unit vector) connecting the sensor location with the magnet location. Using the radial unit vector, and the dipole axis vector, I find the angle between these two vectors. I use the property that the dot product of two unit normal vectors is equal to the cosine of the angle between the vectors.
At this point, I have all the information I need to calculate the magnitude of the magnetic field in the radial direction as well as the theta direction, however I still need to calculate the direction of the theta vector in XYZ space. This is a little bit tricky. This vector is rotated 90 degrees off the radial vector and in the plane formed by the magnetic dipole axis and the radial vector. The surface normal vector of the plane formed by the radial vector and the dipole axis vector can be found by taking the cross product of these two vectors. This surface normal vector is stored in the variable vP in the code above. To find the theta vector, we must rotate the radial vector 90 degrees about the vector vP. Since the vector vP can be an arbitrary axis, I use the axis-angle rotation matrix with the angle set to 90 degrees. This matrix is stored in the variable R, and used to create the theta direction vector rHp. rHp should remain a unit vector under transformation by matrix F, however I normalize it again just to be sure. I now calculate both components and add the vectors to get the resulting field vector. The final step is to project this magnetic field vector onto the axis corresponding to the current sensor.
I use Matlab’s “lsqnonlin” solver to perform the optimization. This takes as one of its arguments the function described above and uses it to find the location and orientation which minimizes the least squared error between the simulated and measured sensor readings. This function is very sensitive to the initial starting position. I’ve used two different initial starting points, one is that I just start centered between the sensors at an elevation of 10 cm with the axis of the magnet pointed up. The other option I use is to start the search at the previous search’s ending point. The advantage of the second option, is that if the sensor hasn’t moved much, your initial guess is likely to be close to the actual location, the danger is that your guesses may begin to diverge if you get a bad estimation.
The following shows some basic position tracking. The magnet’s position is given in blue and the sensor locations are given in red. The first example below shows a square trajectory above the sensors.
The next example I’ll show is moving the magnet forward and backward along the Y-axis.
In the future, I would like to make a table I could put over the sensors that would have pre-cut slots to place the magnet in with a specified orientation so I could get some measure of the absolute positioning error. I’ve also been careful not to bring the magnet too close to the sensor to ensure the devices are not overwhelmed by the strong magnetic field.
The analysis through Matlab is nice for verification and visualization, but I wanted something independent of the Matlab engine and capable of real-time display, so I decided to create a C++/QT based research platform.
The idea was to create a basic framework that would interface with the FPGA to retrieve the sensor data before offloading the data to a variety of modules. There are two main offloads, a data storage module and a locator module.
One of the problems I always face when deciding how I want to save data is what format to use. I use Matlab a lot to process my data, so it’s always tempting save the data in a .mat file. But it can be difficult to use .mat files outside of Matlab and I’m not sure I will always have access to Matlab, so whenever possible I try to use a non-proprietary storage format. In this case, since the data rate is not too high, I was able to save the data in a text based, human readable format. The files themselves contain instructions on how to interpret the data. I’ve included some sample data files in the files contained in the link at the end of the post. There are two sets of data files, the calibration files which were used to calibrate the sensors and the sensor readings that were used to generate the plots above.
To allow for fast testing of various different non-linear optimizers, I chose not to hard-code the solving routines. Instead, the solver is loaded from a run-time DLL. This allows me to try out various solvers without needing to recompile the main program. I can try out a new algorithm by creating a DLL with a predefined function table. The inspiration for this was from the WinRad program I used for my SDR. This program allows anyone to take advantage of the WinRad framework by writing their own DLL interface with their hardware.
The screenshot below shows the program displaying the location of the sensed magnet with respect to the sensors. The sensor readings at the bottom of the screen represent the measured field at the sensor directly above the plot. The position in the Z-axis is represented by the size of the magnet graphic, the larger the graphic, the greater the distance in the Z-axis.
The video below shows the program operation in real-time as I move the magnet in a variety of patterns above the sensors. When the program initially connects to the FPGA the boards are only sensing the Earth’s magnetic field. I initially zero out this field before moving the magnet above the sensors and beginning to solve for the location.
I’m really happy with the overall tracking ability of the current system, but there are a few areas I would like to work on. As I mentioned above, the current magnetic dipole model is perhaps too simple. I would like to perform some tests to quantify the accuracy of this model with this particular magnet. Right now I am assuming the magnetic field is rotationally symmetric about the dipole axis as predicted by the dipole model, it seems possible that impurities in the magnet may invalidate this assumption. I’m not really sure what a more accurate model would look like, would it be a high order polynomial in R and Theta fitted to measured values? I dont’ know.
The other thing I would like to look at is using current carrying coils to generate the magnetic field. I talked about the downsides of this approach, but I think the upsides are significant. Not only could you have a very calibrated field, you could also have an adjustable strength field. One thing I didn’t talk about, but it is a real limitation, is the linear range of the sensors. As the sensed magnetic field increases above 500 uT, the MicroMag sensors begin to behave non-linearly. With a current carrying wire, you could dynamically adjust the current through the coil to ensure that the sensors were all placed in their linear region of operation. It would almost be like automatic gain control for radios.
Another interesting idea that’s a little different than the application here, would be to couple an array of 3-axis magnetic sensors with a metal detector to image metallic objects instead of just giving an audible tone when metal is present. Imagine if you had a metal detector capable of creating a very homogenous, well defined magnetic field, and around the edge of the detector you had a series of 3-axis sensor (could be these MicroMag sensors or perhaps something more sensitive). Then when someone with a metallic object walked through the detector, you could sense how the metallic object distorted the magnetic field, and solve the inverse problem to locate the metal causing the distortion. It would be similar to what’s done in brain mapping with MEG. I’m confident you could locate the general area of the metallic object, but how well you could discern various shapes, I’m not sure.
I’ve collected the files for the image registration and the magnet localization into a single zip file, available here. I hope the file names are self-explanatory. I’ve included M-Files to read the text based data files into the Matlab workspace. There are is also a function to scale the readings based on the calibrations as well as remove the Earth’s magnetic field.