j2me bluetooth gps mo-pho-weblogging

I was going to wait to write this up until I actually had something I considered to be of release quality, but I find my self re-telling the story too much so I'll go ahead and type it out once and for all. Quick Version: I've created a system that allows a Nokia camphone to take a photo, attach GPS coordinates obtained from a bluetooth GPS device, and post it to a weblog along with an interactive map showing the location. I wrote it in J2ME, but I plan on rewriting it in C++. Long Version:

I wanted to let Mie add location information to her camphone blog posts, like I had done for her when she lived in Japan. In Japan, the cellphones have GPS built in, so all I needed to do was extract the information on the server side after she emailed the photos. In the USA no cellphones (yet) have a camera and a GPS built in (US cellphones do have location awareness in emergencies, but that functionality is not made available to the consumer or independent developer), hence the need for a bluetooth GPS solution. I needed to write an cellphone application that would communicate with the bluetooth device. J2ME was chosen since Java is my strongest language. I have written a couple of small J2ME apps before, but they were on a Windows workstation that I no longer have, so I had to set up a J2ME development environement on my MAC OS X Powerbook. For some reason, I was thinking that Russell Beattie was a Mac user, and I knew he did mobile development, so I asked him about setting up the proper environment. It turns out he's not a Mac user, but he pointed me to Rael Dornfest's site which had the information I needed. As I started researching what was needed, I realized that Linux was better supported than Mac OS X, so I ended up just installing Sun's J2ME development kit (WTK2.1) on my Linux box and doing my compiling remotely. Bluetooth hacking was something new to me, and it turns out that the WTK2.1 doesn't have the javax.bluetooth packages in it. WTK 2.2 does, but of course that's only available for Windows. I poked around and found a two week trial version of a commercial bluetooth library and downloaded it, but in the end I went with the opensource JavaBluetooth.org package. After reading numerous PDFs about the Bluetooth protocol, I felt I understood the technology enough to start coding. Russell had sent me his example bluetooth application source, but I ended up not even unzipping it since the example source in the Nokia PDFs worked fine. Soon I had code running on Mie's old Nokia 6600 that could discover all of the bluetooth devices in the vicinity and report each one's unique Bluetooh Address (like a MAC address). W00t! Pretty quickly I hit the first major snag though. Once you've discovered the address of a bluetooth device, the API allows you to connect to it and request a list of services. The request needs to be made in such a way that it includes one or more 128 bit unique identifiers (UID), and the services returned must match all of them. The Nokia example code used a specific UID that was hardcoded into the client and server, but the Belkin GPS device did not come with any information on what 128 bit number it was registering its services under. 128 bits is a lot of possible UIDs. I was stumped on this for a ridiculously long time. The goal was to get the necessary connection string, a URL that starts with btspp:// and identifies the device and service you want to connect to. In the JSR-82 Bluetooth API, this information comes from a call to ServiceRecord.getConnectionURL(). First, I found that there were a few general UIDs that should return generic services for such things as RFCOMM (bluetooth serial communication). I tried these, and indeed I did get back ServiceRecord objects, but calls to the getConnectionURL() method failed mysteriously. In fact, calls to any of the object's methods did the same thing. The code->build->install->test cycle was pretty slow. After building, I had to scp the application to my Mac, and then bluetooth file-transfer it to the phone, then install it on the phone, then run it. I couldn't run bluetooth code in the WTK2.1 emulator There's no console on a phone to help you debug, so I had to explicitly write any and all debug messages to a LogScreen portion of the MIDlet. The code was failing without throwing any Exceptions(!) so I was trying all sorts of crazy debug output with lots of [add new debug lines]->[build]->[copy to mac]->[bluetooth to phone]->[install]->[test] cycles. I was stuck in this mode for hours, which translated into days since I only had a couple of hours a day to work on it. The 128bit UIDs were originally only 16 or 32 bit long, so a lot of early bluetooth code still has the shorter UIDs. There is a base 128 bit string that these can be added to to create a valid 128 bit UID. Since the range of 32 bit values is far less than that of 128 bit values, I wrote an app that ran on the phone and sequenced through querying the GPS device with all the 32 bit UIDs to see which ones worked. I let it run for hours before turning it off (enough to cover all the 16 bit values I think) and ended up with a handful of the generic UIDs that returned a ServiceRecord object, but all the objects still failed quietly. I started to accept that this line of exploration wasn't going to yield any results. I tried it on another Nokia phone, same problem. I was convinced that I needed to know the proper 128 bit UID number in order to get the right ServiceRecord object, the one that would work. I called Belkin sales and begged someone to tell me the s3kr3t UID. A sales drone said she'd have a product manager call me, no one ever did. Finally someone in the Nokia developer forums suggested I change my try-catch code to look for Throwable instead of Exception (Throwable is the parent class of Exception). Doh! /me slaps his forehead. All of the method calls were throwing java.lang.IncompatibleClassChangeError. Compared to Exceptions, Errors are pretty rare in Java development and usually indicate a problem in the JVM environment as opposed to the application itself. To quote Sun's own documentation, "an Error is a subclass of Throwable that indicates serious problems that a reasonable application should not try to catch." Unless you're dealing with J2ME! I changed my code to try ServiceRecords from any bluetooth device, such as my Powerbook. They all failed with the same IncompatibleClassChangeError. Apparently Nokia's java bluetooth implementation is buggy? In any case it appeared out of my hands, there's nothing I'm aware of that I could change in my code to get these method calls to work. Maybe it's a javac verson problem? At this point I was considering tossing in the towell and returning the GPS unit for a refund. But I let it sit for a few days and then went back into research mode. Finally I stumbled across a post by someone who had managed to connect to their bluetooth GPS device using the connection URL of btspp://<btaddr>:1 where <btaddr> is the device's bluetooth address, and 1 is the chennel number. Since I had the GPS bluetooth address since day one, this was an easy thing to try. It worked! I really hope that I've mentioned GPS, J2ME and bluetooth enough so that it ranks high in a google search and saves some other poor developer a headache. Although it was a learning experience, it's frustrating to think how many hours I wasted. Now I was able to read a stream of bytes from the bluetooth connection. They seemed to be ASCII, so I figured I could just parse the data to extract location information. Rich on #geo pointed me at a reference for common NMEA sentences and I recognized several of them in the data stream. In short order I was reporting the latitude and longitude, success! Wednesday afternoon I attended an Institute for the Future geoweb conference I'd been invited to, and when I got home that night I was determined to not sleep until I got the application working. The next step was to somehow connect the coordinates to a jpeg, and the first requirement for that was to get a jpeg. It turns out that J2ME code cannot access jpgs created by the Nokia camera application! J2ME does not have access to the phone filesystem at all, it persists data between invocations in a Record Management System (RMS) that is sandboxed off from the rest of the phone OS. This realization led to some cussing. J2ME does have a Mobile Multimedia API which can access the camera and create its own jpgs, but (and there's always a 'but' in J2ME I've learned) the quality of mmapi jpegs are far, far worse than those the native camera application is capable of producing. I've done a lot of web searching on this issue, and it seems universally accepted. It was past midnight at this point, so I said "screw the quality" and moved on. I found an open source XML-RPC library purportedly for J2ME and tried using it to do some test posts to Moveable Type, but I couldn't get it to work. Older and wiser, I put "try { } (catch Throwable)" statements all over the relevant code and found that the xml-rpc library calls were throwing some other weird Error that I forget now, because I dropped the XML-RPC library route in exasperation as soon as I saw the Error. No problem, I thought, I'll just use the J2ME HttpConnection class to talk to the XML-RPC server and craft my XML data myself. Much weirdness ensued. The HttpConnection could connect to some socket dumping code I was running on my server, and it would pass raw jpeg data but if I tried converting the jpeg bytes to Base64 URL encoding that would be needed for an HTTP POST, the HTTP header (including a correct Content-length: header) would make it across and then the connection would hang and the encode bytes would never go across. Giving up on that finally, I decided to use SocketConnection instead and forget about XML-RPC altogether. Well it turned out that SocketConnection is only available on the MIDP 2.0 version of J2ME. I figured there was a decent chance that the Nokia supported MIDP 2.0, but I couldn't get my build environment to compile in that mode, despite the fact that I was referencing the WTK2.1/lib/midpapi20.jar. It was around 3AM at this point, so I decided to go back to HttpConnection, but treat it like a SocketConnection. That is, I used the HTtpConnection class, but POSTed the data in pure binary format instead of Base 64 URL encoding it. This means I couldn't use a normal HTTP server to accept the POSTed data. Instead I converted my simple socket dumping server code to save the incoming bytes into a jpeg file. This worked. By 3:30AM I had added the latitude and longitude data to the stream. On the server side I parsed the data, wrote the jpeg to my website filesystem and used XML-RPC to create a new weblog post. I used the US Census Bureu's TIGER Mapping Service to pinpoint the coordinates on a map and linked it to Mapquest for interactivity. I noticed that pinpoint was nowhere near where it should be. Rather than being at my house on Potrero Hill in San Francisco, it was down in Silicon Valley somewhere. But it was 4:30AM and I decided to try and sleep. Tossing and turning, all I could think about was that the damn coordinates were wrong, so I got back up and looked into it some more. The coordinates were coming from the GPS unit in this format: 3745.3661N, which I was taking to mean 37.453661 decimal degrees north. I claim the "up all night hacking, brain not working" excuse here. It really means 37 degrees 45 minutes 3661 seconds north...gyagh! In a stupor I hacked out some code to parse and convert it to decimal format (it's (degrees + (minutes/60) + (seconds/3600)) in case you're wondering). This got the pinpoint much closer, but still a block or two away. I think I can fix it by enforcing a certain number of significant figures, but I haven't got around to that yet. By 9AM I was showered, fed and back at the conference. Mike, one of the IFTF organizers and a fellow member of the Locative Media Lab, wanted me to get up and present it, but

1. I didn't think it was at a presentable stage yet, and
2. the mere thought of public speaking puts me into panic mode

I do think this system will be presentable with some more features though, but it nees a lot more work. Minimum:

• The camera quality needs to be better. For this I think my only choice is to rewrite the J2ME part in C++ for Symbian OS so I can just access the jpgs created by the default camera application.
• It needs to be able to send more than one photo, and accept additional text and metadata (e.g. post title).
• It needs to be more configurable. Everything is hardcoded to use my server and GPS device.
• Make the server portion connect to Flickr and Thingster in addition to an MT blog.

Other nice things would be adding del.icio.us-style tags for thingster/flickr posting, creative overlays, maps downloaded to the phone so you can adjust the location in case you took the photos indoors and had to go outside to get a GPS signal, etc, etc, etc. Additionally, Mie really wants an app that will create a GPS tracklog (which this GPS device does not support natively) so when she gets lost (as she often does) she can go back later and laugh about how far off in the wrong direction she went. In fact, now that I can get location information on a cell phone, there are myriad opportunities for locative mobile apps I can play with. It's quite exciting!

Comments

[Mie]

I'm jumping up and down in excitement too! This will be so much fun. I'll be your guinnea pig any time :)

[Russell Beattie]

Nice post! I love narratives of working out how to get shit running.

You should've opened my zip! The Bluetooth stuff wouldn't have done much for you (the sample code you used does pretty much the same). But I did have code in there for posting to Blogger API... it would've saved you some time.

The "Throwable" tip is *very* nice. I've run into the "disapearing app" problem lots of times - it never dawned on me to catch Throwable.

You're working on *which* Nokia phones? They're not generic - even the Series 60 phones. Each version is slightly different than the other one. Something that works on the 3650 won't work on the 6600, bugs in the 6600 aren't in the 7610, but might be in the 6620, etc.

When you get to the part of interpolating over the points on a map and converting from lat/lons to pixel coordinates, taking into account the curvature of the Earth... don't call me. I've played that game and I'm not good at it. ;-)

-Russ

[Ben]

Nice post. You can get the j2me compiles happening on your mac. Its a little bit of an effort but you basically take the classes.zip and the various external classes from the nokia dev kit and put them in the wtk on the mac. Reference them in the classpath and you can then use javac or ant/antenna etc to compile and run the preverifier. Then bluetooth the jar to your phone. Its not as nice as the compile/test/fix/compile/test/fix cycle on the emulators but for a mac developer its not bad.

[Marc Eisenstadt]

Nice one! Gotta get you and Jiri Komzak (Mr HitMaps) to work together... I'm pointing him to this right away...

[Pete11]

Hi,

I have been developing with J2ME for some time and would love to look into developing something like this further.

Would it be possible to obtain a test version of your application to try out?

Thanks,

Pete

[ch424]

This is a really great idea -- and very a helpful article! I'm completely new to java, and was looking for a way to code a bluetooth clicker for my laptop: this was one of the first results on google, and much appreceated. As Pete says, "Would it be possible to obtain a test version of your application to try out?" Or some source please -- I'm struggling!

Thanks,

alex

[ch424]

sorry, my email address didn't appear: [fullname]@gmail.com

alex

[dav]

Oops, the comment-forwarding on this post wasn't being handled right for this post. The 'Pho' part of the title was causing it to get stucking into my "Pho:" mailing list folder instead, so I didn't realize I had gotten any comments! Sorry about that.

[machoman]

Hiya, like all the others I congradulate you on a great writeup. I sent you an email but now I guess you get asked the same question quite a bit. If you are releasing your code I would find it very helpfull for writing my own code for my p900. mail@dave-mitchell.com if you get time. Thanks alot and I look forward to reading about your next programming challenge.

[fisk]

Hi, I stumbled across this page (again) while researching a bit on J2ME / Bluetooth and GPSes, so I think your strategy worked. ;) We're currently in the process of porting an existing application from personal java over to MIDP2, which operates in the same sphere. We presented our work earlier this year at "UbiGIS 2004 - First International Workshop on Ubiquitous GIS" (June 10th 2004) in Gävle, Sweden, were we has done something very similar.

The currently live gateway into our repository can be seen at:

http://www.onemap.org/ch/geometa/browse.php?WMS=Demis

And our presentation etc is available at:

http://www.onemap.org/pages/Archive.html

There is also an article on the way that we have written about the project and metadata in MMSes, which should be published in an upcoming geoinformatics journal. The concept is the same, and this is indeed very much fun :) Hope to see people discover even more fantastic uses for this technology in the future, and don't hesitate to contact me for further information!

[java.fun]

nice post! can the JSR 179 do someting for this topic? I am thinking to wrap a BT GPS receiver as a JSR 179 services on the handset. Then, the user/application should get the location more easily.

[ltr514]

Wonderful! You described so many details that I experienced recently too!

BTW, do you know if btspp://where;1 can access any bt gps receiver? If the BT is class 2 and have no security requirement, we can for sure, but if it is class 1 which usually has higher security level and needs "Authendication", JSR82 leaves the BCC implementation to vendors and does not give you a way to input a password for pairing even though you know the password. Do you have experience on pairing BT devices from j2me level?

Thanks,
ltr514

[Pete11]

Hi,

I am now able to establish a Bluetooth Serial Port connection to my Belkin GPS device and read the NMEA codes, but I seem to have to make 2 connections to make sense of the data.

The first time I connect and receive data back, the codes seem mixed up.

The 2nd time it is fine.

Does anyone know if it is some how necessary to initialise the device first in order to receieve the NMEA codes properly?

Thanks,

Pete

[HardcoreCoder]

Hey
First of all, Great post.
Second I am doing almost the exact same thing for a project ( I need the GPS to tell the PDA to tell the labtop where it is, all through bluetooth).

If you had any ideas/comments besides what is written above, could you plz send them to me. If you had any code/examples, old versions, ect. that you are willing to send I'd appreciate it.
e-mail@s4i75@unb.ca

Thanks in Advance

HardcoreCoder

[mdg]

hi, it's an excellent article, I really want to learn about it, can you send me the source code, or part of if?? Thanks. pablogil@grupomovil.net

[Tonie]

Hi, this is an excellent article. Thank you for including all the useful URL to the resources you used in this project. As some people have requested, can you please post or send me (tygerzone@yahoo.com) the source code for this project? I am working on something similar with PDA and is struggling...

Thanks in advance.

Tonie

[emreceng]

Hi.

Nice Post. I'm student in Dokuz Eylül University Computer Engineering Department. I'm making a project in J2ME,Gps,bluetooth. Something like yours.If you extended your help I will be glad. if you sent me detailed documentation or code. It will help my project.
Thanks.

Mail: edursunl@stx.cs.deu.edu.tr

[carefree]

Great, any hints or experiences with sonyericsson devices? I'm Especially interested in K700i and similar (no pda)
thanks

[jje007]

I'm a Stanford student working on a GPS/Bluetooth project and it would be very helpful to get a copy of your source code. Please contact me at jje (at) stanford.edu if possible.

Thanks,
Jon

[M_Skafte]

Hi,

I'm working on project using Nokia phones for geo-based tracking and communication. I've would be more than happy if I also could get a peek at the source code for project.

Thanks,
Martin, skafte@post1.tele.dk

[rodicaindia]

Hi.

I'm student in Budapest Tech.John von Neumann Faculty of Informatics,Hungary. I'm making a project in J2ME,Gps,bluetooth,Nokia phone. Something like yours.If you extended your help I will be glad. If you sent me detailed documentation or code. It will help my project.
Thanks for your help.

Mail: rodicaindia@yahoo.com

[Sorin]

Hi, I was reading your story while searching for a subject for my graduation paper. I absolutely wanted to do something involving mobile technology and GPS and one of the first things a stumbled in was your narrative story. So i've decided to go with your idea and maybe updating it a bit. So if you are willing to help me in any way possible (giving the fact that i'm brand new in j2me) here's my email adress: sorinbuiga@yahoo.com

[Sorin]

Hi, I was reading your story while searching for a subject for my graduation paper. I absolutely wanted to do something involving mobile technology and GPS and one of the first things a stumbled in was your narrative story. So i've decided to go with your idea and maybe updating it a bit. So if you are willing to help me in any way possible (giving the fact that i'm brand new in j2me) here's my email adress: sorinbuiga@yahoo.com

[Sorin]

Hi, I was reading your story while searching for a subject for my graduation paper. I absolutely wanted to do something involving mobile technology and GPS and one of the first things a stumbled in was your narrative story. So i've decided to go with your idea and maybe updating it a bit. So if you are willing to help me in any way possible (giving the fact that i'm brand new in j2me) here's my email adress: sorinbuiga@yahoo.com

[Sorin]

Hi, I was reading your story while searching for a subject for my graduation paper. I absolutely wanted to do something involving mobile technology and GPS and one of the first things a stumbled in was your narrative story. So i've decided to go with your idea and maybe updating it a bit. So if you are willing to help me in any way possible (giving the fact that i'm brand new in j2me) here's my email adress:
sorinbuiga@yahoo.com