Eye-Fi on Vacation

I just returned from a lovely vacation in Playa del Carmen, Mexico. The hotel I stayed at had Wifi, and I took my trusty D50, Eye-Fi card, and my Linux laptop. But, I never actually used the card. I stuck with my trusty non-Wifi Sandisk card.

Why? Well, I promised my lovely wife that the laptop would stay in its bag except for checking flights (we were affected by American's MD-80 issues). It's a bit hard to program the new network names into the card without the laptop, so it went unused. I also did not want the card sitting in the camera without using its wireless capabilities because the only time I've ever been concerned with battery life in my SLR has been with the Eye-Fi card in it. Contrary to what is in the FAQ I do seem to feel the battery life shorten significantly with the card in there. I wish I had an A/C adapter and an ammeter for my camera to measure this.

Anyway, I really do wish there was some way to configure the card out in the field, or even shut off the radio. I'd love to not have to lug my laptop around (I really need an iPhone). Perhaps the card could populate the image folders on the card with partial ESSIDs for the open wireless networks that it sees. If I changed the active folder on the camera over to one of those, the card could start using the new network. I bet this would be nearly impossible to get working on all cameras, but it might work on a good subset.

Maybe other features of the card could be enabled or disabled by having the user view and delete pictures on the card. Could the card detect when the camera deletes a picture that had an image of "RADIO OFF" in it and actually turn off the radio?

That's really why I'd love to hack the firmware. I could customize the card for me and my own perverted personal use like I do with my kernel.

Add Network Working

As I said, I've been playing with Chris Davies' software a bit. I just got the 'a' command working for the first time, which is the add network command. It's a lot more simple than I thought a first. It's a real advantage decoding this stuff when you have so much smart logic on the card.

Most of what we first thought were complex encodings for adding the card appear to really just be garbage left in memory from previous operations. Even watching the Windows software after a log is dumped, you sometimes see the log data come back in the results of subsequent commands. This makes things a bit harder to decode and it would be really nice if the card zeroed things out, first.

Anyway, in the case of add network, the command is a struct like this:

struct add_net {
char cmd; // always 'a'
char net_name_len;
char net_name[32]; // always null padded
char net_password_len;
char net_password_hex[32];
};

Notice that there isn't a field for the encryption type, channel, MAC address of the AP or anything. The card evidently figures this all out internally, which is really nice.

Oh, and you know how you actually have to let the manager software successfully test for a network before you can add it? I think that's a bit of a disadvantage sometimes. Like if you're going over to your Aunt Betty's house and you know her essid before you go, you can't program it into the card in advance. You would need to be there before you can add it. This is a pretty user-friendly feature because it ensures that people don't mistakenly add networks and get confused about why it never works when they just entered the password wrong. I think generally this is way nicer than what I have on my desktop today.

Well, that feature looks to actually be implemented in the manager software, not the card. I believe you can just issue an 'add' command to the card without actually testing it, first. The manager software just happens to always test it in advance. It might be nice to have a "force-add" feature in the future.

Firmware Fetching

Thanks to EyeFi's Berend and his kind comments explaining the difference between "pure zlib data" and gzip formats I've put together a script to download from the EyeFi web site and decode a copy of the firmware using a random card's mac address. Now if I can just figure out what format the firmware is in...

I also made a mistake. Reading wpa_supplicant's license agreement, I saw the GPL in there and missed the part about it being dual licensed. However, the nice folks at EyeFi were quick to point out:

The Eye-Fi software utilizes LGPL, BSD, Apache, MIT, ZLib, IJG (Independent JPEG Group) and Boost licensed open-source components. None of these licenses require disclosure of the source code.

So, I guess there's no Linux in there after all.

They also pointed out that EyeFi is acting in good faith and sharing improvements they make to open-source components that they use. This bug filing would seem to indicate that they use IwIP for their TCP/IP stack. This might be why p0f doesn't seem to be able to detect the card's OS.

Working Standalone Software!!

So, a reader of this blog has been able to understand many of the interactions with the card. He's written software to:

1. List configured networks
2. List scanned networks
3. Print card MAC address
4. Print card firmware information

and I've added on to it to fetch the card logs. This is really good progress, and the ability to add networks to the card shouldn't be too horribly far away. I'm sure that the author will be willing to share it publicly soon. I'd post it somewhere, but I don't want to be passing around someone else's code for which I have no permission to distribute. I'll post here as soon as it is available.

BTW, a recent commenter noted that the card can only add ASCII WEP keys, and not hex ones. They didin't get much help from EyeFi support. I've seen the same limitation, and I'm happy to say that the card's hardware accepts hex keys. It's just a limitation of the manager software right now that it only takes ASCII. I've actually watched the manager software program the hex values derived from the ASCII keys into the card.

Firmware updates

There have been some people talking in the comments about things to do to the firmware. While I'm sure some of you have actually agreed to the license and are likely restrained by its terms, I bet most of you haven't. You also don't necessarily have to agree to a license to simply download files from a web site.

Simply sitting in an airport next to me (or stalking me at my house) you could easily pick up my EyeFi card, its mac address, and watch it do its firmware updates. You might also see the manager fetch a file from the server for its firmware updates in plain text. If you want a firmware file for a souvenir, you could try fetching from these urls which are completely unauthenticated (except that your card has to be registered). So, try a few random ones until you get a hit:

for ((i=0;i<100;i++));
   url="http://api.eye.fi/api/rest/eyeserver/v1";
   rmac=$(printf '00-18-56-03-%02x-%02x\n'
          $((RANDOM%256)) $((RANDOM%256)));
   wget "${url}/getCardFirmware?Card=$rmac&Version=1.0496";
done

The error result documents are ~100 bytes, while the real ones are a couple hundred kilobytes.

Why I care about EyeFi on Linux

A recent commenter noted that EyeFi simply supports XP and OSX because of the volume on those platforms. It would be silly for them to support the minuscule number of Linux users. Also, most Linux users at least have one XP machine on which to set up the card.

First of all, I kinda hope EyeFi does not release Linux software. We are a small minority and they surely will waste engineering resources on an operating system that they do not understand. I do hope they release information on how the card works. That will let me set it up myself, or let hundreds of others do the same.

Sure, I could go steal my wife's machine to set up the card. But, what do I do when I'm traveling? Isn't that the time I need to change settings in the card the most?

Card communication on the local network

Chris Davies saw this blog and was nice enough to point me to some tools he has written. They somewhat take the place of the Eye-Fi manager. In Chris's words:

> It's all fairly trivial. After the card gets a DHCP
> response, it sends out a flood of ARP requests ... for
> each host in the subnet.  It then does a triple handshake
> on the predesignated port of each host in turn, until it
> finds one that's running a server on it. After that it
> just tries to authenticate with that host, and send it
> pictures.

"That host" is, of course, the one running the Eye-Fi manager. Chris's tools will listen on that predesignated port for the card to contact it, and start accepting pictures.

Well, here's the problem: I don't see any of this behavior!! I set up a second ethernet card in one of my Linux machines and turned on bridging with it and a normal card, then put my wifi access point on that second card. That makes my PC act like an ethernet switch that allows me to monitor all the traffic that goes over it with network monitoring tools.

I used a filter in Wireshark to narrow down the traffic to just the Eye-Fi card with this filter

(eth.addr == 00:18:56:aa:bb:cc) &&
!(ip.addr == 216.218.219.2)

That's because "00:18:56:aa:bb:cc" is the MAC address of the Eye-Fi card, and "216.218.219.2" is the IP address of the Eye-Fi server. So, this let me look at all of the card's communication with things other than the Eye-Fi server.

After the DNS requests to talk to the Eye-Fi server, I don't see the card do much of anything.

Wine patch

Well, it turns out that wine doesn't respect the Windows CreateFile() flags FILE_FLAG_NO_BUFFERING or FILE_FLAG_WRITE_THROUGH. I have a really simple wine patch to "fix" that. I really haven't thought it through in too much detail, but it seems to work.

Basically, whenever wine sees either of those two flags, it just disables all of Linux's filesystem caching with the open() flag O_DIRECT. You can apply the patch to the latest version of wine out of its git repository. and probably earlier versions, too. Here's my patch.

I'll probably be trying to get this into the upstream wine distribution soon.

BTW, if you want some help building wine, I'd be happy to help, so just email me.

Eye-Fi Manager Working in Linux!!!

First of all, this is perhaps the most significant progress I've made so far in unraveling this device. Getting it to work in Linux means that I have a very complete set of debugging tools to use on it to see how it works.

I'm running the "Eye-Fi Manager.exe" with wine. The issues that I was seeing with it were because Linux is not reading back the actual disk contents, but cached disk contents. I noticed this by enabling debugging on the usb_storage driver, then accessing the card multiple times with both O_SYNC and O_DIRECT set.

Those options should have caused Linux to do real reads from the card each time, but I didn't see the usb_storage debugging kicking in each time I did a read. See this post if you're curious why it's important to do real reads each time.

Anyway, I instructed wine to have the kernel throw away its cached copies just before each read() of the control files. I did it with this call:

posix_fadvise(unix_handle, 0, 0, POSIX_FADV_DONTNEED);

The result: a working Eye-Fi Manager in Linux. You need to hack and rebuild wine, but it is at least a proof of concept. It works!!

Now, I just need to go find out why the reads() aren't obeying O_DIRECT. It's probably either the usb_storage driver or the vfat filesystem code.

Alternative to Upload to Computer

Since I run Linux don't have a Windows machine, I can't simply use the Eye-Fi manager's ability to copy all pictures down to a PC. But, I noticed that Google's Picasa has an open API for accessing it (and 1GB of free storage). So, I changed my picture service to Picasa and started digging. There is an apparently Google-written implementation of a perl libarary to access Picasa.

I used this library to create a little script called picasa-upload.pl. You can run this script to automatically download all pictures directly from picasa, and store them on your PC. You can get my script along with a slightly hacked up version of the Picasa libary here:

http://www.sr71.net/~dave/projects/picasa-download

Run this every few minutes in a cronjob and, for me, it's just as good as the built-in Eye-Fi capability!

Eyefi software uses Curl!

I was grepping through the Windows binaries looking to see from where messages come when I ran across a little surprise. The Eye-Fi manager software uses a piece of free software called curl. Specifically, they statically link in a library portion of curl, libcurl. It basically implements a little tiny web browser inside of their application so it can talk back to the Eye-Fi web servers.

It is a bit of a shame that Eye-Fi would choose to use free software inside of its products, but neglect Free Software users by not providing specs or software that run on Free operating systems.

By the way, nobody is violating any license or legal terms here. Curl is free to use in the way Eye-Fi uses it. I just don't think it is very nice.

Request encoding

I've noticed that the data written to the REQM file changes dramatically, despite the command being the exact same. I can only assume that there's some kind of encoding going on there. Maybe not encryption per-se, but there something scrambling it, and with something more than just the simple command. There's some value that changes each time getting thrown into the calculation. But, without going in and doing something against the license agreement like disassembling, I can't think of any easy way to figure out what is going on in there. Suggestions welcome!

More about control files

When I say that the files are 16k in size, I mean that they appear to be 16k on the filesystem. Interestingly, the Eye-Fi Manager also seems to write to them only in 16k chunks. It never writes partial bits of the files. Also, it's good to note that you need to do synchronous file i/o to the files when doing things to them.

Operating systems cache access to files so that if you write to them twice in quick succession, it sometimes only makes one write to the actual disk. It can take seconds or minutes for a filesystem operation to get out to the disk. The Eye-Fi manager ensures that it does synchronous i/o by using some fcntl() calls on OSX. I'm not sure what it does in Windows. I'm using fsync() on the file descriptors in Linux to get the same effect.

New EyeFi Card

I got an EyeFi card for Christmas. It's a wonderful little device. But, sadly, it doesn't come with any software for Linux. I intend to see what I can do to change that. I'd at least like to be able to scan for and add new access points without rebooting into Windows. I'd also like to be able to have it dump its pictures on my Linux PCs instead of uploading to other servers, but that's a way down the line.

The card itself has a number of files on it:

$ find /media/EYE-FI/EyeFi/
/media/EYE-FI/EyeFi/
/media/EYE-FI/EyeFi/reqm
/media/EYE-FI/EyeFi/reqc
/media/EYE-FI/EyeFi/rspm
/media/EYE-FI/EyeFi/rspc

These are how the EyeFi Manager communicates with the card. I think the filenames stand for REQuest or ReSPonse (for the first part of the filename) and Message or Code (for the last character). They are each apparently 16k in size.

From what I can glean so far, the manager starts by making sure the files are there, then zeroing them out. It then writes a command to reqm. I think those commands are single characters because the log has things like this in them:

[00:19] Host control channel received command 'o' (0x6f).

It also writes a character after those command characters, and that's where it starts to get murky. I'll post some more about that tomorrow. In the mean time, if anyone knows anything more about this card, please contact me.