Apple Punishes Those That File Bugs on Their Precious Leopard

Note: This bug only affects the ICBMs. It does not reproduce on any of the three PowerPC Macs I've had it tested on.

In the last two weeks or so, I've noticed that all my cookies were being deleted every so often. A site I'd tell to store my login info would suddenly ask for my password again. Since some of these cookies are many years old, I've song since forgotten the passwords for said sites. Alternatively, I'd go to a site that allows specific view settings to be set (like weather.com) and notice everything was looking like the standard webpage they have for people without cookies set. Even worse was that some of these blogs I always comment on wouldn't let me comment because I didn't have a cookie set that showed I had been approved to leave comments by the blog author.

And in the last four days, I've noticed my cookies were being deleted every day, seemingly at random. This was highly annoying.

Luckily, I use Time Machine so I could easily (very easily) restore my deleted cookies with no issue and once again have Safari work as expected. It was just tedious to restore the cookies file every time they got deleted.

I filed a bug on this issue at the behest of some others that also described a cookie deletion problem. The cookies were not deleted as a result of a Safari crash. I tried revisiting all the porn/not porn sites I visited during that time, but had not been able to reproduce the issue. I then noticed that RadarWeb didn't have my Apple ID which indicated my cookies had yet again been deleted. I found this ironic as I was filing a bug on cookies being deleted.

The "cause" of the issue

During all these cookie deletes, I had the folder ~/Library/Cookies/ opened in the Finder so I could monitor the cookie file's size and hopefully track down the issue as soon as it occurred. I also had it open so restoring from Time Machine when my cookies were deleted was just a click away. Well, 3 clicks away.

I just happened to quit Safari one day before taking a shower. I confirmed that the file size of the cookies file was 2.8MB by glancing at the Finder. While in the shower, I had an epiphany on a completely unrelated bug. As soon as I got out of the shower, I went to RadarWeb to update the unrelated bug with the new information I had discovered while in the shower. I updated the information in the existing unrelated bug, happened to glance at the Finder window, and, behold, the file size of the cookies file was 4KB (the size of an allocation block on my disk).

Yay!. I had reproduced the bug. I restored the cookies file from backup, quit Safari, then repeated each of the actions I had just taken in order to find out which action causes the cookies to be deleted. Yay! I repeated the bug again. ("Yay" in the sense I was able to reproduce it, not "Yay" that I was just hit by a data loss bug.)

Steps to Reproduce:

Please make sure to note the size of your cookies file after the page loads on each step. Note: This will result in the deletion of your cookies, so make sure you have a backup of ~/Library/Cookies/ somewhere.

1. Go to bugreport.apple.com.
2. Log in with your Apple ID. (Your iTunes login and your .Mac login are Apple IDs, but these steps are directed at developers with existing ADC accounts).
3. Click on "My Originated Problems".
4. Click on "Open".
5. Click on any bug to open it.
6. Marvel at the fact your cookies have now been deleted. Your cookies file should now be around 900 bytes (4KB) in size.

I just found out that going to any website after doing Step 3 above will also cause all cookies to be deleted. So you can click "My Originated Problems" then go to any website and your cookies will be deleted. How awesome.

What this bug is not

This is not a bug in Safari/WebKit as far as I or anyone I've asked can determine. This bug is at a much lower level. Perhaps at the NSHTTPCookieStorage level or even at the CFNetwork level. (I'd look through the CFNetwork source code but that source code isn't available, sadly.)

This bug is also not in RadarWeb (the Apple bug reporter website). Even if something in RadarWeb's code triggers this problem, it's still a problem that needs to be fixed on the Mac OS X side, as it is possible for any website to trigger this bug. I've tried other websites that set cookies over SSL, but I can't seem to reproduce the bug on them.

Other Miscellaneous Background Stuff

So far, I've been able to get it to reproduce on 5 separate ICBMs. This problem cannot be reproduced on 3 PowerPC based Macs. So it is an architecture specific bug. So I infer that it is an endianness issue. The network byte order is big endian. The PowerPC is also big endian so no data actually has to be byte flipped. In fact, most of the Network order to Host Endian functions are no-ops on Big Endian systems. They don't do anything. However, on the ICBMs, the bytes need to be flipped. But this is just my uneducated guess at the cause of the issue. It's very possible this isn't related to the bug at all.

I have 70 bugs currently marked as "Open" in RadarWeb. I have 228 bugs currently in the "Closed" section of RadarWeb. I have 1 bug marked as "Needs Attention" which is also in one of the above categories. I'll get to this "needs attention" bug when I can get a chance to be in the environment the attention requires. This means I have filed at least 298 bugs. The pace at which I filed bugs slowed down greatly as I became disillusioned with filling bugs.

Please note that "bug" is very generic and also can refer to "New Feature Request", "Feature Enhancement Request", "You Got the Pixel Wrong", "Typo Made in Documentation", "I'm bored and need to do something today", or "I'm Lonely and Feel the Need to be Insulted by an Anonymous Engineer". The status of "Open" does not mean the bug hasn't been fixed just as the status of "Closed" does not mean the bug has been fixed. They're just categories.

Whenever a new major version of Mac OS X is released, I always find a swath of bugs to file (see above for meaning of "bug"). But this is the reason why I've been hit by this bug so many times in the last few days. Lots of bug filing. Lots of bug updating. Lots of me whining about something or other. Oh well, I won't hear about the status of any of these until at least 10.6 starts seeding in three years. (Hence the "disillusioned" comment above).

Workaround

I don't have a workaround. I don't know how to not cause this problem other than to either 1) Stop filing bugs, 2) use a PowerPC Mac (or use Safari running under Rosetta) to file bugs, or 3) use a different browser when filing bugs. I'm going to go with 4) Continue to file bugs, but restore the cookies file from Time Machine every time it gets emptied.

Update on 11/13 (Zune 2.0 day!)

After more investigation and a lot of help with investigation, it looks as though I was mostly completely and utterly probably wrong about it being an endian issue. But it remains architecture specific. Basically, the issue is that a completely absurd cookie value is sent from the server to the browser. The value is so absurd it causes an overflow condition of some sort. Something changes the absurd value into a fixed number. On the PowerPC, this new value is a still valid value. On the ICBMs, it is an invalid value. When the CFHTTPCookieStorage stuff goes to read in the cookies file, the invalid and absurd value is read in. This is where the CoreFoundation Property List parser comes in an says, "Malformed data byte group at line 1; invalid hex" so the CFPlist stuff returns nothing and a new array is created by the Cookie stuff.

There are actually a few things that need to be addressed here:

Even the "OK" absurd value on the PowerPC is still never going to be seen in the real world as a correct value. The API (this may be CFNetwork) needs to do something reasonable here and actually set a non-absurd value as a replacement. Kind of like when you store NaN to disk as a floating point value and attempt to read it back in, you still get NaN. Such as if the value isn't between two reasonable ranges, snap it to the closest reasonable value, or take it to mean another "value" that is also reasonable.

Why does the CFPlist API just completely give up when one such specific parsing error occurs? Why can't it delete that key/value pair from the read in dictionary. When a parsing error occurs, why doesn't the CFNetwork stuff (or Safari) warn the user that their cookies were just deleted (or at least log a message to the console)?

So the problem isn't the writing out of absurd values, it's the reading in of absurd values. And then when the next cookie, whatever it is, is set, the bad values are committed to disk.

Finally, why is the Cookies.plist an XML Property List when it should clearly be a Binary Property List? For a file that gets written to and read from so dang often, a binary plist would greatly reduce the size of the file, the amount of disk i/o that needs to be performed when operating on the file and the amount of CPU time needed to parse the file. It's a one line change of code to convert to a binary plist and it has no compatibility issue since the Property List APIs were always designed to read XML or binary Property lists interchangeably.