Since version 3 of Firefox, the browser has moved over from using flat files for keeping track of browsing history (history.dat) and bookmarks (bookmarks.html) to using SQLite databases (places.sqlite). This change over was required because the old flat file formats were badly implemented, clunky, and not able to handle the new demands of the location bar and browser history. Using a SQL database was the perfect solution for the complexity brought in with the new location bar and its dynamic searching of previous URLS, as SQL is easy to implement, is mostly compatible against multiple SQL application implementations (removing dependency on a single product), and powerful for cross referencing lookups. As a matter of fact, most of the data Firefox keeps now is stored in SQLite databases.

SQLite was also a good choice for the SQL solution because it can be implemented minimally straight into a product without needing a large install and a lot of bloat. While I like SQLite for this purpose and its ease of implementation, it lacks a lot of base SQL functionality that would be nice, like TABLE JOINS inside of DELETE statements, among many other language abilities. I wouldn’t suggest using it for large database driven products that require high optimization, which I believe it can’t handle. It’s meant as a simpler SQL implementation.

Anyways, I was very happy to see that when you delete URLs from the history in the newest version of Firefox that it actually deletes them out of the database as opposed to just hiding them, like it used to. The history manager actual seems to do its job quite well now, but I noticed one big problem. After attempting to delete all the URLs from a specific site out of the Firefox history manager, I noticed there were still some entries from that site in the SQLite database, which is a privacy problem.

After some digging, I realized that there are “hidden” entries inside of the history manager. A hidden entry is created when a URL is loaded in a frame or IFrame that you do not directly navigate too. These entries cannot be viewed through the history manager, and because of this, cannot be easily deleted outside of the history database without wiping the whole history.

At this point, I decided to go ahead and look at all the table structures for the history manager and figure out how they interact. Hidden entries are marked in places.sqlite::moz_places.history with the value “1”. According to a Firefox wiki “A hidden URL is one that the user did not specifically navigate to. These are commonly embedded pages, i-frames, RSS bookmarks and javascript calls.” So after figuring all of this out, I came up with some SQL commands to delete all hidden entries, which don’t really do anything anyways inside the database. Do note that Firefox has to be closed to work on the database so it is not locked.

sqlite3 places.sqlite
DELETE FROM moz_annos WHERE place_id IN (SELECT ID FROM moz_places WHERE hidden=1);
DELETE FROM moz_inputhistory WHERE place_id IN (SELECT ID FROM moz_places WHERE hidden=1);
DELETE FROM moz_historyvisits WHERE place_id IN (SELECT ID FROM moz_places WHERE hidden=1);
DELETE FROM moz_places WHERE hidden=1;
.exit

This could all be done in 1 SQL statement in MySQL, but again, SQLite is not as robust :-\. There is also a “Favorite’s Icon” table in the database that might keep an icon stored as long as a hidden entry for the domain still exists, but I didn’t really look into it.

Continued from Part 1. Once again, I received another notification of a friend joining from an email I gave to the LinkedIn system. I contacted LinkedIn before writing the previous post on the topic with the following message:

For reference, your privacy policy states the following

Information about your Contacts
In order to invite others to connect with you directly in LinkedIn, you will enter their names and email addresses. This information will be used by LinkedIn to send your invitation including a message that you write. The names and email addresses of people that you invite will be used only to send your invitation and reminders.

I decided to search for accounts through your "Address Book Contacts" function, and manually entered my email contacts. I only used this function to find existing users, and not invite new ones. I expected the information to be immediately deleted from your servers, as it had no more use for the contacts I gave, but I found out today they were still there when one of said addresses was used to sign up a new account and LinkedIn informed me of such. While this is a nice feature, it would have been appropriate to allow the user to opt out of having LinkedIn keep the emails for further use, and downright shady that the user is not informed at all that given email addresses are kept by LinkedIn on your servers.

Dear Jeffrey
We are aware of the issue you are currently experiencing and we are working diligently to resolve the issue. We appreciate your patience while this issue is being resolved.

I thought it obvious from this reply that they did not take what I said into consideration, and a high probability that they didn’t really even read it. I mentioned in the last post this exact thing happened to my friend who was trying to communicate with LinkedIn about a problem he was having with errors with their site code. This kind of thing is typical from large corporations that receive a large amount of communications and do not have the staff to handle it. I consider this practice almost as bad as out-sourced tech support (usually India), another pet peeve of mine, as communication is often hard and the tech support agents often don’t know what they are talking about... at least very much more so than when with other first-tier tech support channels provided in-country ^_^; . I went ahead and contacted eTrust a few days ago in hopes that I get a more personal response from them.

Since I just added my résumé which mentions my LinkedIn page, I thought I’d mention something else I just discovered about LinkedIn.

I would normally never give out any of my contacts’ email addresses to third parties under any circumstance, but I decided there was very little risk to do so at LinkedIn because it is a largely used website with many users that is also eTrust certified. Unfortunately, I have also heard eTrust certification isn’t exactly hard to get and shouldn’t have too much stock put in it, but it is still something.

Anyways, after reading LinkedIn’s privacy policy, I decided it would be ok to list some of my email contacts to discover if they also used LinkedIn or not. I, of course, added in a dummy email address of mine into this to watch for spam or advertisements, and it has to date not received anything, though I’m sure any company that illegally released email addresses wouldn’t be stupid enough to let go of newly acquired addresses immediately, but then again, I always assume too much of people/corporations... but I digress. I have discovered that they keep all the emails you give them because one of the emails I gave was recently used to sign up for a new account and LinkedIn immediately informed me of this.

While this is a nice extension to the "find your contacts through their emails" function, LinkedIn really should have given me an option to opt out of this, or at the very least informed me that it was keeping the emails I gave it on record. Unfortunately, even if they do have a good privacy policy and abide by it, there is still the chance a rogue staff member could harvest the emails and sell them.

Oh, LinkedIn is also a very buggy system in and of itself. I very often get timeout errors and many other errors to the extent of “The server cannot perform this operation at this time, please try again later”. A friend of mine has also been having trouble linking our profiles together for more than a week now, with no response to his email to them… besides a type of auto response he got back that had absolutely nothing to do with the reported problem.