A change in password manager

In my last post I talked about how password manager currently works. Starting in Firefox 26 (assuming all goes well), it will work a bit differently.

Currently, the password manager must look at each page that loads, and process it to see if it might be a login page (and if so try to fill in a saved username/password). This is kind of dumb and has always annoyed me. For one, it’s not really great for performance. We spend a small amount of time to do this on every page, when only a tiny fraction of pages will actually be login pages. Nor is it suited to the modern web — many pages do interesting things after they’ve initially loaded, but the password manager only attempts to do its work at initial page load. If a page dynamically adds a username/password field to the DOM afterward (in response to, say, the user clicking a “login” button), we never notice and thus can’t offer to fill in a login.

Starting with tomorrow’s Firefox 26 nightly build (or the day after, depending on how merges and build go), this whole process will instead be triggered when an <input type=password> is appended to the DOM — be it from loading static HTML, or when the page dynamically adds one later. This means password manager can skip processing most page loads (i.e., the ones without password fields). It also means it can now fill in logins on some sites that it couldn’t before.

This new functionality comes via bug 355063, with major supporting work from bug 771331 (which added the chrome-only DOMFormHasPassword event) and bug 839961 (which refactored password manager to prepare it for using this event). It’s also interesting to mention that bug 762593 has already added code to use DOMFormHasPassword, in order to log console warnings when a page is using password fields insecurely.

There is one potential downside.

This changes the timing of when passwords are filled, so if a site is setting or clearing the contents of login fields, it’s possible it could blow away a login that previously hadn’t been filled yet. For example, I’ve seen sites clear fields as a poor security measure (I guess to deter password managers?), as well as set fields (to values like “Enter username here”). The latter is a great example of when and why the HTML5 placeholder attribute should be used.

I don’t think this change is a significant risk, and is definitely worth it for the other benefits. But should you run across it, there will be a signon.useDOMFormHasPassword pref that can be flipped to revert to the old code. (This is only a temporary pref, it will go away once we’re sure this isn’t a big problem.) And, again, if you find problems please file a bug containing debug info.

On Firefox’s Password Manager

It’s been a while since I last blogged about Firefox’s password manager. No big deal, it really hasn’t fundamentally changed since since I rewrote it 6 years ago for Firefox 3.0. Well, ok, the storage backend did switch to SQLite, but that’s mostly invisible to users. There’s a noteworthy change coming soon (see next post!), but I figured that would be easier to explain if I first talked about how things currently work. Which I’ve been meaning to do for a long time, ahem.

The first thing you should know is that there is no standard or specification for how login pages work! The browser isn’t involved with the login process, other than to do generic things like loading pages and running javascript. So we basically have to guess about what’s going on in order to fill or save a username/password, and sometimes sites can do things that break this guesswork. I’m actually surprised I don’t get questions about this more often.

So let’s talk about how two of the main functions work — filling in known logins, and saving new logins.

Filling in a known login

The overall process for filling in an existing stored login is simple and brutish.

1. Use the chrome docloader service and nsIWebProgress to learn when we start loading a new page.
2. Add a DOMContentLoaded event listener to learn when that page has mostly loaded.
3. When that event fires…
   1. Check to see if there are any logins stored for this site. If not, we’re done.
   2. Loop through each form element on the page…
      1. Is there an <input type=password> in the form? If not, skip form.
      2. Check to see if any known logins match the particular conditions of this form. If not, skip form.
      3. Check to see if various other conditions prevent us from using the login in this form.
      4. Fill in the username and/or password. Great success!

Phew! But it’s the details of looking at a form where things get complex.

To start with, where do the username and password go? The password is fairly obvious, because we can look for the password-specific input type. (What if there’s more than one? Then we ignore everything after the first.) There’s no unique “username” type, instead we just look for the first regular input field before the password field. At least, that was before HTML5 added more input types. Now we also allow types that could be plausibly be usernames, like <input type=email> (but not types like <input type=color>). Note that this all relies on the order of fields in the DOM — we can’t detect cases where a username is intended to go after the password (thankfully I’ve never seen anyone do this), or cases where other text inputs are inserted between the actual username field and the password (perhaps with a table or CSS to adjust visual ordering).

And then there’s the quirks of the fields themselves. If your username is “alice”, what should happen if the username field already has “bob” filled in? (We don’t change it or fill the password.) Or, more common and depressing, what if the username field already contains “Enter your sign in name here”? In Mongolian? (We treat it like “bob”.) What if the page has <input maxlength=8> but your username is “billvonweiterheimerschmidt”? (We avoid filling anything the user couldn’t have typed.)

And then there’s the quirks of the saved logins. What if the username field already has “ALICE” instead of “alice”? (We ignore case when filling, it’s a little trickier when saving.) Is there a difference between <input name=user> and <input name=login>? (Nope, we stopped using the fieldname in Firefox 3 because it was being used inconsistently, even within a site.) What about a site has both a username+password _and_ a separate password-like PIN? (Surprisingly, we were able to make that work! Depending on the presence of a username field, we prefer one login or the other.)

And then. And then and then and then. Like I said, there’s no spec, and sometimes a site’s usage can break the guesses we make.

Saving a new/changed login

In comparison, the process for saving a login is simpler.

1. Watch for any form submissions (via a chrome earlyformsubmit observer)
2. Given a form submission, is there a password field in it? If not, we’re done.
3. Determine the username and password from the form, and compare with existing logins…
   • If username is new, ask if user wants to save this new login
   • If username exists but the password is different, ask if user wants to change the saved password
   • If username and password are already saved, there’s nothing else to do.

Of course, there are still a number of complicating details!

This whole process is initiated by a form submission. If a site doesn’t actually submit a form (via form.submit() or <button type=submit>), but just runs some JavaScript to process the login, the password manager won’t see it. And thus can’t offer to save a new/changed login for the user. (Note that this is easy for a site to work around — do your work in the form’s onsubmit, but return |false| to cancel the submission).

Oh, and there’s still the same question as before — how to determine which fields are the username and password? We reuse the same algorithm as when filling a page, for consistency. But there are a few wrinkles. The form might be changing a password, so there could be up to 3 relevant password fields (for the cases of: just a new password, old and new passwords, and old + new + confirm.). And the password fields could be in any order! (Surprisingly, again, this works.) The most common problem I’ve seen is an account signup page with other form fields between the username and password, such as “choose a user name, enter your shipping address, set a password”. The password manager will guess wrong and think your zipcode is your username.

Oh, and somewhere in all this I should mention how differences in URLs can prevent filling in a login (or result in saving a seemingly-duplicate login). Clearly google.com and yahoo.com logins should be separate. But we also match on protocol, so that a https://site.com login will not be filled in on http://site.com. And what about http://www.foo.com and foo.com or accounts.foo.com? (We treat them separately.) What about mail.mozilla.com and people.mozilla.com? (Also separate.) What you might not realize is that we also use the form’s action URL (i.e., where the form is submitted to), ever since bug 360493. While this prevented sending your myspace.com password to evilhacker.com, it also breaks things when a site uses slightly different URLs in various login pages, or later changes where their login forms submit to.

Oh, bother.

All the gory details

This is one of the benefits of being Open Source. If you want to see alllll the gory details of how the Firefox password manager works, you can look at the source code. See http://mxr.mozilla.org/mozilla-central/source/toolkit/components/passwordmgr/. In particular, LoginManagerContent.jsm contains the code implementing the stuff discussed in this post, with the main entry points being onContentLoaded() and onFormSubmit().

Finally (!), I’ll mention that the Firefox password manager has some built in logging to help with debugging problems. If you find it not working in some particular case, you can enable the logging and it will often make the problem clear — or at least clearer to a developer when you file a bug!