A client wanted to put her browsing history to work. She wanted her own website to display the URLs of selected other sites every time she visited them. It was her way of giving credit on her own website to other sites she frequently referred to. Ideally she wanted to capture her browser URLs in real time and be able to easily filter those she wanted displayed. The idea seemed a simple one - but how to implement it?

Although the client did work across several computers, she was primarily a Firefox user. My proposed solution was to write a Firefox extension that would capture website visits in real time and store them in a database table on the client's own website. The client would have to filter first-time visits through an administrator interface, but only once. If she chose not to display a website, any future visits would be ignored; conversely, websites she accepted for display would automatically be displayed in future.

I'd not previously written a Firefox extension, but there's extensive documentation. There's even a wizard which creates extension stub files, requiring only the blanks to be filled in. What's more, the Mozilla family's visual components are built in XUL, which is a delight to use. It's everything HTML ought to be, as the following code snippet demonstrates, producing the content of the options dialog to the left.

<groupbox>
  <caption label="Settings"/>
  <grid>
    <columns>
      <column flex="1"/>
      <column flex="4"/>
    </columns>
    <rows>
      <row>
        <label control="active" value="Active"/>
        <checkbox id="active" preference="linkharvester-active"/>
      </row>
      <row>
        <label control="url" value="URL"/>
        <textbox id="url" preference="linkharvester-url" size="64"/>
      </row>
      <row>
        <label control="guid" value="GUID"/>
        <textbox id="guid" preference="linkharvester-guid" size="64"/>
      </row>
    </rows>
  </grid>
</groupbox>	      

When it comes to the real code, however - the commands and variables that actually makes things happen under the hood - Mozilla extensions rely on javascript. That's a mixed blessing but, well, it works. I did find a handy tool which no extension-builder should be without: Chromebug provides visibility of javascript code that's being run deep within the browser.

So what does the Firefox extension do? Very simply, when a new web page is loaded into Firefox, it sends the URL and page title to the specified server, along with a unique identifier so the server knows the data is from a legitimate source. Generally browsers don't permit cross-domain ajax calls, and I'd thought I might need to use GreaseMonkey's excellent workaround. However because this code is run within Firefox's chrome, rather than at the page level, the same origin policy doesn't apply and a standard ajax call works well. Here's the code.

var linkharvester = function () {
  var prevURL = '';
  var prefManager = Components.classes["@mozilla.org/preferences-service;1"].getService(
    Components.interfaces.nsIPrefBranch);
  return {
  
    init: function () {
      gBrowser.addEventListener("load", function () {
        var active = prefManager.getBoolPref("extensions.linkharvester.active");
        if (active) {
          linkharvester.run();
        }
      }, false);
    },

    run: function () {
      var logURL = prefManager.getCharPref("extensions.linkharvester.url");
      var logGUID = prefManager.getCharPref("extensions.linkharvester.guid");
      if (logURL != '' && logGUID != '') {
        var newURL = window.top.getBrowser().selectedBrowser.contentWindow.location.href;
        var docTitle = window.top.getBrowser().selectedBrowser.contentWindow.document.title;
        if (newURL != '' && newURL != 'about:blank' && newURL != prevURL) {
          prevURL = newURL;
          ajaxGet(logURL + '?guid=' + Url.encode(logGUID) + '&url=' + Url.encode(newURL) + 
            '&title=' + Url.encode(docTitle));  //, testHandler); // testing only
        }
      };
    }
  };
}();
window.addEventListener("load", linkharvester.init, false);

The process starts with the final line: we add a listener so that when a page is loaded, our object's init method is invoked. If the options dialog's Active checkbox is checked, init calls the run method. If the other two values in the options dialog have been populated the method retrieves the page URL and title and uses a standard ajax call to send the details to the specified server. There are two things to note.

1. Testing showed the run method being invoked multiple times for each web page visited. Storing each URL we process lets us avoid redundant ajax calls.
2. Our ajax method accepts a second, optional, parameter: a reference to a callback which would (usually) process the result returned from the server. We're not going to use the result, but inserting a callback during testing provides an easy check that the method has been successful end-to-end.

Gratuitous screenshot from the Firefox addons menu to provide visual relief.

On to the server, where a PHP method takes the URL and writes it to the database. Well, there's a little more to it than that. First of all, we only work with the base URL, which reduces the data volumes substantially. Secondly, if a URL has previously been inserted, we only update the timestamp.

function Execute($request) {
  $guid = (isset($_GET['guid']))
    ? urldecode($_GET['guid'])
    : "";
  $url = (isset($_GET['url']))
    ? parse_url(urldecode($_GET['url']))
    : "";
  $title = (isset($_GET['title']))
    ? urldecode($_GET['title'])
    : "";
  if ($guid == g_guidLinks) {
    $id = $this->db->queryUniqueValue('
      SELECT id 
      FROM links
      WHERE url = "'.$url['scheme'].'://'.$url['host'].'"
    ');
    if ($id && $id > 0)
      $this->db->execute('
        UPDATE links
        SET date = NOW()
        WHERE id = '.$id
      );
    else
      $this->db->execute('
        INSERT INTO links
        (url, title, date)
        VALUES ("'.$url['scheme'].'://'.$url['host'].'", "'.$title.'", NOW())
      ');
  }
  /* Return values for testing
  $data = array();
  $data['url'] = $url['scheme'].'://'.$url['host'];
  $data['title'] = $title;
  $data['guid'] = $guid;
  return $data;           
  */
}

What the above code doesn't reveal is the presence of another field in the database table. The status field has a default value of 0 which means "I'm new here and haven't been assessed yet". The client's admin page displays all links with a status of 0 and provides the option to change that status to -1 ("Don't tell anyone I visited this site") or 1 ("Put this link on my website").

When building the client's website pages, a database query simply returns the five most recent links with a status of 1 ... easy peasy. The date field is then parsed to deliver a relative time - for example "Trois heures depuis". This is loaded as the title for the link, appearing when the link is moused over.

Creating the Firefox extension proved much easier than I thought it might be. And the client is delighted with the implementation of her idea, although she complains that the list of sites to be assessed sometimes gets dauntingly surdimensionné.

If you liked this post

share it around.

If you really liked this post

a €1 donation helps.

If you liked this post a lot

don't miss the next one.