Selenium – decisions

Blog: Moving to Trunk

Wed, 01 Jul 2020 00:00:00 +0000

Since the project started we have been following trunk based development. This was a very natural fit when we were using SVN over a decade ago on Google Code.

As Google Code shut down we moved to GitHub and the git model of doing things. We moved there mostly due to the gravity that GitHub had created in Open Source projects.

This meant that we followed the standard use of master as our trunk to work off. Now that GitHub, and services that use GitHub, have improved support for non-master branches as default we are moving our default branch to trunk. It describes how we, as a project, work and is a more inclusive term.

If you have pull requests based on master we will see about moving that over to the trunk branch ourselves. If we can’t, we may ask you to help with the rebasing.

We, as a project, want to make our community inclusive and this is just one step in making sure we are. Other steps we taking are improving our Code of Conduct and Community Guidelines.

Join us on Slack or IRC if you wish to discuss this further.

Blog: Source Control

Mon, 14 Jan 2013 00:00:00 +0000

This short technical note is to announce that the Selenium project is now using git on Google Code in place of subversion.

The move has been a long time in the making, and it’s largely thanks to the efforts of Kristian Rosenvold that we’ve been able to do the migration and retain the project history. The project owes him a huge thank you! We’re in the process of migrating the last bits and pieces (none of which are user facing), so there may be some last minute turbulence as we settle everything down.

Although the canonical source will be on Google Code, we’re working on setting up a github mirror. We’ll announce the location of that once it’s set up.

Blog: Going Atomic: How

Sun, 05 Sep 2010 00:00:00 +0000

This is the second of my technical posts. Again, if you’re interested in the internal workings of Selenium 2, then please skip straight to something else. If you’re interested in how and why we made some of the technical decisions on the project, keep reading….

We left our intrepid heroes in a tight spot: they’d decided to write a shared library of code, to be used by the various webdriver implementations and selenium core, but the requirements for doing this seemed to be at odds with it actually happening.

Fortunately, at about the same time we started down this path, Google Open Sourced the Closure compiler. This is a Javascript compiler that takes as input a set of Javascript files, and which outputs Javascript. It can be configured to either pass the code through untouched into a single file, or it can compile a script aggressively, removing unused code-paths and minifying the output as much as possible. The Closure compiler is used on a lot of Google products, so we know that it’s reliable and consistent.

In order to get the best out of the Closure compiler, we’re writing the atoms using the Closure library. This isn’t as well known as some of the other JS libraries out there, but it’s solid, well tested and is being actively developed. It also features an easy-to-use extension of JsUnit, which makes writing tests a far simpler task than might otherwise be the case, and it has an easy to use mechanism for modularizing code.

So, given that we could compile a single Javascript function (and it’s dependencies) into a minified fragment of JS, we were all set, right? Not quite.

The problem is that the atoms are being extracted from two frameworks that have a different way of viewing the world. As an example, Selenium 1’s “getAttribute” method only returns the value of a particular attribute, whereas WebDriver’s “getAttribute” method will return the value of either a property or an attribute (because sometimes it’s hard to remember whether something is an attribute or a property of an element)

As with all problems in computer science, an extra level of indirection is used to solve this issue.

We’re busy implementing the expected behaviour of both WebDriver’s and Selenium’s API on top of the atoms.

There is, of course, the obvious question about how we get this carefully compressed JS into a driver. One option would be to include the raw Javascript as files in each language binding, and pull them in as required. That’s possible, but it would make each language binding bloated, and would introduce a lot of duplication. The alternative is to push the atoms as far into the driver as possible, and this is what we do. As part of the build process for webdriver, we take the compressed JS and convert it into a form that can be consumed by a particular driver. For example, for the IE driver, we convert them into constants in a C header file. These constants can then be referred to by the driver and converted back into a script than be executed via the same mechanism that is used by “executeScript”.

What do we gain from this seemingly baroque approach? Other than the ability to share the same code between drivers? Many things. The cost of maintenance drops dramatically as we can fix a bug in one place and have that fix be picked up by every driver. Because we’re working in pure JS and just querying the DOM, we can run the unit tests in a browser whilst we’re developing the code. This leads to a very tight feedback cycle. It also makes it easier for developers not familiar with the code to take a look at how we do things, and send us patches (always appreciated!) Finally, we can ensure a consistency of result.

Right, any questions?

Blog: Going Atomic: Why?

Mon, 16 Aug 2010 00:00:00 +0000

This is the first in a series of technical posts by me about the internals of Selenium WebDriver. If you’re not interested in technical nitty-gritty, then feel free to step away now.

Still here? Excellent.

Let’s take a step back to just before the Selenium and WebDriver projects merged. There were, very obviously, two separate codebases. Looking closer and with a slightly different perspective, there were more than this. We used the test suites for webdriver to define the behaviour for multiple, largely independent, driver codebases. The IE driver was written in C, the HtmlUnit driver in Java and the Firefox driver is largely Javascript, and so on.

This means that there was a lot of “congruent code”: code that performed the same function but was implemented in a different way. The natural result of this was there was the possibility for behaviour to diverge between drivers. Worse, it meant that when a bug was found, we had to check it in every browser, and it wasn’t certain that an individual could actually fix the code. After all, not everyone is comfortable writing in all the languages we use on the project, or is au fait with all the technologies. For an Open Source project like Selenium, this is a major problem: we rely on a relatively small core of key developers backed up with a far larger team of individuals submitting small changes and fixes. Anything that makes it harder for us to function effectively as a development community is a Bad Thing.

So, we wanted a way off the island; a mechanism that would make it easy to share code between the various drivers and selenium core, that allowed us to fix a bug in one place only and have that fix ripple out to every driver that made use of this mechanism. More importantly, it had to be easy to use, and for someone not familiar with a raft of languages and technologies to quickly get started with.

What would this mechanism look like? Well, there’s a few things that feed into this, but the most important one, was that a majority of the code we’d think of merging was querying the state of the browser (“find an element”, “get the value of this attribute”) and, as Jason Huggins would point out to me at the drop of a hat, the natural language for querying the state of a browser is Javascript. One of the nice things with Javascript is that it’s possible to get a nice, fast development cycle going in it. Just modify a test, save and then hit “refresh” in the browser. That’s kind of attractive. Better still, there are a lot of developers familiar with Javascript.

So, we decided to use Javascript.

Because this shared code was to be composed of the smallest useful fragments of functionality required for browser automation we decided to refer to them as “Browser Automation Atoms”, or “atoms” for short. Rather than write them from scratch, the easiest thing to do was to extract them from the existing code — this is stuff that’s been battle-tested, so we know it’s robust.

There was one very obvious fly in the ointment: not every driver is written in Javascript. Although we have a mechanism available in every browser for executing JS, it’s wildly inefficient to dump an enormous lump of code on to the JS engine of the browser whenever you want to query the DOM. After all, most of the code would not be needed, and not all JS engines have been created equal. Some are blazingly fast. Others, not so much.

It would also be nice to break the code up into manageably-sized modules, rather than being in a single, monolithic file, which implies some clever “module loading” capability. Except this code isn’t always going to be executing inside an environment where writing “script” tags to load additional scripts is possible. You can’t do that in the guts of a firefox extension, though you can load files other ways. However we tie modules together will need to cope with that.

Ah! These opposing requirements: small modules containing the functions we want to use, no extraneous code, and for everything to be in a single file in order to minimize the pain of loading additional modules. That doesn’t sound like a very compatible list. How we resolved those differences is the topic of my next post….