Setting request headers on Selenium Webdriver with Ruby
The Proxy Solution |
(or authorize a proxy)..
(or get/set cookies)..
(or get response headers)..
(or set referer)...
Selenium Webdriver (2.0) is the industry standard for web testing
and it supports many browsers (Chrome, Firefox, IE, Safari, etc..).
Unfortunately Webdriver does not support setting/modifying
the request headers or getting the response codes or response headers,
and in fact the Selenium team has made it very clear that this
will not be supported in a heavily contentious debate with the users.
I have a project where I need to use a proxy with Selenium Webdriver that
uses authorization and it needs to be fully automated. Normally I would
add an HTTP header such as:
Proxy-Authorization: Basic ZG8geW91IHRoaW5rIEkgYW0gc3R1cGlkPw==
Unfortunately, Webdriver does not allow the user to change the HTTP
headers, and it never will.
you can't get the response headers or status either)
You can pass the user@passwd in the URL on some browsers, but that
doesn't work on all browsers, and I think that's only for normal
authentication and not proxy authentication. Regardless, it didn't work
My next thought was that since I could alter the proxy source, I could
send the authorization (I couldn't remove authorization since I needed
to distinguish between users) using a cookie, but that would require
setting a cookie before each web page, and Webdriver does not allow that,
and it never will.
To further complicate the matters, I also need to set the referer
for the pages in my testing since the referer field is used by many
websites and I need the testing to be like real usage.
The Proxy Solution
So now what? I tried to find an alternative to Selenium, but I
needed cross-browser support, and there didn't seem to be any
reasonable options. I looked into a number of Selenium solutions,
but most of them were Firefox only. Finally I settled on using
a second, local proxy, and after trying to deal with other people's code, I realized
that I could simplify my ruby proxy code (which is threaded and supports
the vast majority of sites) to make a local proxy which could rewrite
headers, do authorization, add cookies, set the referer, and so on.
And best of all I realized that since I could just run the proxy
in a thread inside my test/Selenium code, then I could control the
headers directly, though I also wrote code that would let me
change the headers by calling a special proxy URL as well, in case
I wanted to run it as a separate script.
First the script is available as a standalone simple proxy, you can
set proxy or headers in the script or else by calling the proxy URL,
as an example, setting the proxy and an authorization header:
Here's the script - incidentally this also works as just a simple
proxy and there aren't many working, simple ruby proxy examples
available (using just TCPSocket):
And here's a full Selenium Webdriver example that starts up the
proxy in a thread, sets up the proxy settings (to use another proxy
with authorization) and then uses chrome to fetch that page (first
through the local proxy and then through the external proxy). It
also sets the referer, so you could easily remove the setting of
the external proxy if you just want to use the local proxy to
rewrite headers for a regular Selenium test.
- URL user/password
- Example: 'http://user@password:somedomain.com/'
- Selenium RC
Back to Solutions.