Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Saucelabs: SSL connection issues to my Saucelabs selenium hub #23896

Open
Tracked by #23898
zouhir opened this issue Jun 1, 2020 · 6 comments
Open
Tracked by #23898

Saucelabs: SSL connection issues to my Saucelabs selenium hub #23896

zouhir opened this issue Jun 1, 2020 · 6 comments

Comments

@zouhir
Copy link
Member

zouhir commented Jun 1, 2020

Saucelabs provided me with the following URL to create a test:

https://<sauce_user>:<sauce_key>@ondemand.us-west-1.saucelabs.com:443/wd/hub

WPT's sauce runner is unable to connect to that URL, and it throws the following error:

MaxRetryError: HTTPSConnectionPool(host='ondemand.us-west-1.saucelabs.com', port=443): Max retries exceeded with url: /wd/hub/session (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))

This issue is not too rare in selenium-web driver world, and usually people attempt to do the following for situations of self-signed certs, I tried but no luck:

1- Changes in the capabilities object:
In the following capabilities object, we can ask Webdriver to be OK with self-signed certs:

"acceptInsecureCerts": True

2- changes to Sauceconnect args
passing the "--no-ssl-bump-domains=all" flag to the binary executor command in will not result in any change.

self.sc_process = subprocess.Popen([

Here's a stacktrace dump:

 0:24.28 DEBUG Connecting to Selenium on URL: https://REDACTED:REDACTED@ondemand.us-west-1.saucelabs.com:443/wd/hub
 0:24.28 DEBUG Got command: u'log'
 0:24.66 DEBUG Got command: u'log'
 0:24.66 WARNING Failed to start protocol connection
 0:24.66 DEBUG Got command: u'log'
 0:24.66 WARNING Traceback (most recent call last):
  File "/Users/zouhir/Projects/scroll-timeline/test/wpt/tools/wptrunner/wptrunner/executors/protocol.py", line 47, in setup
    self.connect()
  File "/Users/zouhir/Projects/scroll-timeline/test/wpt/tools/wptrunner/wptrunner/executors/executorselenium.py", line 222, in connect
    desired_capabilities=self.capabilities, keep_alive=True)
  File "/Users/zouhir/Projects/scroll-timeline/test/wpt/_venv2/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/Users/zouhir/Projects/scroll-timeline/test/wpt/_venv2/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/Users/zouhir/Projects/scroll-timeline/test/wpt/_venv2/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 319, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/Users/zouhir/Projects/scroll-timeline/test/wpt/_venv2/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 374, in execute
    return self._request(command_info[0], url, body=data)
  File "/Users/zouhir/Projects/scroll-timeline/test/wpt/_venv2/lib/python2.7/site-packages/selenium/webdriver/remote/remote_connection.py", line 397, in _request
    resp = self._conn.request(method, url, body=body, headers=headers)
  File "/Users/zouhir/Projects/scroll-timeline/test/wpt/_venv2/lib/python2.7/site-packages/urllib3/request.py", line 80, in request
    method, url, fields=fields, headers=headers, **urlopen_kw
  File "/Users/zouhir/Projects/scroll-timeline/test/wpt/_venv2/lib/python2.7/site-packages/urllib3/request.py", line 171, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
  File "/Users/zouhir/Projects/scroll-timeline/test/wpt/_venv2/lib/python2.7/site-packages/urllib3/poolmanager.py", line 336, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/Users/zouhir/Projects/scroll-timeline/test/wpt/_venv2/lib/python2.7/site-packages/urllib3/connectionpool.py", line 765, in urlopen
    **response_kw
  File "/Users/zouhir/Projects/scroll-timeline/test/wpt/_venv2/lib/python2.7/site-packages/urllib3/connectionpool.py", line 765, in urlopen
    **response_kw
  File "/Users/zouhir/Projects/scroll-timeline/test/wpt/_venv2/lib/python2.7/site-packages/urllib3/connectionpool.py", line 765, in urlopen
    **response_kw
  File "/Users/zouhir/Projects/scroll-timeline/test/wpt/_venv2/lib/python2.7/site-packages/urllib3/connectionpool.py", line 725, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/Users/zouhir/Projects/scroll-timeline/test/wpt/_venv2/lib/python2.7/site-packages/urllib3/util/retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
MaxRetryError: HTTPSConnectionPool(host='ondemand.us-west-1.saucelabs.com', port=443): Max retries exceeded with url: /wd/hub/session (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))
@stephenmcgruer
Copy link
Contributor

@zouhir I want to make sure I understand the flow here, since I'm unfamiliar with Saucelabs. My understanding is that you have a copy of WPT, and a copy of Selenium running on the localhost (e.g. your desktop). Selenium is then controlling a browser that is running inside a VM in the Saucelabs CI system, and it ultimately is pointing that browser at test files that are... where? Is there a separate copy of WPT running inside Saucelabs, or is it loading them from your desktop as well?

The above may not exactly impact on this bug, but I want to make sure I understand the setup when trying to figure out what may be going wrong.

I think the main thing will be to figure out if Saucelabs is serving us a valid SSL cert. I'm no SSL expert (cc @Hexcles who may be able to help), but I used the following command:

echo | openssl s_client -showcerts -servername ondemand.us-west-1.saucelabs.com -connect ondemand.us-west-1.saucelabs.com:443 2>/dev/null | openssl x509 -inform pem -noout -text

I've attached the resulting cert. At first glance, it looks ok?
sauce-labs-cert.txt

Also, are you able to test the Selenium relay and see if that works? This looks to be what the code was originally doing, since it used localhost and port 4445 - see https://wiki.saucelabs.com/display/DOCS/Using+the+Selenium+Relay+with+Sauce+Connect+Proxy#UsingtheSeleniumRelaywithSauceConnectProxy-SettingUpYourTeststoUsetheSeleniumRelay . Note that that page specifically mentions problems with Selenium + Python 2.7 + SSL certs.

@Hexcles
Copy link
Member

Hexcles commented Jun 2, 2020

The server cert looks good to me.

Are you running on macOS by the way?

Could you try in Python 2:

r = urllib3.PoolManager().request('GET', 'https://ondemand.us-west-1.saucelabs.com')
print r.data

and see if there's any error?

@zouhir
Copy link
Member Author

zouhir commented Jun 12, 2020

@Hexcles using my current device & python 2.7.16, I currently get from the snippet above:
{"ready":true,"message":"Sauce Labs automated testing is fully operational"}

@Hexcles
Copy link
Member

Hexcles commented Jun 12, 2020

Yet in the same environment you got the stacktrace above? That's really not expected and I don't have any great clue at the moment.

@stephenmcgruer
Copy link
Contributor

Would it be exactly the same environment? In the wpt case we'd be using our virtualenv I guess?

@foolip
Copy link
Member

foolip commented Aug 17, 2020

@zouhir were you able to make any progress on this? What was the command line you tried? We used to run WPT over Sauce Labs for wpt.fyi so this code has definitely worked at one point, but it might have bitrotted since we stopped doing that. (All runs are now performed directly on the VMs with the browsers.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants