[permissions] Defining a testing API

# Mike Pennisi (2 months ago)

Hello all,

The Permissions specification operates on state that cannot be influenced in a way that is both standard and automated. This is largely by design. Permission fundamentally originates from the user, so allowing arbitrary code to control it doesn't make much sense.

...in production settings. There is an important use case for controlling state programmatically: automated tests.

The ability to write automated tests for the Permissions API would allow shared test suites like Web Platform Tests [1] to promote consistency among browsers, and it would also allow application developers to more thoroughly test their own code.

The Permissions API isn't the only web standard that would benefit from dedicated testing support. All of the spec's "powerful features" are in a similar position. In all cases, programmatic control is undesirable for production settings but essential for conformance testing and application testing.

This is currently being discussed on the public-test-infra mailing list [2], specifically in the context of the developing Web Bluetooth API. In that thread, we're considering two very different approaches:

  1. Each standard defines a JavaScript API for testing with the understanding that it is only to be enabled under highly-controlled circumstances
  2. Each standard includes extensions to the WebDriver specification using the mechanism it defines for this purpose [3]

Members of the Chromium team have prototyped a straw man implementation of the first approach for the Web Bluetooth API. I'm personally interested in exploring the second approach. I think that the relatively small internal state of the Permissions API make it a great candidate for experimentation along these lines.

Before getting started on a patch, I wanted to gauge interest from those involved with the specification. How do folks here feel about trying this out?

Mike Pennisi

Bocoup

[1] w3c/web-platform-tests [2] lists.w3.org/Archives/Public/public-test-infra/2017AprJun/0018.html [3] w3c.github.io/webdriver/webdriver-spec.html#extensions

Contact us to advertise here
# Mounir Lamouri (2 months ago)

That sounds like a good idea. For what it's worth, the Blink LayoutTests are using a setPermission() which is mostly used as part of a helper [1] for boring internal plumbing reasons. Eventually, clear() or clearAll() methods could be added but so far they have not been needed.

[1] cs.chromium.org/chromium/src/third_party/WebKit/LayoutTests/http/tests/resources/permissions-helper.js

# Simon Stewart (a month ago)

TL;DR: we'd love an API for this kind of thing for use with WebDriver, and I'm on board for trying it out.

Longer version:

We had a discussion about work for Level 2 of the WebDriver spec at our face-to-face meeting in Lisbon. One of the topics we discussed was an API for dealing with "door hanger [https://wiki.mozilla.org/Firefox/Projects/Doorhanger_notifications](https://wiki.mozilla.org/Firefox/Projects/Doorhanger_notifications)"

notifications --- or, in a more general sense, popups that display a limited set of options, possibly allowing some text input. This would cover things such as alerts for desktop notifications, allowing geolocation, and Web Bluetooth, as these are typically presented through the same style of UI.

Andreas and I sketched out a basic API which was essentially similar to that presented for user prompts [https://w3c.github.io/webdriver/webdriver-spec.html#user-prompts](https://w3c.github.io/webdriver/webdriver-spec.html#user-prompts), with

the option of sending additional text and perhaps a button identifier when accepting the prompt. I'm sure Andreas will correct my recollection if it's not quite right.

Other than solving the general case, the other option is (as you say) to use the WebDriver extension mechanisms to add specific and focused APIs for each spec. If you'd like a member of the group working on WebDriver to provide some help, I'm sure we'd be able to offer some.

Kind

# Mike Pennisi (a month ago)

One of the topics we discussed was an API for dealing with "door hanger" notifications --- or, in a more general sense, popups that display a limited set of options, possibly allowing some text input. This would cover things such as alerts for desktop notifications, allowing geolocation, and Web Bluetooth, as these are typically presented through the same style of UI.

Andreas and I sketched out a basic API which was essentially similar to that presented for user prompts, with the option of sending additional text and perhaps a button identifier when accepting the prompt.

This sounds like a necessary capability for testing, but I don't think it can be designed in advance of a standard mechanism for specifying these prompts (certainly not their look-and-feel, but at least their input elements and their effects).

Up until now, I have been assuming that the Permissions specification was the appropriate place to address this. If there are use cases for rich browser-provided prompts beyond permissions, maybe it makes sense to define them in a dedicated spec instead. Does that sound right to you?

# David Burns (a month ago)

Yes, these should be in a dedicated section for the specification in question. If the base primatives are missing from WebDriver there should be a request to get those in to build the rest. E.g. If its a door hanger we can use the user prompts API (though I am not conviced that is the right API but now isnt the time to bikeshed.) but if its controlling hardware (Web Bluetooth as an example) then we may need something in WebDriver AND web bluetooth APIs.

David

David Burns Email: david.burns@theautomatedtester.co.uk URL: www.theautomatedtester.co.uk

# Philip Jägenstedt (a month ago)

Alex. Gouaillard is also looking to defining a WebDriver API for getUserMedia() prompts. You should probably collaborate :)

Does anyone have a rough idea of the shape for this extension yet? Would it be reacting to permission requests, taking the place of the prompts, toggling the permissions independent of requests, or both?

# Mike Pennisi (a month ago)

Does anyone have a rough idea of the shape for this extension yet? Would it be reacting to permission requests, taking the place of the prompts, toggling the permissions independent of requests, or both?

Reading David's response again, I'm not so sure. My previous message was a little unclear, so my original question still seems relevant. I'll try to rephrase in a more coherent way:

Simon mentioned how he and Andreas were designing an API for interacting with "door hanger" notifications. This sounds completely necessary, but I suggested that first, we would need a standard mechanism through which those notifications could be created. My question was: does that mechanism need to be defined in a dedicated specification? It seems possible that it could be contained within Permissions, but if other specs need this ability in contexts other than permission management, then maybe not.

If there was a standalone "Privileged User Prompt" spec, then introducing automation abilities there might preclude the need to do so within Permissions. In that world, Permissions would reference the "Privileged User Prompt" spec in "Prompt the user to choose" [1]. Instead of scripting the internal "permissions" state directly, test code would control permissions through scripted interaction with the notifications themselves.

Although this would be less artificial, I'm not sure the distinction would matter in a practical sense. This supposed "Permissions-to-Notification" interface would be an implementation detail to web developers, so whether it was exercised or not probably wouldn't be observable from application code anyway.

...but I'm getting ahead of myself. Can anyone say whether a new specification is appropriate?

[1] w3c.github.io/permissions/#prompt-the-user-to-choose

# Simon Stewart (a month ago)

Inline, and to everyone this time.

On Fri, May 12, 2017 at 3:54 PM, Mike Pennisi mike@bocoup.com wrote:

Does anyone have a rough idea of the shape for this extension yet? Would it be reacting to permission requests, taking the place of the prompts, toggling the permissions independent of requests, or both?

Reading David's response again, I'm not so sure. My previous message was a little unclear, so my original question still seems relevant. I'll try to rephrase in a more coherent way:

Simon mentioned how he and Andreas were designing an API for interacting with "door hanger" notifications. This sounds completely necessary, but I suggested that first, we would need a standard mechanism through which those notifications could be created.

As a browser vendor, or someone implementing specifications, I'd agree with that ordering. As a user of a browser, I'd disagree: the UI is already in place in both Chrome and Firefox (IME --- those are the browsers I use most), and users have already been trained to look for them in those browsers.

A standard mechanism would be lovely from an implementation point of view.

My question was: does that mechanism need to be defined in a dedicated specification? It seems possible that it could be contained within Permissions, but if other specs need this ability in contexts other than permission management, then maybe not.

It needs to be defined somewhere. One approach would be to define it in the Permissions specification with an eye to sharing it between different specs if it proves generally useful. Another approach would be to define the webdriver extensions required by the Permissions specification to be exactly specific for that spec. Either way, I think it's best done as part of the Permissions specification for now.

In both cases, I'd be happy to advise on the webdriver specific parts, as, I'm sure, would others in the WG.

If there was a standalone "Privileged User Prompt" spec, then introducing automation abilities there might preclude the need to do so within Permissions. In that world, Permissions would reference the "Privileged User Prompt" spec in "Prompt the user to choose" [1]. Instead of scripting the internal "permissions" state directly, test code would control permissions through scripted interaction with the notifications themselves.

Agreed.

Although this would be less artificial, I'm not sure the distinction would matter in a practical sense. This supposed "Permissions-to-Notification" interface would be an implementation detail to web developers, so whether it was exercised or not probably wouldn't be observable from application code anyway.

...but I'm getting ahead of myself. Can anyone say whether a new specification is appropriate?

The lowest friction thing to do right now is to add a section in the Permissions spec that defines the webdriver URLs and messages that would be needed. If this WG has a F2F at TPAC, we could add this an agenda item too.

Kind

# Mike Pennisi (a month ago)

Okay, great! I've created a Google Document to begin brainstorming what this patch would look like. If you folks could validate the scope of work I'm proposing, then I would be happy to draft a patch for the Permissions API. From there, we could have a more concrete discussion via a GitHub pull request, knowing that we were on the same page in terms of the desired outcome.

docs.google.com/document/d/1Oe4VhgdFnZ6ID3WGyG97n_b1khvYsRcX7T4ddNcyJ9A/edit#

I'm specifically interested in what Mounir and Simon have to say, but I'd welcome input from anyone on the list.

Thanks! Mike

# Simon Stewart (a month ago)

It's going to be at least a week until I can look at this, but I'll do so as soon as I can.

Kind

# Mike Pennisi (a month ago)

Understood. Thanks, Simon

# Alexandre GOUAILLARD (a month ago)

same on my side, nothing before june 1st. Sorry about the inconvenience.

# Philip Jägenstedt (24 days ago)

Thanks for preparing that doc, Mike! I've commented and asked for some discussion of the options and which we should pursue first.

On Wed, May 24, 2017 at 11:41 PM Alexandre GOUAILLARD agouaillard@gmail.com

wrote:

# Simon Stewart (21 days ago)

Thanks for preparing the document. I've added a couple of comments to it.

Do we feel the best place to have the discussion is in the comments of that doc, or here in this thread? I'm happy either way, but (ignoring extensive evidence to the contrary) I'm not fond of repeating myself :)

Cheers,

Simon

On Mon, May 29, 2017 at 4:45 PM, Philip Jägenstedt foolip@google.com

wrote:

# Philip Jägenstedt (20 days ago)

+Youenn Fablet youenn@apple.com FYI, this will have some relevance for

WebRTC test automation.

On Thu, Jun 1, 2017 at 2:40 PM Simon Stewart simon.m.stewart@gmail.com

wrote:

# Mike Pennisi (a day ago)

I've submitted a patch to the Permissions specification based on our discussion here and in that document:

w3c/permissions#151

It's still very rough, but it's about time we moved the discussion forward with something more concrete. Please go poke holes :)

Mike

Want more features?

Request early access to our private beta of readable email premium.