FPWD of Pointer Lock 2.0

# Léonie Watson (6 months ago)

Hello WebPlat,

With thanks to Vincent and Xiaoqian, the FPWD of Pointer Lock 2.0 has been published [1]. Comments and contributions from the WG are welcome [2].

Changes from the Pointer Lock 1.0 Rec focus on Shadow DOM content, and are noted in the Status section of the 2.0 spec.

Léonie. [1] www.w3.org/TR/2016/WD-pointerlock-2-20161122 [2] w3c.github.io/pointerlock

Contact us to advertise here
# Florian Bösch (22 days ago)

I have just encountered an awkward usage scenario in a client project where pointerlock is used in conjunction with fullscreen. I believe this would be instructive to review to inform future spec-work.

The usage scenario involves a WebGL "viewer" component that may operate in different camera modes. Usually a fullscreen button is offered that sets the container containing the canvas (and some UI elements) into fullscreen mode.

One of the camera modes offered is a first-person camera where pointerlock is used to capture the mouse to control the viewpoint, this is another button in the UI. When pointerlock is requested, the only way to exit it is the ESC key (as implemented by the UA and which for good reason cannot be overriden).

If a user enters fullscreen and then activates first-person camera, the ESC key throws the viewer component both out of fullscreen and out of pointerlock.

This is awkward because from the sequence of events leading up to being in pointerlock/fullscreen does not indicate that the users intention was to end both pointerlock and fullscreen. The viewer components UI can only be operated if pointerlock is ended however, and so the user only gets to have a choice of this pointerlock+fullscreen UI interaction:

  1. enter fullscreen by clicking on the fullscreen button
  2. enter first-person view by clicking on the first-person camera button
  3. exit by ESC in order to click on the UI which ends both pointerlock and fullscreen (and ends first-person camera view mode because without pointerlock it's painful to use)
  4. can now interact with the UI
  5. enter fullscreen again by clicking on the fullscreen button
  6. enter first-person view again by clicking on the first-person camera button

The workaround adopted in this instance was to assume that first-person view, pointerlock and fullscreen are intended to be used together (which probably more likely than not), and hence trigger all three together, such that all three may be re-engaged once the user has pressed ESC to escape all of them. This is not optimal by any stretch of the imagination.

Many applications (particularly games) may wish to let the user interact with UI elements (often represented by the DOM), while staying in fullscreen, but may wish to offer pointerlock nevertheless. Emulating a cursor is not desirable because it's not as smooth as native cursors, cannot match the native cursors representation and has difficulty synthesizing DOM events for full compatibility.

# Vincent Scheib (22 days ago)

Thanks for the feedback Florian.

My suggestion is to use something other than ESC to exit pointer lock via javascript, and thus not exit fullscreen. E.g. the 'f' key, '~', space bar, 'q', etc.

The reason ESC was specified to exit both was to satisfy concerns of users not knowing how to exit these features. The concept of stacking the permissions adds conceptual complexity.

Essentially, we believed there would be many users confused by a page entering both fullscreen and pointer lock, wanting to get out, trying ESC and not having fullscreen exit. They barely understand that pointer lock was also entered and is now exited. They may not understand the difference between an application drawn cursor and the now visible system cursor. They may not think to keep repeating pressing ESC.

# Vincent Scheib (22 days ago)

Also, Keyboard Lock is a potential future solution for fullscreen applications that want to handle ESC presses. www.chromestatus.com/feature/5642959835889664

# Florian Bösch (22 days ago)

On Fri, Apr 28, 2017 at 6:59 PM, Vincent Scheib scheib@google.com wrote:

My suggestion is to use something other than ESC to exit pointer lock via javascript, and thus not exit fullscreen. E.g. the 'f' key, '~', space bar, 'q', etc.

You cannot override ESC, and the UA informs the user that to regain his cursor, he should press ESC. It's inevitable that the user will actually do so, since he's told to do so. From an UX point of view, designing an info box explaining to the user to avoid pressing ESC (and ignore that other info box), because that may not be what he wants to do, and instead press another key, while another info box is hovering on screen telling him to press ESC sounds really bad, but I'm not expert, so... why don't you ask an UX expert how confusing that will be to a user?

The reason ESC was specified to exit both was to satisfy concerns of users not knowing how to exit these features. The concept of stacking the permissions adds conceptual complexity.

Essentially, we believed there would be many users confused by a page entering both fullscreen and pointer lock, wanting to get out, trying ESC and not having fullscreen exit. They barely understand that pointer lock was also entered and is now exited. They may not understand the difference between an application drawn cursor and the now visible system cursor. They may not think to keep repeating pressing ESC.

Here's an example of a game (counter-strike) coming from a first-person control to using a pointer based modal to operate a menu and then go back to the first-person control: youtu.be/u8YbYBZtbAs?t=442 . The menu is triggered by the key B. However counter-strike does not have to deal with the fact that whenever a user returns from the menu to the first-person view, that a black box is drawn over their viewport informing the user to press ESC...

# Florian Bösch (22 days ago)

ESC is actually a common key to trigger a menu in many games, and it usually does not mean exit fullscreen.

# Vincent Scheib (22 days ago)

My proposal for today is to use other buttons. I'm aware of the common game paradigm of ESC exiting games. I've played plenty of counter strike, ;) but we also needed a reliable unlock gesture.

My implementation in Chrome does not re-show The ESC instructions when a page calls exitPointerLock and then requestPointerLock again - specifically to avoid the nuisance you mentioned.

Keyboard Lock, if it progresses, will offer applications & games the ability to override taps to ESC key (the user agent will use a sustained hold of ESC to force exit). See linked explainers and spec drafts here: www.chromestatus.com/feature/5642959835889664

Want more features?

Request early access to our private beta of readable email premium.