facing Mode does not allow to use rear camera for iOS11. about react-qr-reader HOT 39 CLOSED

I debugged this a little bit. My environment is Safari on iOS 11.1 (15B5066f) on an iPhone 6s.

It looks like the response to navigator.mediaDevices.enumerateDevices() changes depending on whether or not the user has given permission to the website to access the devices.

When it is called in getDeviceId(), before the user has given their permission, the response is something like this:

[  
   {  
      "deviceId":"FF7FE1B777981878FE764FCBCB51FA7BB0096E9C",
      "kind":"audioinput",
      "label":"",
      "groupId":""
   },
   {  
      "deviceId":"A18607B2B11F842DCAD5DF278AA227E315FBA807",
      "kind":"videoinput",
      "label":"",
      "groupId":""
   },
   {  
      "deviceId":"B82973FC74E4024B7E59AB229CDB1968D4BA9B6B",
      "kind":"videoinput",
      "label":"",
      "groupId":""
   }
]

However, once the user has given their permission (i.e. getUserMedia was called, and the user approved), if you connect your iPhone to Safari and call navigator.mediaDevices.enumerateDevices() again, the response is something like this:

[  
   {  
      "deviceId":"F739A7B0AFBE13E7C11BBDD5EA1CA6359287244D",
      "kind":"audioinput",
      "label":"iPhone Microphone",
      "groupId":""
   },
   {  
      "deviceId":"9F24387DF50E28745ACA95DC41247E749918B848",
      "kind":"videoinput",
      "label":"Back Camera",
      "groupId":""
   },
   {  
      "deviceId":"562B31EE4CFB8B883B1AA6A4C98474B681C0B4F5",
      "kind":"videoinput",
      "label":"Front Camera",
      "groupId":""
   }
]

Thus, our code fails because the labels aren't provided until user has granted permission. The trick I suppose is to ask the user for permission on one of the video devices. Then call enumerateDevices() again, and pick the right one.

In addition, I also noticed that for our code to work with the new labels, our pattern test will need to be updated (https://github.com/JodusNodus/react-qr-reader/blob/master/src/getDeviceId.js#L43).

It is currently:

const filteredDevices = videoDevices.filter(({ label }) => pattern.test(label))

For labels like Back Camera and Front Camera to match, we should test on lowercase label, and thus it should be:

const filteredDevices = videoDevices.filter(({ label }) => pattern.test(label.toLowerCase()))

from react-qr-reader.

Hey i updated the ios11-fix branch with a lot of changes and improvements. Works on firefox android & macOs, chrome android & macOs, and Safari macOs. Can somebody please test IOS out?
You can test out it here https://thomasbilliet.com/react-qr-reader/

from react-qr-reader.

👋 @JodusNodus I can confirm that https://thomasbilliet.com/react-qr-reader/ works on my iPhone 6 w/iOS 11, will test now the branch directly, thanks!!

from react-qr-reader.

@JodusNodus The output I am seeing has two devices with unchanging IDs.

{
  "filteredDevices": [],
  "videoDevices": [
    {
      "deviceId": "F2EF6FA859E5BF554BEB52FF4CD11A57813DA14E",
      "kind": "videoinput",
      "label": "",
      "groupId": ""
    },
    {
      "deviceId": "621CC1DA97607D5E6DDC1BCCA918419DD4D725C5",
      "kind": "videoinput",
      "label": "",
      "groupId": ""
    }
  ]
}

from react-qr-reader.

@n8io Thanks alot! Seems that the problem is that webkit on IOS does not provide labels. Which i use for choosing the facingMode. I don't use the facingMode option in getUserMediabecause choosing the device manually gives me more consistent results accros browsers. Seems like i will have to make an exception for IOS Safari 🤦‍♂️. Safari on mac works fine.

from react-qr-reader.

Actually, perhaps a better solution, as @JodusNodus referenced, is to use the option in getUserMedia.

Perhaps we can feature detect to see if navigator.mediaDevices.getUserMedia is available, and if so, use its API to detect front and rear?

The presences of the newer navigator.mediaDevices.getUserMedia, as opposed to the older navigator.getUserMedia, suggests that the newer API is supported and the browser is sufficiently modern to give reliable results.

from react-qr-reader.

Sadly i don't have an iPhone to currently test it on. Probably something wrong with the listed devices available to use. If you want to help provide the following deviceIdChooser. Remote debug using chrome and post the output.

function deviceIdChooser(filteredDevices, videoDevices, facingMode) {
  console.log(videoDevices)
  return videoDevices[1].deviceId
}

EDIT: chooseDeviceId is the prop you can supply the function to

from react-qr-reader.

I just updated my iPhone 5s today to iOS 11.0.1 (from iOS 11.0) and both the front and rear camera selections correctly switch, display, and decode for me in Safari. Interestingly, I cannot get the demo site to work on Chrome for iOS as far as using the cameras - it defaults to legacy, but that could just be the demo site's configuration, etc. I love that webkit now supports WebRTC!

from react-qr-reader.

Yes finally support across platforms! @frinko Could you try out my instructions in my first response to this issue?

from react-qr-reader.

@JodusNodus Yes I will try that out and post the logs. When I saw this issue posted, I just tried while using the demo site because I knew that my phone had recently updated to the newer iOS version which was supposed to include the WebRTC support inside of webkit.

from react-qr-reader.

Same problem here. Any luck @frinko ?

from react-qr-reader.

Strangely both cameras work for me on iOS 11 on the demo site, but when I copy-paste the demo-site code to my own site, only the front camera works. I'm pretty confused as to why this might be the case.

from react-qr-reader.

Note that the device IDs change on each invocation of enumerateDevices. Presumably this is to prevent fingerprinting of users.

from react-qr-reader.

@athyuttamre Thanks a lot for doing the digging! This permission issue has been irritating me for a long time now. It also causes not being able to pick the right device on first load across browsers. Indeed i would need to preload a random device to request the permissions and then request the available devices. The filtering doesn't need to change as the regex pattern uses the case insensitive flag.

By specifying the device ID i am able to work around many bad browser facingMode implementations (or the lack of). It also allows the user to choose the device.

If this doesn't work i'll have to use the the actual facingMode API in the newer getUserMedia but the permission issue would still exist for older implementations.

from react-qr-reader.

@JodusNodus thanks for the quick follow-up! I'm still learning the API and this codebase, so sorry for not catching that the regex was actually correct.

Any thoughts on using this polyfill (https://github.com/webrtc/adapter) and simply sticking to the new API? Or, if you have a list of older browsers where the new API fails, perhaps we should special case them?

According to MDN, looks like the new API is supported pretty deep on most browsers: https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia#Browser_compatibility.

from react-qr-reader.

Ah, I see what you mean by the permission issue still existing in older browsers. I think it might be useful to ensure support for newer browsers, and then in time adding support for older ones? We can leave a note in the README saying that if you require older browser support in the meantime, please don't upgrade.

from react-qr-reader.

@athyuttamre No problem!
I already use that polyfill. It does a pretty good job but it ain't perfect. The problem even occurred in Chrome a couple versions ago. Browsers would throw OverconstrainedError if you specified a facingMode. Sadly i don't have a list of browser support.

I'm going to experiment with some stuff and see what would best solve this problem.

from react-qr-reader.

Ah, I see! And awesome, thank you for looking into this. Happy to help in any way I can, just let me know.

from react-qr-reader.

@athyuttamre Could you try out the new ios11-fix branch on IOS? It creates a temporary media stream to get permissions. It works on chrome android and desktop. Firefox doesn't follow deviceId or facingMode constraints so it is ignored.

from react-qr-reader.

For some reason while using ios11-fix branch Safari on OS X or iOS 11 don't ask for camera permissions at all.

from react-qr-reader.

@TheWildHorse Issue with the constraints. Fixed it. Works on Safari OSX.

from react-qr-reader.

@JodusNodus Hmm, this seems to break my app on all platforms, including OSX. :/

No permission prompt, no camera view, anything.

from react-qr-reader.

@TheWildHorse Any error logs? Because i updated the demo to use it and hosted it here https://vengeful-spark.surge.sh It works on all browsers on mac and android that i have tested.

from react-qr-reader.

@JodusNodus Testing it now, will let you know!

from react-qr-reader.

@JodusNodus It works, but still does not allow me to use the rear camera on iOS 11. This is how I call the QrReader (using the latest revision of the ios11 branch) and the front camera opens up.

<QrReader style={style} facingMode="rear" onError={this.handleError} onScan={this.handleScan} />

from react-qr-reader.

The actual filtering works fine, the back camera gets filtered and selected for the permission access request. Not sure where it goes downhill from there...

from react-qr-reader.

@TheWildHorse thanks for doing the testing. Haven't been able to work on it. I'll try to mock it here and see where it breaks.

from react-qr-reader.

@JodusNodus Found the problem, and made a hotfix for my purposes. Feel free to tidy it up and include it in the master: #40

Not sure if the "exact" syntax works everywhere so I just made it trigger on iOS devices.

from react-qr-reader.

@TheWildHorse Great. Seems like using exact also gets around the Firefox problem (ignoring deviceId en letting the user choose with a native dialog). I'll test it out accros browsers to see what the support is. If it doesn't cause regression i will just make it the default. Otherwise an exception for firefox and IOS like your PR

from react-qr-reader.

@JodusNodus any progress on this? Love the work so far btw.

from react-qr-reader.

@n8io Thanks, no i haven't had much time to work on this as i am a student and this is a side project. This weekend i should be able to spend some time on it.

from react-qr-reader.

@n8io Feel free to contribute your time! :)

from react-qr-reader.

not working on ios11.. stuck on rear camera

from react-qr-reader.

@zeusbangayan Weird, does it throw any errors? Could u try the demo bellow too.
https://react-qr-reader.surge.sh

from react-qr-reader.

im using iPhone 7 Plus. would that make a difference?

from react-qr-reader.

@zeusbangayan I don't see why it would make a difference. No clue what is the problem. Are there any errors thrown?

from react-qr-reader.

I can confirm that it works on ios. So i will publish version 2.0.0

from react-qr-reader.

When are you rolling out 2.0.0?

from react-qr-reader.

@rajivramv Already on NPM.

from react-qr-reader.