2011.04.08

Unsolved HTML5 video issues on iOS

On my last post I focused on issues that I’ve found a solution, this one is going to be about bugs and issues that I couldn’t find a solution and/or things that are impossible to do on the iPhone and iPad. It is important to note that I didn’t spent too much time searching or trying to find solutions for each problem but I consider most of them to be impossible to solve without a new OS release…

Screenshot of a video frame

With the “new” canvas API it is possible to capture video frames using the drawImage method[1] which is supposed to accept an HTMLImageElement or HTMLCanvasElement or HTMLVideoElement. The only problem is that on iOS this method doesn’t work with video elements. The same code that runs on other browsers (including Safari 5 desktop) doesn’t work on the iPad or iPhone.

The “solution”: export every single frame as an image and load it instead of capturing the video frame on real-time. – This is considered “cheating” not a “real” solution, but I had to do that in one project…

Can’t play sound and video together or multiple sounds or videos simultaneously

The media player on the iOS is a singleton, which means you can only have a single media element playing at each time[2], it doesn’t matter if it is audio or video. So if you try to play a sound file while a video is playing (e.g. sound effect for an user interaction) the video will be unloaded from the memory and you will see the Quicktime logo where the video was supposed to be - bummer.

No way to control volume using JavaScript

The HTML5 spec says that every media element have a property called volume which is used to get and set the volume of the audio or video playback, this value should be a number from 0.0 to 1.0 (inclusive), but again Apple decided to not follow the standard on the iOS and only allow the volume control using the physical control [3]. Changing the property muted also doesn’t have any effect on the iOS.

No way to play video inline on the iPhone

Videos always plays full-screen on the iPhone (as of iOS 4.2), I’m “sure” that it isn’t a technical issue since iAds can play inline videos without any problems and the iAd is simply an embedded UIWebView. – it seems that Apple doesn’t want webapps to succeed since they don’t pay 30% of the sales to them… – And don’t say that it is because the user won’t have enough available screen area to watch the video, that it is for a “better experience”, that it “saves battery”.. it is all bullsh*t, the browser UI don’t need to be so obstructive and in some specific cases it is totally fine to have the video playing on a small area, they are “doing it on purpose” just to push users to the app-based browsing (which they control and profit with).

No way to play or preload sound and video without user interaction

Media files are only downloaded and played if triggered by a click event, this can be very frustrating and limiting if your webApp requires those things to work properly. I think the right approach would be to prompt the user asking for permissions to download and play media files (the same way they do for getting the user location) and if the user allow it then you could load/play as many files as you want without asking again for permissions or requiring user interaction. If the user denied than it would require a click event to trigger the media elements. - No more lame excuses about bandwidth…

Multiple HTML5 video elements on the same page causes bugs

I tested this a couple months after they released the iPad and didn’t tested it after that, but on my simple test the iPad had issues with multiple video elements on the same page even if they aren’t playing simultaneously, that is another reason for using plain image tags as a poster image and only insert the HTML5 video element when you need it. – I’m not sure if this bug was fixed on newest versions though but I’ve seen some other people complaining about the same issue.

No way to play HTML5 video backwards added 2011/08/23

It doesn’t seem possible to play a video backwards smoothly. Trying to update the video.currentTime on a timeout doesn’t update the frames fast enough and setting video.playbackRate[4] to -1 have no effect.

Conclusion

Apple only “embraces” the standards when it is good for them, don’t believe in pure PR… the HTML5 talk is only to cause buzz and make people believe they are the “good guys”. If they really “embraced” standards you wouldn’t see implementations that doesn’t follow the specs and the iOS Safari wouldn’t have that many issues. They are clearly investing more money where they make more money (native apps and PR) and holding the “full” potential of the web as much as they can…

I’m going to end the post since it is getting lengthy and I can’t remember other things that made me upset but I’m sure I forgot something and/or I’m going to find other stuffs on the next projects… and yes, I love HTML/CSS/JS, I really enjoy challenges (it is one of the only reasons I decided to work with development), I know what I’m talking about and I’m not giving up (as some people suggested on the comments of my rant), but two things that I don’t like: lies and fanaticism – don’t be a moron.

References

  1. Context2d.drawImage()
  2. Multiple simultaneous audio or video streams – iOS
  3. Volume control in JavaScript
  4. media.playbackRate

Related

Tags: , , , , , , ,
Comments (37)

Comments

Giles
2011-04-20 21:08:15

Great list.

The volume issue makes sense from Apple's perspective: the idea of more than one volume control (software and hardware) being applicable at any time is likely to cause confusion with highly non-savvy users who try to turn up the volume and don't understand why it's not working. Like the preloading issue, it's a quick fix they've made to address a potential customer satisfaction issue which I would guess they'll re-evaluate in the future.

The media singleton issue is utterly killer though. It torpedoes music in games. I tend to be on Apple's side, preferring their approach of "a great user experience, or no user experience at all" and giving them the benefit of the doubt, but that's a huge crippling of web app capabilities and really, i can't understand any UX rationale for it.

Bruce Lawson’s personal site  : Reading list
2011-05-09 03:22:35

[...] video issues on the iPad and how to solve them, Unsolved HTML5 video issues on iOS (Miller H. Borges [...]

Hugo Ribeiro
2011-06-01 02:52:17

Anyone can bind onclick or touch event in player? Or this is another issue?

Miller Medeiros
2011-06-01 07:39:46

Touch events doesnt work on videos on iOS, the way to make it work is to add an overlay on top of it and attach the listeners on the overlay.

Im going to update the other post since I forgot that this bug existed and that I solved it.

HTML5 video on iPad gives black screen - Applerr.com All about Apple Products - Applerr.com All about Apple Products
2011-06-28 05:02:05

[...] videos being ancient might have something to do this, but perhaps that’s just superstition. http://blog.millermedeiros.com/2011/04/unsolved-html5-video-issues-on-ios/ says that iPad’s only play one video in the page at a time, and I would like to have [...]

cazanave
2011-07-18 08:54:45

Hi, Thanks for the list. You say that the only "solution" to take a screenshot of a video frame is to export every single frame. What do you mean exactely? How can we proceed? Thanks in advance.

Miller Medeiros
2011-07-18 09:22:32

You would open the video on any video editing software like Adobe After Effects, Adobe Premiere or FFmpeg (command-line) and export all the frames as individual images and store it on a folder.. later you just reference the frame by number (e.g. "img/video_stills/0123.jpg" would load frame 123 of the video..) - so you are not really taking a screenshot, you are just loading an image of that frame.

Chritoph
2011-07-19 04:19:39

If you use a custom UIWebView, you can actually make the video run inline and automatically: When constructing the web view, do:

myWebView.allowsInlineMediaPlayback = YES; myWebView.mediaPlaybackRequiresUserAction = NO;

and in html-file:

<video id="myVideo" poster="trailer.jpg" webkit-playsinline autoplay>

Pete
2011-09-01 09:29:00

MULTIPLE HTML5 VIDEO ELEMENTS ON THE SAME PAGE CAUSES BUGS <- Thank god I found this, was about to go nuts!

I've tested this and it seems that mulitple html5 video elements on same page works in iOS5, but will not work properly in earlier versions. I'm really looking forward to iOS5 release :)

Nelson Monteiro
2011-09-13 03:51:12

Hi,

I've tried to put a div over the video with a link to somewhere else, but when i try to click on the div it did not recognize or catch the event. if the div is not over the video all works fine. This problems occurs in my iPad, on desktop browser its ok. have you experience problems like this before?

here is the code:

CSS .video { width:100%; height:100%; position:absolute; top:0; left:0; z-index:990; }

showcontent {

  position:absolute;
  width:100%;
  height:100px;
  bottom:0;
  left:0;
  z-index:999;
  background-color: rgba(90%, 90%, 90%, 0.8);

}

HTML ...

...

Nelson Monteiro
2011-09-13 03:54:34

Hi I realize that the html tags didn't work. trying again: without html tags

section id="video" video src="http://cf.cdn.vid.ly/4y8e8y/uspB.mp4" id="vid1" poster="http://cf.cdn.vid.ly/4y8e8y/poster.jpg" autobuffer controls class="video" /video /section div id="showcontent" a href="index.html" img src="18.jpg" width="78" height="42" /a /div

Miller Medeiros
2011-09-15 23:21:00

@nelson try to add some CSS transform to the div, it will make it go over the video. You can only click on elements that are over a video if it doesn't use the native controls.

TomSiwik
2011-10-03 15:02:37

Thank you for pointing out the issues. Have searched for the Video-Reset-Bug a long time now. It seems that two videos in one window was causing them and could be reproduced many times. It's strange, that all webkit-browsers i have tested (mobile, desktop) have many of the same issues. So not every bug is an Apple-bug.

Holli
2011-10-06 02:03:32

I just got ios5 on my ipad2 and i still can Not Play more than 1 Video at the Same Time (Youtube iframe API) on a Single Page ....

Jeisman
2011-11-09 16:26:21

Of course Apple only embraces open standards when it benefits them, they're a corporation.

They never do anything for PR. They don't give a shit what anyone thinks in the first place.

Why the hell would you ever think otherwise

Jeroen Wijering
2011-12-06 08:17:37

Good, extensive list of iOS limitations. I don't think the conclusion is fair though. Apple did not impose limitations because it wanted to implement only half a spec. The problem is that the HTML5 spec was never intended for mobile devices. And mobile devices are vastly different from desktops. They have limited processing power, touch inputs, small screens and hardware volume controls.

Other mobile platforms (Android and WinPho) instituted exactly the same limitations. I think they all make perfect sense:

*) No playback of multiple videos *) No inline playback on phones *) No software volume *) No preloading / autostart

There are still the occasional bugs in iOS video playback. But these bugs are also present on other HTML5 browsers. Submitting them to the respective vendor is the best way to get them out.

Unsolved HTML5 video issues on iOS | Blog | Miller Medeiros | Programmer Solution
2012-02-01 16:34:42

[...] the “new” canvas API it is possible to capture video frames using the drawImage method[1] which is supposed to accept an HTMLImageElement or HTMLCanvasElement or HTMLVideoElement. The only [...]

Unsolved HTML5 video issues on iOS | Blog | Miller Medeiros | Programmer Solution
2012-02-01 16:34:42

[...] the iOS is a singleton, which means you can only have a single media element playing at each time[2], it doesn’t matter if it is audio or video. So if you try to play a sound file while a video is [...]

Limitations of HTML5 Video and Audio in iOS | Kevin Leetham's Techno Babble
2012-02-08 22:09:55

[...] plug-ins, in iOS (mobile Safari), it still has some problems.  Miller H. Borges Medeiros has a good blog post covering the issues.  Here are topics he discusses in more [...]

Da Scritch
2012-05-08 02:47:20

This is not iOS specific.

All the described bugs come from the mobile driver arch : an hardware video H264 rendering.

HTML5 Video « Appcube Blog
2012-06-27 09:23:03

[...] Die Unterstützung des HTML5 Video Elements auf mobilen Endgeräten ist zur Zeit (Juni 2012) teils noch sehr eingeschränkt. Das iPhone und Windows Phone 7 erlaubt z.B. nur ein Abspielen des Videos im Fullscreen Modus. Ein paar interessante Fakten gibt es hier. [...]

HTML5 Video « Appcube Blog
2012-06-27 10:25:39

[...] Die Unterstützung des HTML5 Video Elements auf mobilen Endgeräten ist zur Zeit (Juni 2012) teils noch sehr eingeschränkt. Das iPhone und Windows Phone 7 erlaubt z.B. nur ein Abspielen des Videos im Fullscreen Modus. Möchte man das Video inline abspielen, muss man zur Zeit leider noch eine richtige App veröffentlichen und in dieser allowsInlineMediaPlayback=true setzen. Ein paar interessante Infos dazu gibt es hier. [...]

Anatoly
2012-08-24 17:57:48

Found that you are not able to make any DOM manipulations with video node, like moving it into another element on page, using as example element.appendChild or element.replaceChild. Video tag itself moved and looks correct, but none of its methods are working.

Ernest Leitch
2012-10-16 18:04:11

This was a fantastic post. It helped me solve a lot of weirdness I ran into while trying to get audio to work in iOS 5 and 6.

Mike
2013-02-19 13:02:05

It would be really nice to see the same kind of info for the native Android browser and both flavored of mobile chrome browser. I would assume iOS chrome would be very close to mobile safari.

Shay Pierce
2013-06-10 13:00:19

It's two years later, how much of this is still applicable? (From what I've heard: all of it.)

A follow-up post would be great!

pg
2013-08-15 15:26:40

Everything is still applicable! currently developing a interactive video experience and had to rework a lot of things for ipad only. Web and android worked as designed but many special cases had to used for ipad. It is so true ipad is the new ie6

@Da Scritch Android does not have any of these limitations. So i have no idea what your talking about.

Faking Interactive Video on Mobile | Shine Technologies
2013-12-12 04:10:40

[…] That side experiment wasn’t without its insights; it proved that there’s a lot that can be done to enhance the interactive experience on mobile if you can control the video player’s HTML. At one point YouTube had partial support for annotations but presumably this was dropped when they were unable to provide a consistent experience across all mobile devices due to iOS constraints. […]

Lasse Rafn
2014-04-29 17:55:59

I'd be pissed if I had to load a HTML5 video before the website was accessible, if I was on a Edge connection.. There's also another side to it

jay bharat
2014-07-18 11:23:32

Hi sir. I am using in this case I am having one issue, Fast Forward and slow backward is not working. Can you please help.

jay bharat
2014-07-18 11:25:15

Hi sir. I am using < vide > < /video > in this case I am having one issue, Fast Forward and slow backward is not working. Can you please help.

Appcube » HTML5 Video
2014-11-28 06:39:33

[…] Die Unterstützung des HTML5 Video Elements auf mobilen Endgeräten ist zur Zeit (Juni 2012) teils noch etwas eingeschränkt. Das iPhone und Windows Phone 7 erlaubt nur das Abspielen eines Videos im Fullscreen Modus. Möchte man das Video inline abspielen, muss man zur Zeit noch eine richtige App veröffentlichen und beim iPhone das allowsInlineMediaPlayback=true setzen. Ein paar interessante Infos dazu gibt es hier. […]