2011.04.08

Unsolved HTML5 video issues on iOS

On my last post I focused on issues that I’ve found a solution, this one is going to be about bugs and issues that I couldn’t find a solution and/or things that are impossible to do on the iPhone and iPad. It is important to note that I didn’t spent too much time searching or trying to find solutions for each problem but I consider most of them to be impossible to solve without a new OS release…

Screenshot of a video frame

With the “new” canvas API it is possible to capture video frames using the drawImage method[1] which is supposed to accept an HTMLImageElement or HTMLCanvasElement or HTMLVideoElement. The only problem is that on iOS this method doesn’t work with video elements. The same code that runs on other browsers (including Safari 5 desktop) doesn’t work on the iPad or iPhone.

The “solution”: export every single frame as an image and load it instead of capturing the video frame on real-time. – This is considered “cheating” not a “real” solution, but I had to do that in one project…

Can’t play sound and video together or multiple sounds or videos simultaneously

The media player on the iOS is a singleton, which means you can only have a single media element playing at each time[2], it doesn’t matter if it is audio or video. So if you try to play a sound file while a video is playing (e.g. sound effect for an user interaction) the video will be unloaded from the memory and you will see the Quicktime logo where the video was supposed to be - bummer.

No way to control volume using JavaScript

The HTML5 spec says that every media element have a property called volume which is used to get and set the volume of the audio or video playback, this value should be a number from 0.0 to 1.0 (inclusive), but again Apple decided to not follow the standard on the iOS and only allow the volume control using the physical control [3]. Changing the property muted also doesn’t have any effect on the iOS.

No way to play video inline on the iPhone

Videos always plays full-screen on the iPhone (as of iOS 4.2), I’m “sure” that it isn’t a technical issue since iAds can play inline videos without any problems and the iAd is simply an embedded UIWebView. – it seems that Apple doesn’t want webapps to succeed since they don’t pay 30% of the sales to them… – And don’t say that it is because the user won’t have enough available screen area to watch the video, that it is for a “better experience”, that it “saves battery”.. it is all bullsh*t, the browser UI don’t need to be so obstructive and in some specific cases it is totally fine to have the video playing on a small area, they are “doing it on purpose” just to push users to the app-based browsing (which they control and profit with).

No way to play or preload sound and video without user interaction

Media files are only downloaded and played if triggered by a click event, this can be very frustrating and limiting if your webApp requires those things to work properly. I think the right approach would be to prompt the user asking for permissions to download and play media files (the same way they do for getting the user location) and if the user allow it then you could load/play as many files as you want without asking again for permissions or requiring user interaction. If the user denied than it would require a click event to trigger the media elements. - No more lame excuses about bandwidth…

Multiple HTML5 video elements on the same page causes bugs

I tested this a couple months after they released the iPad and didn’t tested it after that, but on my simple test the iPad had issues with multiple video elements on the same page even if they aren’t playing simultaneously, that is another reason for using plain image tags as a poster image and only insert the HTML5 video element when you need it. – I’m not sure if this bug was fixed on newest versions though but I’ve seen some other people complaining about the same issue.

No way to play HTML5 video backwards added 2011/08/23

It doesn’t seem possible to play a video backwards smoothly. Trying to update the video.currentTime on a timeout doesn’t update the frames fast enough and setting video.playbackRate[4] to -1 have no effect.

Conclusion

Apple only “embraces” the standards when it is good for them, don’t believe in pure PR… the HTML5 talk is only to cause buzz and make people believe they are the “good guys”. If they really “embraced” standards you wouldn’t see implementations that doesn’t follow the specs and the iOS Safari wouldn’t have that many issues. They are clearly investing more money where they make more money (native apps and PR) and holding the “full” potential of the web as much as they can…

I’m going to end the post since it is getting lengthy and I can’t remember other things that made me upset but I’m sure I forgot something and/or I’m going to find other stuffs on the next projects… and yes, I love HTML/CSS/JS, I really enjoy challenges (it is one of the only reasons I decided to work with development), I know what I’m talking about and I’m not giving up (as some people suggested on the comments of my rant), but two things that I don’t like: lies and fanaticism – don’t be a moron.

References

  1. Context2d.drawImage()
  2. Multiple simultaneous audio or video streams – iOS
  3. Volume control in JavaScript
  4. media.playbackRate

Related


Comments

Great list.

The volume issue makes sense from Apple's perspective: the idea of more than one volume control (software and hardware) being applicable at any time is likely to cause confusion with highly non-savvy users who try to turn up the volume and don't understand why it's not working. Like the preloading issue, it's a quick fix they've made to address a potential customer satisfaction issue which I would guess they'll re-evaluate in the future.

The media singleton issue is utterly killer though. It torpedoes music in games. I tend to be on Apple's side, preferring their approach of "a great user experience, or no user experience at all" and giving them the benefit of the doubt, but that's a huge crippling of web app capabilities and really, i can't understand any UX rationale for it.

[...] video issues on the iPad and how to solve them, Unsolved HTML5 video issues on iOS (Miller H. Borges [...]

Anyone can bind onclick or touch event in player? Or this is another issue?

Touch events doesnt work on videos on iOS, the way to make it work is to add an overlay on top of it and attach the listeners on the overlay.

Im going to update the other post since I forgot that this bug existed and that I solved it.

[...] videos being ancient might have something to do this, but perhaps that’s just superstition. http://blog.millermedeiros.com/2011/04/unsolved-html5-video-issues-on-ios/ says that iPad’s only play one video in the page at a time, and I would like to have [...]

Hi, Thanks for the list. You say that the only "solution" to take a screenshot of a video frame is to export every single frame. What do you mean exactely? How can we proceed? Thanks in advance.

You would open the video on any video editing software like Adobe After Effects, Adobe Premiere or FFmpeg (command-line) and export all the frames as individual images and store it on a folder.. later you just reference the frame by number (e.g. "img/video_stills/0123.jpg" would load frame 123 of the video..) - so you are not really taking a screenshot, you are just loading an image of that frame.

If you use a custom UIWebView, you can actually make the video run inline and automatically: When constructing the web view, do:

myWebView.allowsInlineMediaPlayback = YES; myWebView.mediaPlaybackRequiresUserAction = NO;

and in html-file:

<video id="myVideo" poster="trailer.jpg" webkit-playsinline autoplay>

MULTIPLE HTML5 VIDEO ELEMENTS ON THE SAME PAGE CAUSES BUGS <- Thank god I found this, was about to go nuts!

I've tested this and it seems that mulitple html5 video elements on same page works in iOS5, but will not work properly in earlier versions. I'm really looking forward to iOS5 release :)

Nelson Monteiro

Hi,

I've tried to put a div over the video with a link to somewhere else, but when i try to click on the div it did not recognize or catch the event. if the div is not over the video all works fine. This problems occurs in my iPad, on desktop browser its ok. have you experience problems like this before?

here is the code:

CSS .video { width:100%; height:100%; position:absolute; top:0; left:0; z-index:990; }

showcontent {

  position:absolute;
  width:100%;
  height:100px;
  bottom:0;
  left:0;
  z-index:999;
  background-color: rgba(90%, 90%, 90%, 0.8);

}

HTML ...

...

Nelson Monteiro

Hi I realize that the html tags didn't work. trying again: without html tags

section id="video" video src="http://cf.cdn.vid.ly/4y8e8y/uspB.mp4" id="vid1" poster="http://cf.cdn.vid.ly/4y8e8y/poster.jpg" autobuffer controls class="video" /video /section div id="showcontent" a href="index.html" img src="18.jpg" width="78" height="42" /a /div

@nelson try to add some CSS transform to the div, it will make it go over the video. You can only click on elements that are over a video if it doesn't use the native controls.

Thank you for pointing out the issues. Have searched for the Video-Reset-Bug a long time now. It seems that two videos in one window was causing them and could be reproduced many times. It's strange, that all webkit-browsers i have tested (mobile, desktop) have many of the same issues. So not every bug is an Apple-bug.

I just got ios5 on my ipad2 and i still can Not Play more than 1 Video at the Same Time (Youtube iframe API) on a Single Page ....

Of course Apple only embraces open standards when it benefits them, they're a corporation.

They never do anything for PR. They don't give a shit what anyone thinks in the first place.

Why the hell would you ever think otherwise

Good, extensive list of iOS limitations. I don't think the conclusion is fair though. Apple did not impose limitations because it wanted to implement only half a spec. The problem is that the HTML5 spec was never intended for mobile devices. And mobile devices are vastly different from desktops. They have limited processing power, touch inputs, small screens and hardware volume controls.

Other mobile platforms (Android and WinPho) instituted exactly the same limitations. I think they all make perfect sense:

*) No playback of multiple videos *) No inline playback on phones *) No software volume *) No preloading / autostart

There are still the occasional bugs in iOS video playback. But these bugs are also present on other HTML5 browsers. Submitting them to the respective vendor is the best way to get them out.

[...] the iOS is a singleton, which means you can only have a single media element playing at each time[2], it doesn’t matter if it is audio or video. So if you try to play a sound file while a video is [...]

[...] the “new” canvas API it is possible to capture video frames using the drawImage method[1] which is supposed to accept an HTMLImageElement or HTMLCanvasElement or HTMLVideoElement. The only [...]

[...] plug-ins, in iOS (mobile Safari), it still has some problems.  Miller H. Borges Medeiros has a good blog post covering the issues.  Here are topics he discusses in more [...]

This is not iOS specific.

All the described bugs come from the mobile driver arch : an hardware video H264 rendering.

[...] Die Unterstützung des HTML5 Video Elements auf mobilen Endgeräten ist zur Zeit (Juni 2012) teils noch sehr eingeschränkt. Das iPhone und Windows Phone 7 erlaubt z.B. nur ein Abspielen des Videos im Fullscreen Modus. Ein paar interessante Fakten gibt es hier. [...]

[...] Die Unterstützung des HTML5 Video Elements auf mobilen Endgeräten ist zur Zeit (Juni 2012) teils noch sehr eingeschränkt. Das iPhone und Windows Phone 7 erlaubt z.B. nur ein Abspielen des Videos im Fullscreen Modus. Möchte man das Video inline abspielen, muss man zur Zeit leider noch eine richtige App veröffentlichen und in dieser allowsInlineMediaPlayback=true setzen. Ein paar interessante Infos dazu gibt es hier. [...]

Found that you are not able to make any DOM manipulations with video node, like moving it into another element on page, using as example element.appendChild or element.replaceChild. Video tag itself moved and looks correct, but none of its methods are working.

This was a fantastic post. It helped me solve a lot of weirdness I ran into while trying to get audio to work in iOS 5 and 6.

It would be really nice to see the same kind of info for the native Android browser and both flavored of mobile chrome browser. I would assume iOS chrome would be very close to mobile safari.

It's two years later, how much of this is still applicable? (From what I've heard: all of it.)

A follow-up post would be great!

Everything is still applicable! currently developing a interactive video experience and had to rework a lot of things for ipad only. Web and android worked as designed but many special cases had to used for ipad. It is so true ipad is the new ie6

@Da Scritch Android does not have any of these limitations. So i have no idea what your talking about.

[…] That side experiment wasn’t without its insights; it proved that there’s a lot that can be done to enhance the interactive experience on mobile if you can control the video player’s HTML. At one point YouTube had partial support for annotations but presumably this was dropped when they were unable to provide a consistent experience across all mobile devices due to iOS constraints. […]

I'd be pissed if I had to load a HTML5 video before the website was accessible, if I was on a Edge connection.. There's also another side to it

Hi sir. I am using in this case I am having one issue, Fast Forward and slow backward is not working. Can you please help.

Hi sir. I am using < vide > < /video > in this case I am having one issue, Fast Forward and slow backward is not working. Can you please help.

Leave a Comment

Please post only comments that will add value to the content of this page. Please read the about page to understand the objective of this blog and the way I think about it. Thanks.

Comments are parsed as Markdown and you can use basic HTML tags (a, blockquote, cite, code, del, em, strong) but some code may be striped if not escaped (specially PHP and HTML tags that aren't on the list). Line and paragraph breaks automatic. Code blocks should be indented with 4 spaces.