Playing web audio offline on mobile Safari. Mission impossible?

Mark Boas — Tue, 29 May 2012 20:13:54 GMT

Several people had asked me if it was possible to cache audio off-line in a web app using mobile Safari on iOS. I hadn't yet seen anything to indicate that this was possible and my gut feeling was that it wasn't. However I decided to find out for myself.

Storified by Mark Boas · Tue, May 29 2012 13:13:54

Twitter is a great medium. I often feel that as a remote worker it has changed the way I work. Today thanks to some helpful fellow Tweeters I was able to determine whether it was possible to cache audio in mobile Safari in double quick time. Here's the write up.

Wondering if it's even possible to store an mp3 in localstorage in mobile safari.
— Mark Boas · Tue, May 29 2012 01:08:24

Bruce Lawson. Web Evangelist, Opera.

@maboa why wouldn't it be? Space limits?
— bruce lawson · Tue, May 29 2012 01:11:05

Robert Nyman. Technical Evangelist, Mozilla.

@maboa Turn it onto a string, and then store?
— Robert Nyman · Tue, May 29 2012 01:11:58

@brucel @robertnyman Well I'm pretty new to local storage but it appears there is a 5MB limit unless you use the local DB? Then 50MB?
— Mark Boas · Tue, May 29 2012 01:16:23

@robertnyman @brucel so to take advantage of 50MB I need to put it in a Web SQL format - I'm guessing a data URI would do.
— Mark Boas · Tue, May 29 2012 01:18:21

@maboa @brucel I think the key things are, like you say, size and then make into a Data URL. More info at http://html5doctor.com/introducing-web-sql-databases/ :-)
— Robert Nyman · Tue, May 29 2012 01:20:29

@robertnyman @brucel Shame the file api isn't pronto http://caniuse.com/#feat=fileapi
— Mark Boas · Tue, May 29 2012 01:22:40

So the first port of call - the file api is out because it's not supported by mobile Safari.

@maboa @brucel I wrote http://hacks.mozilla.org/2012/02/saving-images-and-files-in-localstorage/ and http://hacks.mozilla.org/2012/02/storing-images-and-files-in-indexeddb/, that might be interesting
— Robert Nyman · Tue, May 29 2012 01:24:47

Matteo Spinelli. Freelance Developer.

@maboa couldn't find a way to cache audio on iOS. did't try with BlobBuilder/FileReader, though http://www.html5rocks.com/en/tutorials/file/filesystem/
— Matteo Spinelli · Tue, May 29 2012 01:34:43

@cubiq Thanks! My gut feeling is that it isn't possible. Won't stop me trying though :)
— Mark Boas · Tue, May 29 2012 01:41:02

@maboa nope, just tried with blobbuilder+filereader, it works on chrome, still no go on iOS
— Matteo Spinelli · Tue, May 29 2012 02:00:30

So the next thing to look at is storing them in the browser's database. WebSQL in mobile Safari's case.

Andrea Giammarchi. Software Engineer. Facebook.

@maboa encoding/decoding takes a while plus more space as base64 ... but it kinda works
— Andrea Giammarchi · Tue, May 29 2012 02:09:51

@WebReflection Thanks. May be an acceptable price to pay. This is where I'm up to http://jsfiddle.net/Yz9vD/ /cc @cubiq @brucel @robertnyman
— Mark Boas · Tue, May 29 2012 02:18:48

Flash vs HTML5 Video and the Codec thing

Mark Boas — Mon, 27 Feb 2012 19:52:48 GMT

The ongoing debate about whether it is better to use Flash or HTML5 to deliver web based video and the related kerfuffle regarding H.264 and WebM.

Storified by Mark Boas · Mon, Feb 27 2012 11:52:48

By Mark Boas

As one half of the team that produces jPlayer - a JavaScript media library that uses both HTML5 and Flash, this affects me directly. There also seems to be a lot of interest in this subject at an organisation I've just started working with. I'm not sure if this heightened my senses to the subject, but over the last day or so I've noticed some interesting discussion on Twitter. So I thought it was worthwhile documenting for the next time it crops up.

The first discussion takes place between Mike Shaver (Facebook and ex-Mozilla) and Brendan Eich (CTO of Mozilla and inventor of JavaScript) - I include a portion of it below.

Did Chrome ever actually drop H.264? I haven't seen anything since the blog post 13 months (!) ago.
— Mike Shaver · Fri, Feb 24 2012 05:14:22

H.264 is a video compression format, which although fairly recently open-sourced - H.264 is 'patent encumbered' in other words royalties for its use could be claimed at any time. Currently latest versions of Chrome, Internet Explorer and Safari all support H.264 but Firefox and Opera do not but support an alternative format known as WebM (as does Chrome). However around 13 months ago Google stated their intention to drop H.264 support from Chrome 'in the coming months'.

@shaver Not yet. If/when, NBD because content authors put Flash fallback inside , and Chrome has a "special" Flash deal from Adobe...
— BrendanEich · Fri, Feb 24 2012 05:16:38

A Flash fallback is used when we detect whether HTML5 or a particular format is supported and fall back to a Flash solution if not. At this point it's worth mentioning that Mobile Safari does not support Flash. NBD in this case means no big deal (for Chrome). The special Flash deal probably alludes to Chrome and Adobe's deal to bundle Flash with Chrome and Adobe's intention to drop support for Flash on Linux for browsers other than Chrome. Note that Adobe plan to not support 'new mobile device configurations' going forward.

@BrendanEich yeah, and it looks like B2G will support H.264 as well, so to the extent that mobile is the future, open codecs probably aren't
— Mike Shaver · Fri, Feb 24 2012 06:14:22

B2G refers to Boot-2-Gecko a new initiative by Mozilla to create a web-based open source mobile operating system.

@shaver We have to play in order to place or win, but we're playing the longer game (Broadway.js is a first move) too.
— BrendanEich · Fri, Feb 24 2012 06:19:57

Note. Whereas there is more room to maneuver with desktop browsers, with mobile devices, speed and battery life are of prime concern. Since currently most, if not all mobile devices that play video, support H.264 decoding directly via hardware, it would be difficult to enter the market with a mobile OS that didn't support H.264.

Broadway.js is an open-source library that decodes H.264 in JavaScript by taking advantage of the newer faster JavaScript engines that are built into the latest browsers. This means that in theory a developer can add H.264 playback capability to any modern browser. See also JSMad for MP3 playback and ALAC.js for lossless playback.

I encourage you to expand their tweets to see who else has chimed in. You can see the full discussion here.

WebM vs H.264 Showdown

The second discussion involves David Storey (former Opera Web Evangelist now working for Motorola which has recently been acquired by Google) and Faruk Ateş ('Entreprenerd' and former web-standards specialist for Apple). Excerpts follow.

MS “Motorola is on a path to use standard essential patents to kill video on the Web” Even if true it would be H.264 not video. WebM anyone?
— David Storey · Thu, Feb 23 2012 16:01:31

…oh I forgot, MS and Apple don’t support the open format. My bad. Supporting WebM would solve their issues, but do they want to do that?
— David Storey · Thu, Feb 23 2012 16:03:00

@dstorey WebM is an idealist’s pipe dream, completely devoid of any shred of pragmatism.
— Faruk Ateş · Thu, Feb 23 2012 16:03:52

@KuraFire I'm not sure how H.264 is pragmatic for the web. Open source can't use it. Browsers like Opera can’t afford to licence it.
— David Storey · Thu, Feb 23 2012 16:14:00

@dstorey Not saying it’s pragmatic for the web per sé, but at least it works well on the 350+ million hardware devices people have bought.
— Faruk Ateş · Thu, Feb 23 2012 16:15:06

@dstorey Unlike WebM, which is a dreadful experience having to rely on far-slower software decoders, killing battery life.
— Faruk Ateş · Thu, Feb 23 2012 16:15:46

@KuraFire that is/was a temporary issue. Hardware decoders are available for WebM http://en.wikipedia.org/wiki/WebM#Hardware
— David Storey · Thu, Feb 23 2012 16:20:34

@dstorey Yeah, "temporary". Who’s going to insert hardware WebM decoders into those 350+ million devices? Dude, seriously not an argument.
— Faruk Ateş · Thu, Feb 23 2012 16:22:24

@dstorey That the HW decoders are now available for it means WebM is useful once _every popular consumer electronics device_ ships with one.
— Faruk Ateş · Thu, Feb 23 2012 16:23:09

@KuraFire how quick will you upgrade to the iPhone 5? Phone upgrade life cycles are short. Who says existing decoders can't be reprogrammed?
— David Storey · Thu, Feb 23 2012 16:28:40

Mark Boas · Storify

Playing web audio offline on mobile Safari. Mission impossible?

Flash vs HTML5 Video and the Codec thing

WebM vs H.264 Showdown