deliciously simple web design & creatively technical writing

• Home
• About
• Gallery
• Archive

Resolution Independence and Core Animation equals Multi-Touch in Leopard

I have a love/hate relationship with cliffhangers. You’re watching a great TV show, your heart is pumping and you’re wondering what is going to happen next. All of a sudden, the words “to be continued…” flash on the screen. ARGGHGHHHH!1!

Thats exactly how my heart sank when Steve Jobs previewed Mac OS X 10.5 Leopard at the 2006 Worldwide Development Conference and announced that top secret stuff existed that wouldn’t be revealed.

Before going down the list of 10 important Leopard features, he announced that there were going to be some top secret features in Leopard that wouldn’t be revealed right at the moment because of his concern that they might be copied. I’m a nerd. I live for this stuff and I have to admit I got a little ticked off at this. But then I realized that just like a kid anxiously waiting for Christmas morning to roll around, good things come to those who wait.

As this was a development conference, most of the publicly announced features have at least something to do with development. For example, developers will have to do some work to make sure that temporary scratch files aren’t unnecessarily copied by Time Machine and in the process wasting resources.

I hate the petty back and forth bickering between Apple and Microsoft about who is copying who. When it comes to features, what really matters isn’t the idea, what matters is implementation: and this is where Apple really shines and Microsoft is behind the curve.

The end-user Leopard features that have been revealed thus far are impressive and very user-friendly even for computer newbies. My three favorites are…..

Spaces: For anybody who has used various distributions of Linux in the past, virtual desktops aren’t a new thing. What matters is the implementation, and Apple has created the most usable implementation of virtual desktops I’ve ever seen: fully drag and drop with smooth animations throughout.

Time Machine: Backup technology isn’t anything new either, but Time Machine looks like something that anybody could actually use and understand. And the fact that you can use it to backup just a single file that you need is really cool.

iChat: The iChat video effects are going to be fun to mess around with when drinking, but the screen-sharing and collaboration features look like real winners. With more and more people having high-speed internet connections and web-cams, this is the type of thing that becomes really useful.

Trying to figure out what these top secret Leopard features are has always been infinitely more fascinating to me than the inane speculation regarding a release date. Apple is a quality company that rarely releases shoddy products. Leopard was never going to be released until it was ready. The question is, what are they really trying to get ready?

I believe there are 2 main categories of top secret features: one is related to the interface, and the other is related to how we work with data.

I believe that Apple is getting ready to change the computer world yet again by introducing a multi-touch capable operating system. There are a number of public pieces of knowledge that point to how Leopard’s already released features open the door for multi-touch: specifically core animation and resolution independence.

Sort of like what Veronica Mars would do, lets sassily sift through the evidence.

Apple’s Leopard Technology Overview confirms that Resolution Independance will be available in Leopard and Apple’s 2007 Worldwide Development Conference Sessions page announces that a Resolution Independence Lab for developers will take place.

Apple filed for a Resolution Independent User Interface Design patent in July 2006.

From the Abstract of patent 20060284878

In plain English, this seems to be a tool for Apple to create the next generation Apple interface, possibly as vector buttons and other interface elements. Hrmpf has a great writeup on this and other Apple patents.

One obvious benefit of this move is allowing higher resolution displays in the future that would make reading text and viewing photos even more pleasant on all screen sizes. But there is another possible benefit here.

How would resolution independence and a vectorized interface affect multi-touch? The end result of this can be an interface that can zoom the display in and out extremely quickly, and with much more polish than currently exists. Right now in 10.4.9 if you enable it in system preferences, you can hold down control and scroll up and down to increase or decrease the size of your display contents. Unfortunately, while this is a really useful feature in limited use, the zoomed in interface is really pixelated. There’s no real Apple polish here.

So why is zooming in and out important? Simply this: fingers are too large to consistently be able to use multi-touch (without zooming) with current interfaces that are designed for use with the precision that a mouse gives you. Selecting a single link in many current sites would be difficult at best.

In order to continue to use already existing software and websites with a multi-touch system, you’d need to (at times) zoom in and out quickly to pick out one check box, or to hit one control point. If you look at Jeff Han’s videos, particularly his photolightbox application, you’ll notice that he’s constantly zooming in and out effortlessly.

The eventual goal of computer operating systems should be to make the entire interface largely disappear by making actions possible by intelligently created gestures, but until that time comes, being able to zoom in and out of current applications would be necessary, making vectorized interface elements an absolute necessity.

But the multi-touch future envisioned here can’t exist until new hardware exists to take advantage of it, and Apple apparently has told developers to have their apps resolution independent by 2008. Is it possible that this is when multi-touch consumer displays will become available? Maybe even sooner?

Says Apple’s Leopard Preview…..

There doesn’t appear to be anything about Core Animation that wasn’t technically possible with OpenGL all of this time. However, Core Animation should make implementing 3d and motion graphics interface features much easier. That being said, take a look at these images of a Perceptive Pixel demonstration by Jeff Han and the example Core Animation video given in Apple’s Leopard Preview.

Notice any similarities?

Yes, the similarities are sort of superficial at the moment.

But the point is that Core Animation could easily be used in the same type of manner. Notice all of the types of media (such as photos and videos) that Core Animation works with? Why, thats the same exact type of stuff you’d want to be able manipulate in a next generation interface. I’m guessing that Apple has set it up so that each Core Animation layer will be able to moved by multi-touch (or mouse and keyboard obviously) input really easily, possibly with each layer being affected by special gestures. A programmer using multi-touch in a Core Animation application wouldn’t even have to do much extra work because almost everything would be handled by Core Animation behind the scenes. A side benefit of this is future flexibility: Apple would have the ability to build in a type of physics engine into Core Animation, and virtually anything else they’d come up with, and it would be able to be used by developers as is because they’d already be using Core Animation for their products.

There has to be a reason that the developer community is so excited for Delicious Library 2.

Researching at NYU’s Courant Institute of Mathematical Sciences. Jeff Han achieved internet fame rivaled only by Ron Paul with his multi-touch technology demonstrations at TED .

Shortly after the iPhone was introduced, David Pogue of the New York Times tried to reach Han for his reaction on the iPhone given that a lot of what Apple was talking about in their iPhone introduction such as multi-touch and the pinch, are things that he’s been doing for a while. Here is what Pogue wrote.

Ultimately, this message appeared on Jeff Han’s site and still exists to this day.

The fact that Han’s displays are larger than the iPhone and the sensing technology is a little different doesn’t imply that Han can’t work with Apple as the basic multi-touch concepts as expressed in software are the same.

Another update later appeared on Han’s site.

As of right now, there appears to be no confirmation that Han has been hired by Apple or that Perceptive Pixel has been bought that I could find.

So he can’t talk about this, and there’s more news coming soon? Whether it is actual involvement or some type of legal settlement, at the very least, it looks like this adds up to some kind of involvement with Apple, otherwise Han wouldn’t have known about the iPhone’s technology and there would be no reason for Han to not give a real comment.

We know that the iPhone runs OS X, or at least a version of it that has some needless files stripped to save storage space, which is always at a premium on a handheld device. We know that the iPhone has multi-touch and Core Animation. We know that Apple has gotten an absolutely insane amount of buzz about the product even before its existence was ever announced, to the tune of over 400 million dollars worth of free publicity making the entire technology incredibly important to the future of the company. We know that Steve Jobs has pimped the shit out of the fact that they’ve patented multi-touch a million times over. And despite Apple’s relatively small R&D budget, it is reasonable to assume they have millions of dollars invested in researching multi-touch. Maybe even tens of millions, but I can’t count that high.

So why would they do all of this, and not think about including multi-touch in the desktop version of Leopard? They have the technology, they have the patents, they’re the only major player that controls both the hardware and software, and most importantly, they have interface designers and long-term vision courtesy of Steve Jobs to to get this right.

Apple withheld the Aqua user interface until OS X developer preview 3.

This demonstrates Apple’s willingness to work on interface features while keeping the developer community in the dark.

OS X 10.0 was originally released in March 2001. That is over 6 years since an interface refresh, which is a heck of a long time in the technology world.

While it is well-known that Microsoft likes to promise the world and sometimes fails to deliver (as Windows Vista had to famously drop the planned WinFS and other features), Apple still has had to have planned a realistic challenge to Vista to maintain its technical lead in the OS war. A quantum leap with multi-touch and a much richer interface would embarrass Microsoft. Regardless of Apple’s small market-share, they have an incredible amount of mind-share at the moment and adding this type of feature would only increase the insane buzz around the company.

And Apple as mentioned is the only major player in the industry that completely controls their own hardware and software, giving them an advantage that they haven’t really taken full advantage of yet.

It has recently been revealed that Microsoft is also working on Surface, their own brand of multi-touch technology. While this largely just appears to be a tech-demo at the moment, Apple has to have some type of plan in the works in the labs in Cupertino.

While there is no guarantee that multi-touch will be arriving with Leopard, I’m optimistic that this is one of the big secrets. The potential that multi-touch has in music and photography and games and so many other tasks is amazing.

Posted on June 10th, 2007 by Chris Papadopoulos.

Comments

1. Hiya @ Jun 10

   Is this your first post? You speak a lot of sense, welcome to my RSS aggregator.

   I’m all for including multi-touch in the desktop version of Leopard but I’d like to move/resize interface elements with the mouse (to point, click and resize with the scroll wheel) and eventually some keyboard shortcuts. I don’t want to slide my fingers across the screen, using fingers is better suited for the iPhone (no mouse, no keyboard).

2. Heghog @ Jun 10

   Great review and some good ideas.

3. Chris Papadopoulos @ Jun 10

   Thanks for the comments Hiya and Heghog.

   Hiya,

   Touchscreens can be useful on mobile devices, but the iPhone’s small screen size really limits what Apple can do there. Multi-touch on the iPhone, as far as I can tell, is really only used to perform the pinch to increase the size of photos. Thats cool, but just scratching the surface.

   A really interesting multi-touch device would be large enough to spread both hands out on it, so that you can perform various types of gestures. Think about setting down one hand as a pivot point and using the other hand to navigate around a 3d model. Think about cupping your one hand and just “throwing” files there to create a new smart folder. There are so many ways to use both hands to create gestures to make various tasks easier.

   Figuring out how to make multi-touch understandable and easy for users to use really is a software problem more than anything. Apple is the best company at making usable interfaces, and I have no doubt that if they’re working on multi-touch, they’ve considered the challenges of the larger form factor and have been studying this problem for years.

   You can look at Jeff Han’s work for some innovative ideas about how a next generation interface could work.

   http://www.youtube.com/watch?v=ysEVYwa-vHM

4. Chris Papadopoulos @ Jun 15

   My prediction seems to be wrong unless Jobs busts out a surprise in October. Developers are still under NDA so there may still be some goodies there.

   Nevertheless, my comments on Core Animation and Resolution Independence enabling multi-touch in the future still hold up.

   Between AppleTV, the iPhone, Leopard, iLife and iWork updates, Final Cut, and other recent and near future products, Apple has a ton on its plate right now so maybe they just weren’t able to get things done in time.

   I’m still hopeful this technology will emerge in the future.