Notes

These are a collection of items related to my Books in Browsers 2013 talk. I’ll be uploading the slides here, along with some notes.

Stuff to read

The following are a few related texts to read. Most of this is basic interactive media stuff. While reading is always good, making things and sharing them with people whose criticism you trust is more important.

I might also also be adding references to more texts as I think of them over the next few days.

With that caveat…

Reading list

Beginners

The Design of Everyday Things, Donald Norman.
Don’t Make Me Think, Steve Krug
About Face, Alan Cooper, Robert Reimann, and David Cronin.
Information Architecture for the World Wide Web, Peter Morville and Louis Rosenfield.

Intermediate

Humane Interface, Jef Raskin.
Writing Space, Jay David Bolter.
Anything you can find by Alan Kay. Especially the more complex stuff on programming languages and hardware.

Put the pieces together yourself

With these you need to figure out why they are important to interactive media yourself.

Art as Experience, John Dewey. The talk is based heavily in ideas from this book.
Thinking , Fast and Slow, Daniel Kahneman.
Universal Principles of Design, William Lidwell, Kritina Holden, Jill Butler. AVOID THE KINDLE/EBOOK VERSION! This is not a book that adapts well to the limitations of ebooks.

The Examples

These Pages Fall Like Ash, Tom Abba and Duncan Speakman.

Malcolm Tucker’s Lost Phone

Warning: the following is incoherent and vague. Most this stuff didn’t make it into the talk and a lot of the stuff in the talk didn’t make it in to the notes.

Interactivity is what you do, not what you see, hear, or read

The heart of creating interactive media is when I, the creator of an interactive piece of media, get you to perform an action that is meaningful to you. All art hinges on creating meaningful experiences but interactive art is about being a catalyst for meaning in your own actions.

Tactics for doing so:

Interfaces/Design
Notifications
Responses/Context

A piece of interactive media is composed of a succession of meaningful actions surrounded by a fluid context that changes based on a pre-programmed set of variables. The actions can be prompted indirectly through design, directly through notifications, or as a part of a tight feedback loop through a responsive context.

Notifications aren’t a direct response to user actions. They are separated from their originating subject either through space or time and can originate from the author, the user, other users of the same work, or other outside variables.

An interface is a design that affords action. It exists as a part of a context that it exposes to the user and changes to the context are, again, exposed to the user in a tight feedback loop between their actions and the changes they cause.

The work itself: subject > subject-matter > matter

The subject is easy, it is the material of the work as it exists outside of the work itself, e.g. the economic collapse of Iceland, a doomed mountain climb, or an accident and its consequence.

Where it becomes tricky is with the subject-matter. This is the material as it will exist in the work and filtered through your interpretation, experiences, and choices. The six days of the collapse as seen from the perspective of one of the involved bankers. The circumstances that lead to the tragic failure of the mountain climber’s rescue. The aftermath of the accident scene itself in anal-retentive detail.

Matter is the work itself, the subject-matter as experienced by the user, mediated by the work’s form. To the user the matter is the subject-matter.

This is often pithily described by Marshal McLuhan as ‘the medium is the message’, but I prefer John Dewey’s less cryptic and more verbose take on it from Art as Experience, several decades earlier (on works of art):

This is what it is to have form. It marks a way of envisaging, of feeling, and of presenting experienced matter so that it most readily and effectively becomes material for the construction of adequate experience on the part of those less gifted than the original creator. Hence there can be no distinction drawn, save in reflection, between form and substance. The work itself is matter transformed into esthetic substance.

Or, in other words, the medium is the message.

There’s a good chance that this is where McLuhan got his idea.

While this is true for the user (or reader, viewer, whatever) and is an important fact for you to bear in mind for reasons I’ll get into later, the division of form and subject does exist insofar as it the separation is a cognitive tool useful for practitioners intent on improving their craft. If you are a creator, how a subject-matter is transformed into form is vital. That bit is exactly the part of the process you want to be able to repeat and improve.

But you can’t improve the process of turning a subject-matter into form if you don’t understand that the two are indivisible in the end artefact. The only meaning it has to the user is in what it helps them experience. And in interactive media, that experience is composed of the meaningful actions taken while exploring the artefact.

Three kinds of interactive works based on three kinds of actions:

Functional goal completion. (i.e applications). If Doom was designed in terms of being functional, it’d be a 2D map—click on a target and it’s dead.
Skill-based feedback loops. (i.e. Games). Skill is a determining factor. Applications want to minimise the role of skill. Games put the need for skill in even basic tasks front and centre.
Emotional or intellectual experiences. (Interactive media.) Skill isn’t a factor and you don’t have a functional goal. The experience is its own end.

As far as taxonomies go, this is somewhat useful, but you shouldn’t feel bound by it.

Any piece of work can be all three because all interactive works are based on a multitude of actions contained by a single context.

(Farmville, by the way, is an application, not a game or interactive media artefact. It is almost entirely based around functional goals. The fact that those goals have no meaning outside of Farmville is irrelevant.)

Branching narratives are a red herring in and of themselves. They only matter if the branches or changes in the narrative are a meaningful consequence of the reader’s actions.

In interactive terms what makes hypertext interesting is how exploration creates an emotional experience.

Enhanced ebooks are a red herring. The subject matters where it’s appropriate (certain kinds of non-fiction) are already aggressively integrating video and similar interactive widgets. But, fiction? No. Especially not existing fiction because both text and video need to be adapted to the new context to work. Text that is integrated with video needs to be more staccato, more atomised, with more ‘air’ to leave space for the video. Any enhanced ebook needs a thorough rewrite of the text and so ceases to be ‘enhanced’ and becomes something new, severed from its original context.

An interactive version of an existing text is a retelling, a complete adaptation from scratch. Glossing it over with a few branches and video is guaranteed to fail because you haven’t stopped to think ‘what would this story look like if it had been originally told as an interactive work of art’.

There are huge issues with hanging interactive media ‘bits’ off a linear interface. Any time you require a reader or viewer to break off the main story or film to experience the interactive additions you risk ruining both.

They aspire to mimicry.

Almost every feature people aspire to add to digital is something they see as having lost in the transition from print.

The vision people have of rich interactive content is the vision print has of its own future. It is not a vision of the future of digital, or even of its present.

The truth of what interactivity looks like when it is rich, involved, detailed, and deep is so alien to the print-acclimated mind that they do not recognise it for what it is.

Snow Fall and is ilk are morality plays performed with the intent to demonstrate the redemption of print values and the salvation of design.

And like any other morality play they are propaganda, public acts of faith wrapped in a performance intended to distract you from your nagging doubts, dispell your skepticism, and bring you back into the flock as a loyal believer, or prevent you from straying in the first place.

“Don’t think about the big red button that says ‘press’ or the tools that beg to be used, look at this beautiful static object instead and admire it from afar.”

As interactive media, they are failures.

Why?

They are not nearly as satisfying on mobile phones as they are on desktops and laptops.

The visual effects do not work properly on tablets or mobile phones.

Their unique features consist entirely of making things twitch, move, and animate.

The features that are responsible for their emotional resonance, their true effect, are a result of good design sensibilities that are applicable to both interactive media and print.

Its structure is monolithic, linear, and hard to atomise. It is a blob of content whose structure is fixed and can’t be adaptive or responsive.

None of the labels on the videos or maps are clickable.

It’s hypertextual features are naive and devoid of context, demonstrating a complete lack of understanding of what a link can do. I.e. they use hypertext to decorate, not to explain or expand, linking merely to fluff photo galleries instead of expanding on the narrative, or for redundant link to sidebar content whose role is already indicated through conventions of print design.

The map is non-interactive. Nor is the image of the group.

It breaks almost completely when you disable javascript.

There is nothing in there that brings in outside context.

There is nothing in there that connects interior context to the exterior, it has no boundary surface, there is no seam with the outside world.

Both Medium and Kindle ebooks are better examples of interactivity that add value. Also Readmill.

You can’t separate the structure of any given piece of media from the money that funds its creation. Any content created with funds from NYtimes will have print structures and sensibilities because NYtimes is a print organisation.

Interactivity is what you do, not what you see or read.

Studio Tendra

Explorations of digital publishing, literary apps, bookish websites, and other readable creations

Notes – Books in Browsers 2013