An experimental adaptation of James Joyces’ A Portrait of the Artist as a Young Man. Designed from the ground up for Virtual Reality. Explore generative visuals in a techno-abstract world. Experience a classic that defined literature for the 20th century.
Finding Focus for Reading
Reading has become more important than ever for many since COVID-19. Finding the focus can be difficult through changes in our home lives and mobility. Virtual Reality offers an opportunity to rethink reading as an immersive experience.
The Infamy of Literature
Classic literature can be overwhelming to read. Often requiring large amount of context from outside of the book. Dated language and long text are common barriers to the average reader.
Joyce was born 1882 in Ireland and became one of the key writers of the modernist avante-garde movement of the early 20th Century. Best know for his complex, symbolic and lyrical writing. His writing is also notorious for being difficult to read.
Characters’ inhabit their inner psyche through verbose expositions.
Stream of Consciousness
Prose flows seamlessly from dialogue to inner monologues. Joyce shifts styles, invents new words and rejects tradition.
Joyce wrote about intimate and mundane events based on his own life. At the time of publishing he was banned for obscenity.
These are affordances that Virtual Reality provides to rethink and adapt classic literature for our contemporary times.
Virtual Reality focuses sight and sound, removing distractions. Full body immersion creates presence like no other media.
Visual abstraction allows readers agency while generating new understandings of the work.
Audio is critical for creating immersion in VR. Joyces’ writing is highly rhythmic and connected to the Irish language.
Interviews and a survey revealed comprehension as a barrier to enjoying classic literature. I explored how to leverage the extended space of VR to provide more context while reading.
Book Discovery Themed Discovery: Dorian Gray VR Modal Footnotes Audiobook Controls
Reimaging traditional reading activities for VR. Discovery of new books, footnotes and hybrid reading and audiobook listening.
Joyce uses repetitive symbolism in his prose to create vivid descriptions. To highlight Joyce’s work as satirical, I explored exaggerating the text by pairing contemporary imagery with excerpts.
My goal had been to improve comprehension, which failed badly during testing. At the time this seemed a failure, I later realized I was using the wrong metric. Reading Joyce is inherently difficult, and does not require comprehension to be rewarding.
Select Word Pull for “snap” Research panel appears below Save notes in sketchbook
Select words and phrases for research. The sketchbook presents the definition, images, wiki, google, translation or note saving functionality.
Testing revealed a lack of play, wonder and awe. The product was focused on utility, but did not amplify or evoke Joyces’ ethos within it’s style. I realized I needed to select a single piece of writing, and commit to adapting it for Virtual Reality.
Joyce writes about early 20th Century life in Ireland. Technology has changed, but our internal struggles for self-realization are persistent. Through a series of co-creation workshops I developed mood boards to re imagine the book through our collaborative lens of pulp culture.
Collaborative Machine Learning
Who would Joyce be in the 21st Century? Stephen Daedalus, our protagonist is heavily based on Joyces’ own life. The story follows him from a child into a young adult. Through found photographs and an array of young Irish men and women, we can visualize a modern Joyce. This is powered by styleGAN collaborative machine learning (Artbreeder).
Stream of Conscious Landscapes
Every reader has their own mental image of a books world. Stephen Daedalus’ crisis and epiphanies are from within his own mind. Rebound amplifies Joyces’ stream of conscious style with fluid AI generated landscapes. Sweeping the reader between sections and chapters in parallel to the books’ dream-like prose.
Beauty can be found in the absence of comprehension. Embracing the bewilderment of Joyces’ prose, visuals are generated through a series of symbolic references found within the text. These are bred together through machine learning to create abstract yet familiar images, a liminal state of understanding.
Elements of the 3D environment react to audio input. Natural pauses in the reading create a visual rhythm. Intensity is expressed through particles and flares when channel thresholds are hit.
Danny Bittmans’ Tilt Brush paintings are used for abstract environments while reading. Post-Processing volumes shift the ambient mood as the user moves through a location. Hybrid experience of reading and listening.
Drifting through the wild while listening to an audiobook was effective in transporting the reader and creating a meditative state. Abstract environments devoid of symbol or language did not distract from audiobook listening.
Post-Processing volumes in Unity Desaturating the spawn location Vibrancy Boost Bloom boost from distance Immersive scene Comfort Settings
VR Product Design
This is only the beginning of understanding literature and new media. Joyces’ experimental writing provided a structure and reference point to create with an array of tools. Each tool provided it’s own quirks which I incorporated into the project. Thinking through making, similar to Joyces’ own stream of conscious style, allowed the project to be amorphous and wildly transformed through iterations. Delegating labor to machine learning allowed me to creative direct asset creation, independent of a support team. Design language for VR is still forming and currently embedded in complex tools like Unity. I’m excited to continue developing new workflows, tools, and communication strategies for XR applications.
- Audio by Audible, Colin Ferrall narrating James Joyces’ “A Portrait of the Artists and a Young Man”
- Video from Pexel: Taryn Elliot, Cottonbro, Zen Chung, Nothing Ahead
- Unity: Oculus Integration SDK, 3D Visualizer Spectrum Vu Meter by Bad Racoon
- Poly Toolkit: Cave Alley // Day 1 (CC-BY) & Snow Melt (CC-BY) by Danny Bittman
- Machine Learning: Artbreeder