SoundDroid pt.1: the Timeline

Here's the organized output of my researches about the early days of the digital audio workstations. You can track the sources at the end of each excerpts.

Before Skywalker Sound there was Lucasfilm's Sprocket Systems, which opened in 1979 at 321 San Anselmo Avenue in San Anselmo. Film editors worked upstairs on "Raiders of the Lost Ark" and "The Empire Strikes Back" while sound editors downstairs worked on "Alien" and "E.T." Kentfield resident Pat Walsh, discovered while shopping at Seawood Photo, provided the voice for the loveable alien, with the help of actress Debra Winger. Sprocket's parking lot was also notable: It's where Harrison Ford practiced snapping a bullwhip for his role in "Raiders." Sprocket's work on "The Return of the Jedi" came to a halt when the Flood of 1982 ruined equipment. The division moved to Lucafilm's Kerner complex in March of that year.

At the peak of San Anselmo's post-production era, another space was rented on Tunstead Avenue to house Lucasfilm's new Computer Division, located initially on Bank Street. The division would move to the Industrial Light and Magic complex in San Rafael before it was spun off in 1986, relocated to Point Richmond, and became what it is today, the Oscar-winning Pixar Animation Studios.

[sananselmohistory.org]

In 1984, Lucasfilm and Convergence Corp. formed The Droid Works, and under the leadership of James A. Moorer, showed its SoundDroid workstation at NAB in 1985. The product was years ahead of its time and too expensive for the typical studio.

[mixonline.com]

The ASP hardware–based audio processor, designed by Andy Moorer after his work on the legendary Hydra at Stanford's CCRMA, was a proof of concept for what is now considered a digital audio workstation. The ASP's design started life in 1980 and was designed for real–time, multichannel EQ and mixing. SoundDroid, an in–house project of Lucasfilm's Ltd.'s Sprocket Systems that was later spun off as part of The Droid Works, was a hard disk–based, non–linear, 2nd generation digital audio workstation that leveraged the research done on the ASP. Though the SoundDroid project was never commercialized and The Droid Works was later sold to Avid, the audio development team went on to first create the NoNOISE restoration system in 1987, hosted on a Motorola–powered SUN 1, the first true computer "workstation," which had been developed in cooperation with Lucasfilm.

[sonicstudio.com]

The landmark invention of the horizontal timeline for the EditDroid and visible sound waveforms in the SoundDroid, "soft" mixing functions, a "spotting" system for searching sound effects libraries and digital signal processing and noise reduction for the SoundDroid were even bigger milestones.

[editorsguild.com]

The big guys like Lucas had the Sound Droid, which not only edited audio to picture, but performed noise reduction for noisey film sound tracks. I approached Andy Moorer about licensing the noise reduction software to clean up old masters for record companies. CDs were still new, and the record companies did not yet realize the value of their old catalogs (and I am finding out that they still don't). Andy Moorer and Bob Doris went on to form Sonic Solutions and offer No-Noise software that runs on a Mac based DSP card. The rest is history.

[rogernichols.com]

------------------------

George Lucas in his own words:

In the days when punch-in recording and high-speed rewind were on the leading edge, you went from being a user to becoming involved in the development of high technology, such as the EditDroid and SoundDroid systems in the early '80s. Could you talk about that transition?
I started putting together a computer division right after Star Wars and as one of the centerpieces of that division, I wanted to build a new [picture] editing system that was nonlinear and disk-based. It was not done like the CMX and other systems that were out at that time, which were simply designed around the offline/online [tape-based] television post process. I wanted to build something that actually included and focused on the art of editing as I learned it in school. When we did the SoundDroid, one of the things we included was the ability to see the striations [waveforms] of everything just like an optical track so that we could use some of the advantages that optical cutting had. You can see where the words are and see where the sounds are and cut accordingly, which a lot of the older editors really thought was great.

How about the transition to actually manufacturing EditDroid and SoundDroid?

When we were about to go to the next level, I realized I didn't want to be in the hardware business and run a company that built machines and things. We decided to sell EditDroid to Avid and have its ideas incorporated into the Media Composer [picture editing system]. I was very focused on wanting to have an integrated sound system so that the sound editing and the picture editing could be integrated. Unfortunately, I'm still fighting that fight.
[Larry Blake via mixonline.com]
more soon! MM

P.S. Sam shows you his love for Skywalker Sound @ skywalkersound.blogspot.com, it's an unofficial blog and it's not formally related to Skywalker Sound (or LucasFilm) in any way.

Ben Burtt receives Silver Award from the Town of San Anselmo

[Photo Credit: berndt2]

Ben Burtt, who helped preserve San Anselmo history in films, has been named a Silver Award winner. Burtt documented the local floods of 1982 and 2005 in “The Great Flood of 1982” and “12/31.” He says, he “collected footage from 15 or 16 different people and added that to my own. The film shows people being rescued, basements being flooded and so on.” 

Burtt, who says he’s always had “a big interest in history,” first became interested in San Anselmo’s while working on the first “Star Wars” film with George Lucas. He became fascinated with pictures he found in the local library that showed some of the changes the town had gone through.

Burtt also made a lengthy audio recording of the town’s historical walking tour, complete with sounds of old San Anselmo. That recording, a celebration of the town’s 2007 centennial, is available for download from the museum’s website.

[via marinscope.com]
[see footage online of Burtt’s film @ filmnight.org]

Before Skywalker Sound there was Lucasfilm's Sprocket Systems, which opened in 1979 at 321 San Anselmo Avenue in San Anselmo. Film editors worked upstairs on "Raiders of the Lost Ark" and "The Empire Strikes Back" while sound editors downstairs worked on "Alien" and "E.T." Kentfield resident Pat Walsh, discovered while shopping at Seawood Photo, provided the voice for the loveable alien, with the help of actress Debra Winger. Sprocket's parking lot was also notable: It's where Harrison Ford practiced snapping a bullwhip for his role in "Raiders." Sprocket's work on "The Return of the Jedi" came to a halt when the Flood of 1982 ruined equipment. The division moved to Lucafilm's Kerner complex in March of that year.

[sananselmohistory.org]

'The Prisoner of Zenda'
- Playing at the Rafael Film Center on June 13th, 2010 -

Craig Barron, Oscar®-winning visual effects supervisor and Academy governor, and Ben Burtt, Oscar-winning sound designer, will introduce a rare screening of the 1937 adventure-romance 'The Prisoner of Zenda'. Burtt will demonstrate how the sound effects from the film's famous swordfights were achieved.

[via California Film Institute]

An interview with James A. Moorer, pt.1

by Matteo Milani, February 2009

I had the pleasure of interviewing James A. Moorer, an internationally-known figure in digital audio and computer music, with over 40 technical publications and four patents to his credit. He personally designed and wrote much of the advanced DSP algorithms for the Sonic Solutions "NoNOISE" process which is used to restore vintage recordings for CD remastering.
Between 1980 and 1987, while Vice-President of Research and Development at Lucasfilm's The Droid Works, he designed the Audio Signal Processor (ASP) which was used in the production of sound tracks for Return of the Jedi, Indiana Jones and the Temple of Doom, and others.
Between 1977 and 1979, he was a researcher and the Scientific Advisor to IRCAM in Paris.
In the mid-seventies he was Co-Director and Co-Founder of the Stanford Center for Computer Research in Music and Acoustics. He received his PhD in Computer Science from Stanford University in 1975.
In 1991, he won the Audio Engineering Society Silver award for lifetime achievement. In 1996, he won an Emmy Award for Technical Achievement with his partners, Robert J. Doris and Mary C. Sauer for Sonic Solutions "NoNOISE" for Noise Reduction on Television Broadcast Sound Tracks. In 1999, he won an Academy of Motion Picture Arts and Sciences Scientific and Engineering Award for his pioneering work in the design of digital signal processing and its application to audio editing for film. He is currently working at Adobe Systems as Senior Computer Scientist in the DVD team.

[James Moorer (second from left), who gave the 2000 Richard C. Heyser Memorial Lecture at the 108th AES Convention in Paris, Audio in the New Millennium, receives Technical Council recognition - via aes.org]

The SoundDroid is an early digital audio workstation designed by a team of engineers led by Moorer at Lucasfilm. It was a hard-disk–based, nonlinear audio editor developed on the Audio Signal Processor. Only one prototype was ever built and it was never commercialized. Lucasfilm  started putting together a computer division right after Star Wars as an in-house project to build a range of digital tools for filmmaking. The audio project that became SoundDroid was done in close collaboration with the post-production division, Sprocket Systems, and later spun out as part of a joint venture called The Droid Works. Complete with a trackball, touch-sensitive displays, moving faders, and a jog-shuttle wheel, the SoundDroid included programs for sound synthesis, digital reverberation, recording, editing and mixing. EditDroid and SoundDroid were the beginnings of the desktop tools digital revolution.


MM: Mr. Moorer, who developed the concept of a "digital audio workstation", during those early days? How did you collect the ideas for the ASP?

JM: It was my idea from back in my days at Stanford University. My 1977 paper describes a "digital recording studio". The digital audio processing station and digital audio workstation came out of that work. I first coined the term "digital audio workstation" in a talk I gave at the AES. A paper from that talk came out in 1982, but I don't think the term "digital audio workstation" made it into the paper.

Andy Moorer, the director of Audio Research, was moving ahead with the design of the digital audio processor, the ASP. [...] The sound work centered on the invention of a brand new set of chips that could perform the special kinds of calculations required for digital audio. [...] Designing new hardware chips was a long and complex process. Once the engineer determined precisely what needed to be done, and perhaps executed the various algorithms in sofware, he needed to translate the software into fundamental logical steps that could be built with wire and chips. [...] This process of chip design, finding the best position of the chips on a board and the most efficient way to wire them together, was expedited by using specialized CAD software.
By summer of 1982, after most two years of work, Andy Moorer and his cohorts finally completed the first working prototype of the audio computer, the ASP. The massive ASP consisted of eight oversized boards assembled entirely by hand, requiring more than 3.000 IC chips. It represented a number of significant engineering breakthroughs.
[excerpt from Droidmaker]


MM: Who's missing from this list of your crew at that time: John Max Snell, Curtis Abbott, Jim Lawson, Bernard Mont-Reynaud, John M. Strawn?

JM: That about it. John Snell worked on user interfaces (touch-sensitive screens, assignable knobs, moving faders and shuttle wheel). The other folks worked on software.

John Snell had met repeatedly with Ben Burtt and the rest of the Sprockets sound team. They all felt that the audio computer had to look and work pretty much like the traditional tools they knew. [...] Though Ben wanted the flexibility that digital processing and random access might provide, it couldn't be at the cost of changing the interface.
"We felt that you could use one knob or slider and have the sofware change its function," said Snell, "and a console could be built with perhaps eight sliders, or twelve, but not a hundred. [ The old console design ] simply wasn't practical."
Before Peter Nye arrived to the audio project, the ASP was controlled by a command-line interface, much like MS-DOS, and a small box of knobs that John Snell had built for Ben Burtt. Peter Nye, a computer-music student of Moorer's, implemented the paradigm on the SoundDroid for a film-oriented system, which traditionally viewed tracks running vertically, from top to bottom. The new SoundDroid interface featured Snell's touchscreen and a novel by-product of the digital audio: a waveform of the sound. [...] The Droid Works trademarked the feature as "See the Sound."
[excerpt from Droidmaker]


MM: Are you the co-founder of CCRMA at Stanford University? Later, how was your experience in Paris @ IRCAM?

JM: Yes, with John Chowning, John Grey and Loren Rush. It was an interesting experience at IRCAM. I enjoyed my time there. I did not enjoy the internal politics.


MM: Did you help John Chowning during his research of FM synthesis? What kind of sound synthesis run in the ASP?

JM: Yes, of course. I designed the original hardware FM synthesizer that was the basis of the FM patent. Yamaha later built that particular device. We could run all kinds of sound synthesis on the ASP - FM, additive (Fourier), subtractive, whatever.

MM: Thanks to George Lucas who subsidized pure research, for the first time in the movie history scientists and filmmakers were together under the same roof (The Droid Works). What were the big hurdles from the machine-side and from the human-side during the development years?

JM: Technically, the problem was just that we didn't have a lot of the modern technology, like computers with 1 GHz processors and 1 GByte of memory. We had to build all the technology we used. From the human side, people were having trouble understanding the workflow in the digital audio world. I had a very, very difficult time selling people on the idea that you could do "scrub" or "reel-rock" on a computer, rather than by sliding a tape over a head. Someone even suggested that I use the top of a tape recorder as a computer interface, even with no tape on it, so it would "feel" the same. Today, nobody even knows that audio was edited using razor blades and scotch tape.

ASP does for sound tracks - the band of squiggly lines at the edge of a film that encode the film's sounds - what EditDroid does for images: it uses its silicon circuitry to mix, edit, and even synthesize the music, speech, and sound effects so important to film makers, especially somebody like Lucas for whom sound is as important as sight in stirring the emotions of an audience. ASP has already been used to heighten the drama - by changing the pitch of a scream, for example, to make it more chilling - in the current hit Indiana Jones and the Temple of Doom, co-produced by Lucas and Steven Spielberg. Lucasfilm's sound mixer is even more ambitious technologically. Today's films have as many as six sound tracks to drive the multiple speakers in many theaters. But even these are usually the distillation of many more tracks. Project chief Andy Moorer points out that in a film like Return of the Jedi the sounds in a typical scene may represent a mix of 70 separate tracks of dialogue, music, and audio effects. A single change in one of the original tracks - say, the boom of a rocket or the pitch of a siren - would require remixing all of them. Jedi needed only five film editors but 17 sound editors. "What we set out to do," says Moorer, "Was to put a computer in the middle of all of this." That was not easy: just as the visual images must first be converted to electronic signals for EditDroid, so the sounds must be turned into digital form for ASP. This means every flutter of noise has to be translated into the "on-off" binary language of the computer. In other words, the sounds become numbers.
[excerpt from Discover Magazine, August 1984]

[ 1 | 2 | 3| Next ]

An interview with Gary Rydstrom

by Matteo Milani and Federico Placidi - U.S.O. Project, 2011 

Gary Rydstrom was born in 1959 in Chicago, IL. He graduated from the University of Southern California - School of Cinematic Arts in 1981. He began his career at Sprocket Systems, formerly Skywalker Sound, in 1983. Offered the job by a college professor, Gary received the opportunity to work with his mentor, Star Wars sound designer Ben Burtt. He created sound for numerous successful films including Backdraft, Terminator 2: Judgment Day, Jurassic Park, Titanic, Saving Private Ryan, Minority Report and Finding Nemo. Through this work he has won 7 Academy Awards.
Rydstrom did his first work for Pixar on the short film Luxo Jr.. John Lasseter has said it was Rydstrom's work on Luxo Jr., such as creating the lamp's voice from the squeak of a lightbulb being screwed in, that taught him how sound can be a partner in the storytelling of a film. In 2006 he has made his directorial debut with the Pixar animated short Lifted. He recently jumped again into the director's chair to create his second animated short Hawaiian Vacation, set to play in front of Cars 2. 

[Rydstrom talks about the first of Pixar's new brand of Toy Story shorts @ AWN] 

For every director and for all of us the goal is at any one moment in a movie to have the audience on the edge of their feet and can't wait to see what happens next. You are moved by these characters, you believe in these characters. All the meticulous and hard work should be completely invisible. We wanted to be involved in the scene and we don’t you to think that Nemo is a bunch of computer layout. We do not want you to think the many hours that took to create that scene. You are just carried away by the scene and every focus in every step of the production that we do at Pixar is about the story. It is about entertaining the audience. - John Lasseter, 66th Venice Film Festival 


USO: Gary, you’re the third person – of our personal “respect-list” – who came from a sound career and crossed over to a director’s chair: Walter Murch did it first, Ben Burtt came after him. They still design their own sound for their works, just as you did for your first short movie Lifted (with a tribute like the Wilhelm Scream at the very end). How did you manage the transition from sound designer/re-recording mixer to director? How about your feelings?

Gary Rydstrom: Working in film sound is a great way to immerse yourself in the rhythms of a movie. Having done sound for so many movies, and so many different kinds of movies, I’m hoping I’ve developed some mysterious “film instinct.” I think the best directors – and best sound designers – work more from their gut than their brain. In other words, to make movies, your gut has to be bigger than your brain… or maybe I should rephrase that?


GR: "It was a little scary but I always wanted to make films and wanted to have opportunities to write and tell stories. So Pixar being old friends, they gave me this opportunity to come over there and do that. So it was a little scary because there were a lot of things I had to learn. But doing sound for so many years, what I found was that the big similarity is that sound is all about rhythm. It’s about using sound, rhythm helps delineate sound effects, sound tracks, and telling a story with these kind of rhythms is really key. Animation is really about rhythms and timing so I think working in sound gave me a great sense of timing. In fact, on Lifted I used, before animation had been started, used a temp soundtrack to express the timing that I wanted to the animators so they had some reference of what I was after. It was a way for me to even communicate to the animators." - CanMag

[Lifted producer Katherine Sarafian and director Gary Rydstrom © AWN Inc]

USO: Did you develop and learn your new craft in sound room working movie by movie?

GR: Central to sound design is finding what feels right for a movie, what matches its look and heart. So every movie I worked on taught me different lessons. I kept thinking I was close to end of my lessons, but they never ended. Turns out that’s what makes working in film fun. It’s never the same twice. 


USO: Being close to first-class directors during all your sound career has thought you well?

GR: I felt like a spy, watching a lot of directors. I’ve been very lucky to have worked with many of the best – Spielberg, Cameron, Lasseter, Redford – and you probably don’t need me to tell you (their movies do) how different in approach they were. One thing the best have in common: they inspire their crews.


GR: "... I've had this long, great relationship with John Lasseter and Pixar. I've felt involved throughout the whole filmmaking process on their films. They offered me an opportunity to develop and direct films, maybe because I bring an outsider's perspective while still being a Pixar guy through and through.
My friends there know that I've had a long-standing love of comedy. When I first told Steven Spielberg and George Lucas that I was doing this, they were touchingly supportive and generous with advice. I'm grateful for my sound career. It gave me the equivalent of 50-yard-line seats, second row, during a fascinating era in film history." - Mix Magazine 


USO: Can you describe what you were feeling when you left the Technical Building (where you spent almost 20 years), and all your friends at Lucas Skywalker Sound, to your new life/experience in Emeryville?

GR: Luckily I get back to Skywalker Sound quite often, otherwise I’d REALLY miss my old friends and career. Pixar’s no slouch, but there isn’t a more beautiful place to work than Skywalker Ranch. 


I've known Gary for over 20 years and he's really been a mentor and a role model for me, as to how to do this work. His standard for quality, inventiveness and humor is really always in the back of my mind when I work. No one is a quicker study when you break down a scene or a film in terms of what, soundwise, is the best direction to go to serve the story. - Tom Myers, Skywalker Sound 


USO: What does it mean for you to associate a particular sound to a visual event (identifying it in a vast catalogue as big as the sound library of SkySound)? What are the mental or purely instinctive paths competing in making the choice?

GR: Something magical happens when a sound effect is added to picture – and it’s not predictable. After all my years of doing it, I still depend on experimenting, putting sounds against image and seeing what happens. First time I did this, as a film student, it amazed me how sound could “open up” a movie, how the combination of sound and visual could create something greater than the sum parts. Having a great sound library is essential, but the real secret is how one uses it. 


GR: "I wanted to give the lamps in Luxo Jr. character through sound. I told John (Lasseter) that I'd come up with these voices. He'd never imagined they'd have voices and was wary of the idea. But I experimented with taking real sounds — a lot of it as simple as unscrewing a light bulb or scraping metal. Every once in a while, a sound would be produced that would remind you of sadness or glee. I always think of sound design being like prospecting for gold. Start by, say, goofing around, making lots of sounds, then find the one percent that has something interesting about it. Put this against the film, and there's a magical moment when the sound, if it's right, merges into the image, brings it to life. They were not cartoon-y. They were fun, reality-based sounds. It felt like the birth of something new, even then." - Mix Magazine

USO: Many sound artists working in other domains like electroacoustic music, musique concrete, environmental sounds has been strongly influenced by the contemporary cinema, and by its ability to create stories with the help of sound effects, soundscapes and even to define the personality of some "objects" with unmistakable sounds. How it is possible to interrelate these multiplicity of experience? 

GR: What all sound artists share is a desire to convey emotions, so I certainly was inspired by non-movie sound work. Sound is emotion. Not just music, but all sound. Humans (who can hear) seem to take sound in general for granted, which is frustrating, but liberating. How manipulative can we be when no one’s paying attention! 


USO: While Europe was experiencing electronics and sound design like natural language consequence of the avant-garde of 50's, in the U.S. this journey took place independently in the field of the film industry with other purposes and objectives. You were aware of what was happening in the old continent and its experiments?

GR: I’m certainly aware of the European traditional of producing “soundscapes” (for lack of a better word) for radio. In some ways, I was jealous of sound work that didn’t depend on the visual – how free it seemed! Best I could do was build an “off-screen” world in film. But I always had a movie to be influenced by – how scary to have sound work stand on its own. 


USO: You've been one of the most assiduous "regulars" of the Synclavier, an instrument widely used in various musical fields. Can you describe the creative approach with this tool? What were your procedures? What made it such as a instrument so unique? To create the sound of the engines of the Titanic you and Chris Boyes worked long on the Synclavier to reproduce the effect in question. Do you remember how you did it?

GR: I fell in love with the Synclavier early in my career because it was such a powerful instrument for shaping natural sounds. I never used the FM synthesis – even making electronic sound effects I would try to use natural sounds, just because I find real sounds are more interesting. Sampling sounds and putting them on a keyboard allowed me to quickly experiment with pitch and layering, which are my primary tools for bending real sounds to my will. The Titanic engine sounds took advantage of how the Synclavier could speed up and slow down a sound pattern. For me, nothing beats using a “musical” instrument for creating sound effects.


GR: "The idea of using a sampler for sound effects work had astonishing potential. With sampled sounds in RAM, you can instantly pitch-bend it and layer it and play it and shape it, without using any processing time. You can layer on the same key and very finely manipulate the pitch and delay and merge them together in ways that were harder to do in the tape-to-tape days. It allowed me to create the dinosaurs in Jurassic Park, in which I took several layers and blended different animal sounds into what sounds like one animal.
With the Synclavier, I have a library of sound “parts,” little snippets that are like phonemes in language. Interesting bits of sound that can be rearranged in multitudes of ways. It's a library of raw material, and it's valuable still." - Mix Magazine


USO: How about your relationship with Ben Burtt along your sound career and especially now in Pixar? I mean crosstalk and dialogue, to share ideas. Could you describe him as a mentor, a friend, a colleague, a druid?

GR: All of my approaches and philosophies about sound come from Ben Burtt. At the time he was revolutionizing sound design, I was lucky enough to get a job at Sprocket Systems and see how he did it, how much he knew about film sound history, and most importantly how much he cared about using sound to tell a story. When I started working on Jurassic Park, I remember Ben was away on vacation. I stumbled along until he came back – for all sorts of reasons, psychological and practical, I couldn’t get started until he was in the building.



[Gary Rydstrom Talks About Cinema Sound - via DolbyInsider]


USO: Nowadays many sound designers are working in nonlinear environments like Pro Tools, Nuendo with tons of plug-ins to make everything. One thing we always admired in your work is the exquisite and unfailing "organic" sound and his innate musicality. It seems that sounds "exist" in nature and are not a product of a skilled craftsman. Would you like to talk about that?

GR: There’s a danger in processing sound too much. I believe the best sound effects come from the best raw recordings, and are tweaked as little as possible. The world is so full of amazing sounds – sounds no synthesizer can match – so why not find them and use them?


GR: [...] I remember a scene in the first Mission Impossible in which Tom Cruise breaks into a computer room at the CIA, for which we’d added all these sound details for equipment he was using to lower himself in. Yet the idea was that if he made any sound over a certain level, he would trip the alarm. Brian De Palma ultimately said, “No, take it all out.” And for the most part, that scene plays with nothing on the track. I went to see it with an audience and it had the desired effect: It made everyone lean in, pay closer attention, get nervous. Tension comes from the silence of that scene.
[...] Silence can be thought of as a type of sound. It’s like when somebody years ago figured out that zero was a number. And silence is just as valid as an amazing sound. Every sound editor can’t help but think of how to fill up a track; it’s what we’re paid for. - excerpts from “From here on in, absolute silence.” [via Benjamin Wright]

Thanks Gary... keep up the good work! M&F