- 1 Introduction
- 2 What Went Well
- 3 What Went Poorly
- 4 Lessons Learned
- 5 Next Steps
Ripieno is the Spring 2017 animation project. In following the tradition of the past few semesters, the project had two separate deliverables: an animated film built from the pre-production document provided by the past semester, and a pre-production package that we would hand of to next semester’s team. In this instance, the animation was a VR short film called Melody of Life, which centered on showcasing three different cultural dances as portrayed by toys. Our pre-production package, which was selected from five pitches we gave to faculty, was called Agloe and was meant to focus on the disparity between artistic intent and audience perception.
Ripieno consisted of six team members: Melissa Schoeller (producer, writer, story artist, 2D artist), Mengyang Li (environment concept artist, 2D artist), Flora Cheng (character concept artist, 3D character modeler), Prasanth Iyappan Raviraman (tech artist, motion capture artist, animator), and Julian Korzeniowsky (sound designer, composer).
As a team, we were lucky because we were all incredibly dedicated to the work. The animation project is an intense challenge; most of us had never made an animated short in our lives, and all of us were learning new skills as quickly as we could. Therefore, when the technology broke or the render farm took longer than expected, nobody blinked twice before agreeing to put in the extra time to make the project great. If team members disagreed, they were able to find common ground due to their shared dedication. It made all successes sweeter and all challenges more bearable.
What Went Well
The purpose of this project was to create a deliverable, but it was also to learn the entire pipeline as it relates to animating in 360. Melissa knows more about managing the pipeline than she could have learned from any book. Flora learned to design for both motion capture rigging artists and for models that required outside consultation. Prasanth had never before encountered motion capture, and had only ever done the most basic of animation. Mengyang is new to concept art, and had to learn the ins and outs of what is required to communicated to a modeling team. Sharon wanted to learn about lighting and VFX, and she gained so many skills in implementing these for 360. Julian, being last in line, not only learned out to compose for the 360 space, but how to make incompatible software work together. We faced a fair number of production challenges that required us to learn, and we did so enthusiastically and with vigour. We can all agree when we say that we know far more about how to make an animation and how to work in 360 than we did going in, and it has prepared each of us for our next positions in both project semesters and industry.
Pre-Production on Melody of Life
Although we were given a pre-production package from the previous team, Catharsis, it was a bit thin. It contained a creative directive, but no assigned cultures which we should portray, no follow-able storyboards, and no concept art. In considering how to handle the amount of work we had ahead of us, we decided it was best to take our time and focus on how we were going to make the piece before we started making it. We iterated on story and storyboards, we acted out scenes, and we filmed in 360.
Although we did not necessarily make concept art that was meant to be mimicked in the scene, just about everything else was carefully planned. By the time we went into filming the final piece, we knew exactly what objects would be placed where, what physical motions were interpretable by our audience, and how transitions would shifting gaze. While it was terrifying to not see the animation in its entirety until softs, all that hard work in the planning phase meant that the amount of iteration we needed to do on the story and character acting in the final weeks was minimal.
Part of the directive was to have three independent cultural dances. We knew we wanted them to be from across the globe, and we knew we would need them to fit our narrative, so as we alternated back and forth between narrowing story and narrowing culture, each informed the other. Finally, when we settled on India, Ghana, and Brazil as our countries of choice, we knew exactly what order we wanted them presented and what sort of dances we wanted.
More importantly, we knew that we knew very little about each of the dances we wanted in our film. Of course, having Prasanth on our team meant we knew more about how to portray India’s environment, but Bharatanatyam was definitely not a dance he grew up learning. We couldn’t just give random dancers some choreography from youtube and throw them in the motion capture studio. We found dancers and musicians who were either part of the culture or trained in the art of that dance or music, and we asked them to consult on all phases of the design process; they gave final approval on character models, environment design, choreography, and music. It was scary giving up that much creative control to outside resources, but on the other end of it we feel our final product is more authentic for having done it.
What Went Poorly
A Team Unsuited to Assignment
For a team that was tasked with animating in VR, we were shocked to find we had no one trained in animation and no programmers to build in the game engine. The one thing Catharsis gave us, beyond a thematic pitch, was a VR sandbox, and we couldn’t even use it. We ended up losing two weeks to simply figuring out how we could change the project so that it was still in a virtual reality headset, but would not require the use of any programming. We were also animating with no sense of the principles of animation. This was most famously evident when, after losing two hours removing exaggerations from the motion capture data to make one dance look more realistic, Dave Bossert came to critique us and told us to do the exact opposite.
Admittedly, it is odd to say that having an unbalanced team is a thing that went poorly, because in the end we overcame this setback. However, at the end of this project, we all wish we had one more month to get the experience to a level that meets our standards. Between having a programmer - which would have allowed us to circumvent the render farm - and an animator, we might have been able to focus our attentions more intently on choreographing in VR, rather than simply getting the work through the pipeline.
We also do believe that it is an impossible task to assume what team is going to be on the project in coming semesters, and therefore designing a pre-production package for next semester is prone to intense rewrites, setting next semester back the same way that we were.
Working in the Midst of an Upgrade
In 2017, Maya went through a huge upgrade. This upgrade came with brand new render software: Arnold. Arnold was not yet ready for common consumption, and required numerous upgrades throughout the semester. These upgrades caused a delayed access to the render farm, access to a spherical camera, problems transferring animation, problems render textures... you name it, we encountered it.
The problem was, we were so far into the production with each new upgrade that we felt scared to update the software, lest something we had previously built broke as a result of something new in the code. However, holding off to upgrade also meant that we were a reactionary team, rather than a proactive one. It took sending something through the render and seeing it look wrong, or the export from Motion Builder glitching out when imported into the environment in Maya. We firmly believe in the saying, “a stitch in time saves nine,” and it was how we approached the production as a whole, but we were taking nine stitches every single time when it came to dealing with Maya and Arnold.
Scheduling Motion Capture and Recording
There were many barriers preventing us from completing the motion capture sessions in a timely manner. First, we spent three weeks simply selecting and confirming which cultures we were going to be able to use for our animation. From there, we needed to consult with them, record music for them, and give them time to choreograph their independent dances. The final dance could not be composed until after we had all the music, and could not be choreographed until all other dances were done. What this meant was that, despite wanting to be done recording motion capture by halves, we were not able to complete even the cultural dances until after spring break.
In addition to the three cultures, there was the issue of casting Manny. We had initially tried to go through Attack Theater, a troupe that Catharsis had connected to the project last semester. They had expressed interest, and so we thought we would follow up. While they were still interested, it would typically take two weeks to a month between emails and, finally, when we didn’t hear from them, we reached out to the CMU School of Drama. Looking back, we probably should have started there: 1) students in the school of drama do not necessarily expect payment, 2) their schedules are significantly more flexible than an outside troupe, and 3) getting them to either or project room or the motion capture studio was incredibly convenient. Had we found our dancer, Javier Spivey, much sooner, we could have commandeered more time in his schedule, conducted motion capture sessions sooner, and had time for reshoots.
Completing Two Pre-Production Packages
It is hard enough to complete a single pre-production package in a semester, but because the package we were handed was insufficient to begin production, we ended up having to make two independent packages in addition to our final animation. The pre-production package for Melody of Life did not even have a story (it was heavily dependent on non-narrative “experience” and gaze-activated art), and writing that story meant that we were not able to start true production until week 7. These events resulted in us not being able to move our concept artists to Agloe until well into week 12. While we were iterating on Agloe story all throughout that time, putting out metaphorical production fires in Melody of Life meant that there was rarely a moment before halves that Agloe could even get our writer’s full attention (since the writer was also the team’s producer). We are not sure at this juncture, given the amount of modification each team does to the package they receive, as well as the amount of time teams have available to focus on the package, that it is wise to make the pre-production package anything more than a pitch. Providing a story summary, concept art, and resources for modeling, motion capture, and animation is in greater service to the team than anticipating incoming skill and being too proscriptive with storyboards, or too loose with creative direction.
What you learned through the process of the project and the course of the semester and a wrap up with next steps (i.e. the client is taking it life on a certain date, or the team plans to continue working on it, etc.)
360 and the Complexity of Animation
One thing we learned very early on is that creating a single shot in 360 completely changes the way in which we might have strategically approached animating and rendering. Normally, with a multi-shot sequence, events to not have to be made in order. However, a film in 360 is a single shot. What this meant was that Manny, who moved in every single moment of the animation, was the defining factor of the animation schedule. We had to render in order because any small changes that occurred in a particular frame of Manny’s animation would have a waterfall effect on anything he did later on. The rest of the team did not understand that soon enough, so we created all other scenes out of order, thinking we could render sections in order of degrees of difficulty to construct (allowing the render farm to do work while we did), only to find out mid way through this plan that this would not work for our animator. When it came to future passes, we made sure to create cutoffs based on sequence, rather than ease of delivery.
Nearly every time we rendered frames, we did so at optimal resolution. We were concerned that we and playtesters alike would be distracted by the fuzzy images and not be able to critique appropriately. This was an incorrect assumption, and it doubled our render time. Renders that took weeks to see and respond to could have had a 3-day turnover, allowing us more opportunities to fix animation, lighting, and texture. If we could do it again, up until final were finalizing our content, all renders would be done in lower resolution. We could then, as scenes were locked in, be re-rendered at higher-res.
As of right now, Melody of Life is ready for release for specifically Oculus viewing. Despite 360 video being very accessible (it can be viewed on all headsets, as well as smartphones and computer screens), 360 audio requires a little more effort. Because there is no universal system for distributing 360 video and audio between platforms, Julian will be spending some time after our final presentation making sure we have a Facebook and Youtube accessible version of the final product. We will then give these versions to Jaehee Cho so that the ETC can display it on its youtube and social media accounts.
In addition, Prasanth and Sharon have discussed the possibility of doing additional passes to add improvements to the animation sequence, but they make no promises. One thing we know we would like to do is submit Melody of Life to a number of festivals, and are doing research on which would be appropriate.
Regarding Agloe, we will be passing it along to Fall 2017’s animation project. We have included our names in the package, with the promise that they can reach out to any of us with questions. However, as was the case with projects in the past, we anticipate that they will make vast changes depending on the individual skills on their team and the portfolio-building interests each person has.
One thing we would like to stress is that, despite all these challenges, we do hope that the animation project continues to be an animation project. Three of the six people on our team want to go into the animation industry, as do a number of people in the building. It would be a shame to turn it into an overall film project, as that would reduce the opportunities for them even more, and discourage future animators and animation industry-focused individuals from enrolling in the program.