CCNMTL (1999-2015) pages for archival purposes only. Please visit CTL.columbia.edu.

QuickTime Darwin Server to Go Silent

Earlier this month, a short innocuous announcement appeared on Columbia’s IT department’s notifications page alerting everyone of the upcoming “QuickTime Streaming Server Retirement” scheduled for July 1, 2013. The three-sentence message referenced a machine named Mineola, which probably made sense to about four people outside of the IT department. And that was that.

The announcement stayed in my mind for a few days and it eventually occurred to me how irreverent this message was compared to the value of that server over the years. After I mentioned this to a colleague about how critical the QuickTime Streaming Server (QTSS) had been to the use of media in the classrooms at Columbia and how quickly such a valuable technology loses traction and is “retired,” he quipped, “You should write an ode to Darwin.” I have.

Darwin was an early open-source application and for this reason we gravitated to it...Darwin was Apple’s code name for the open-source version of their QTSS commercial product, although that difference is now largely irrelevant. (Now is also a good time to get out of the way any snide comments about “natural selection.”) Released in 1999, Darwin was an early open-source application and for this reason we gravitated to it at a time when audio and postage-size video distribution was overwhelmingly done via RealMedia servers. To put this into context, this was a time when Columbia still had an enormous 56Kbps dial-up modem pool and YouTube was still at least a half dozen years in the future. (Interestingly, the pages for the dial-up modem pool still seem to be active on the Columbia site.)

The RealMedia servers allowed some level of programmatic control over the media using SMIL (Synchronized Multimedia Integration Language), an HTML-like markup language. We made extensive use of SMIL in some of our early projects--such as multimedia study environments for Midnight’s Children and Danto’s “Artworld” for indexing the videos. Some of these projects still function today, notwithstanding some limitations. One of these early CCNMTL projects that used RealMedia was called Third Space which allowed the use of video snippets in a discussion board, a novel idea at the time. In 2001, Third Space was used extensively in courses where the analysis of video segments, such as reviews of clinical interviews, could be discussed online, quoting video segments in a similar way one would quote a text passage. The project became successful enough that at one point a third-party discussion board was purchased because it could more easily accommodate the Third Space code than the local offerings.

The Third Space code made use of URL parameters (those long strings that follow a question mark in the URL) to manipulate the start and end times of the clip of interest. The Columbia Innovation Enterprise, the tech transfer department of that time, found the implementation novel enough that it applied for a patent for Third Space and eventually sold the patent application, with CCNMTL receiving about $4,000 from that sale.

By 2002, Darwin had matured enough to become a feasible product and we started running the server on our local machines. The first QTSS server we put into service was on an aging PowerMac on April 1, 2002, according to the ticket from Columbia’s hostmaster, the arbiter of local internet nodes. As interest and use of QuickTime media increased, we convinced Melissa Metz and her team at Academic Information Systems (AcIS) to run the server centrally, especially since Darwin had been ported to Solaris, SUN Microsystems’ OS that was standard for the central IT organization. In September 2002, Darwin was brought online on two machines for redundancy and capacity, Mineola and Bergamot.

CCNMTL experimented with QuickTime Streaming because it offered something quite useful: JavaScript control over a movie embedded on a web page. This control allowed us to take the Third Space idea and turn it into what eventually became VITAL, a technology that led to CCNMTL’s first large NSF award (~$5 million over 5 years) with Teachers College professor, Herb Ginsburg. The grant, received in 2004, supported the development of an environment for courses that focused on the close study of video or filmic material. Over the term of the grant, many improvements were made to VITAL, and eventually hundreds of courses ran on the platform each semester using one of its two main features: the ability to compose multimedia essays using clips from the class media library and guided video lessons. The guided video lessons were akin to the use of video on current platforms like Coursera where students watch a short video and then answer questions about what they just watched. VITAL’s responses were in the form of short paragraphs rather than the more common multiple-choice questions.

For the last couple of years, we have been moving courses out of VITAL and into the next generation media analysis and annotation tool, Mediathread, partly in anticipation of the QTSS shutdown. So, as the Spring 2013 semester closed out, the last students using VITAL completed their coursework and the system will be made unavailable for new courses.

While VITAL and other similar tools relied on the central QTSS servers, these servers encountered a number of problems the last few years. The servers became “orphans” when the academic and administrative IT departments merged in 2005 to form Columbia University Information Technology. The servers were mostly supported through the kindness of former AcIS employees, while oversight was bounced around from department to department. Then things really started to unravel: first, the load balancers that offered the redundancy began to fail due to incompatibilities; the idle Bergamot machine was moved out of the pool and used for something else; Solaris became passé and was eventually replaced by Linux as the preferred server OS; and Apple stopped development on the software (last stable release is from 2007). Since then, the QTSS have been on life support. With the server now needing constant rebooting to keep it chugging and with the departure of several CUIT system engineers that knew how to coddle the server, we knew that QTSS was doomed. It is now scheduled for a July 1st shutdown as per the announcement and the Darwin era will end at Columbia.

The existing QT media will be accessible using the central RealMedia servers, now called Helix, and also with a suspect future. Helix has integrated QTSS support into its servers and for the most part, the media plays fine, but they are not useful for the fine-grained control needed by tools like VITAL.

The demise of the Darwin QTSS server highlights a growing important issue with digital assets: their short lifespan.The demise of the Darwin QTSS server highlights a growing important issue with digital assets: their short lifespan. These digital assets are created (born) in a three-dimensional cage composed of file format, media player, and media server. Without major investments in refreshing vast catalogs of media and assets, the loss of just one of those dimensions is sometimes enough to make the asset unavailable forever. Thus far, the only relief has been to depend on services like YouTube which take on the responsibility of maintaining the media up to date, such as when YouTube added HTML5-based versions of the media for iOS devices a few years ago. University holdings are also quite dependent on the existing local content expert that curates and maintains media. Personnel transitions usually lead to a loss of access when there is no one available to shepherd a transition or refresh to a new format.

Lastly, there are always many key system engineers that never get credit for keeping these systems running smoothly year after year. In the decade of Darwin, the aforementioned Melissa Metz, Rimma Ashkinadze, George Giraldi and probably a couple of others took turns over the years to coax and cajole the Darwin servers at Columbia, answering pagers at all hours to keep the servers streaming so students could do their close observations and analysis of the media. Thanks.