• BBC

BBC Digital Media Initiative (DMI)

Summary

Project Length: 1.5 Years

Products: Custom Codec and File Container, Custom Sequence Renderer Supporting 25 effects from Avid MC / FCP / Premiere and rendering them all identically, Custom Disk and Tape MAM, Partial File Restore from Disk and Tape Application, Custom Avid MC / FCP / Premiere EDL conversion application

Roles: Test and Development Lead, Solution Architect, Technical Project Manager

HOW WE MADE A DIFFERENCE

A challenging project with a large scope and number of ‘firsts’ to deliver upon with complex change-requests to handle. Despite the BBC cancelling the overall programme, our project was a success and I’m proud of what my team and I delivered.

The top achievements were:

  • Create a custom and non-proprietary media format
  • Move entire edit sequences between Avid MC, FCP and Adobe Premiere
  • Render and encode entire sequences from EDLs in Avid MC, FCP and Premiere outside of the applications, allowing editors to continue editing while the sequence was exported in the background on another server
  • Respond to a change request to move from sequences, formats and players to go from being frame accurate (approx 1/25th of a second) to audio sample rate accuracy (approx 1/48000th of a second)
  • Maintain the references back to the master file even if a sub-clip is made of a clip of a clip (etc.) in a 3rd party NLE

We were required to develop and deliver:

  1. A custom codec and file container that was platform agnostic
  2. A custom disk and tape MAM that could manage the same assets across multiple physical sites (for redundancy) and access the assets from the fastest / most available based on the location the user was accessing content from
  3. A transcoder and transwrapper that supports a wide range of codecs and wrappers from a very wide range of devices and software
  4. A custom sequence renderer supporting 25 effects from EDLs from Avid Media Composer 3, 4, 5 and 5.5, Adobe Premiere and Final Cut Pro
  5. An application that can restore user selected parts from a rough cut sequence of media assets from tape / disk to disk rather than the entire asset (for network transfer efficiencies) and rebuild the sequence in a local Avid MC, FCP or Adobe Premiere NLE
    • This needed to include configurable ‘handles’ so that editors can trim / extend the length of a clip on a timeline to tweak/improve an edit while only showing the part of the media that was originally specified in the sequence against the full-length source asset
  6. Custom Avid MC / FCP / Premiere EDL conversion application

ROLES

  • Test and Development Lead

    • Wrote project test strategies and plans
    • Managed 11,000+ automated and manual tests per day
    • Managed local and remote test and development teams
    • Took a ‘risk-based’ approach to testing
    • Led daily stand-up meetings

  • Solution / Software Architect

    • Reviewed and designed our response to requirement specifications
    • Redesigned low-level software workflows and UUID management
    • Defined software tolerances that would calculate resulting system behaviour

  • Technical Project Manager

    • Managed the project for Mediasmiths
    • Reported to the CEO and CTO
    • Reported to and liaised with the client
    • Responsible for ensuring project milestones were met
    • Managed a RAID log

I was originally hired as a tester, however as the pressure, project complexity and responsibilities proved to be too much for the test manager. I stepped up to fulfil the role.

The SoW specified that a large range of codecs, in a range of containers/wrappers from a number of cameras (etc.) was in scope. While all of the codecs and containers are based on standards, from experience, each camera, recording device and NLE encodes slightly differently.

As test manager, I began writing the test plans for the products we were delivering. To comprehensively test the many variances of media and application configurations  (all the varieties of supported codecs, file containers, metadata, rendered effects, EDLs, etc.) I calculated that we needed to perform over 11,000 tests with results accurate down to 1/48,000 of a second. New versions were planned to be released to the test team every day. This clearly wouldn’t be possible with the few testers available and the limited amount of time between functional releases so I put together a proposal for a different approach to the testing. This consisted of the following:

  • We identify, collect and organise a full complement of test media
    • Every video and audio codec in every media container (muxed/interleaved and demuxed/atomised) from every supported camera and NLE
      • Short promo length (30 seconds) versions for faster regression testing
      • Long-Form episodes (~40-50 minute) versions for more realistic load testing
  • We develop an automated test suite
    • New tests would be created alongside the planned delivery of new functionality until all of the 11,000+ automated tests were developed
  • We hire another 2 testers
  • We take a ‘risk-based’ test approach. This included:
    • Identifying the scenarios with the highest chance of failure and the highest impact on users/business should it fail
    • Human testers would then manually perform or review the automated test results in more detail for these tests

After significantly improving the testing effort I was then promoted to test lead on the supplier side. This involved writing an updating the test strategy and plans. I became responsible for managing the full test team of 6 people of varying levels of experience. Some members of the team were new to testing and all but 2 had no previous experience of testing with media assets. In addition, I became responsible for liaising with the leading our UAT sessions, where we had to identify and agree a subset of tests that would prove our solutions to be working as required.

As different parts of the project were nearing completion, it became possible to widen the integration testing with other supplier applications and hardware to test our workflow applications as part of the wider workflow. I wrote the test strategies and plans and was responsible for achieving their sign off with the BBC.

This included managing both local and remote development teams.

Due to my success in managing the test team and reporting defects/working with the development team and my understanding of the solution, I was also promoted to manage and lead the development team. I employed an agile-like development methodology (I say ‘agile-like’ as there was a fairly clear specification in place in advance) with daily stand-ups and a lot of interaction between development and test teams. This meant that we could immediately test and feedback the results of test into development to maintain a high pace and quality of the applications.

Once leading the development team, I came to discover that the design of  a key part of the workflow  was incorrect and would not work. For example, it required that media would maintain genealogy by inheriting the ID of the parent media. Avid Media Composer required this ID to be unique, otherwise it would fail to load the assets at all. I escalated the issue to the suppliers CTO and ran tests to provide conclusive truth that the current designs wouldn’t work and presented it to the client. I then worked with the client’s Solution and Enterprise Architects to redesign a suitable alternative that could achieve the requirements in a way that would work with the required 3rd party tools. By identifying this early, it greatly improved the chance for success as the project timescales were already very short.

After successfully managing the test and development teams and also leading our technical design response to the BBC’s requirements specifications, I then led the project, reporting into the CEO and CTO and directly to the client. I became responsible for ensuring that all milestones were met, liaising both to internal and the client’s senior management teams and managing a Risks, Assumptions, Issues and Dependencies log.