Siemens engineers and the Hallé have developed a virtual conductor for an orchestra using cutting edge technology.
The avatar is being used to direct the world-famous Hallé orchestra as part of the Siemens Hallé International Conductors Competition which takes place at Hallé St Peter’s in Manchester from February 20 – 22.
The concept is the brainchild of young engineers on Siemens’ graduate programme.
The technology will be deployed as part of the innovative Digital Challenge element of the inaugural competition and is designed to test how a candidate adapts to difficult and stressful situations. Siemens will use an Intel RealSense camera to capture a three-minute performance from all 10 semi-finalists. The data will be put through Nuitrack Motion Tracking Middleware to create an animated moving avatar. It will then be projected in front of the orchestra, recreating the same three-minute performance by each conductor. After the avatar’s performance, a judging panel will lead a discussion with the candidate and the musicians about the experience.
John Summers, Chief Executive of the Hallé, said: “The Digital Challenge encapsulates the innovative partnership between the Hallé and Siemens which brings technology and music together. “As far as we are aware this is the first time a virtual conductor has been used in this way to direct an orchestra which, hopefully, will make it both fun and exciting. The manner in which the candidate manages a difficult and perhaps stressful situation with good humour and positivity will be a key part of a dialogue with them in the final round.”
The Siemens team includes Natalie Gristwood, John Mackey and Joe Tasker, who are led by Gary Provis, Business Manager for Factory Automation at Siemens Digital Industries, and have been working on the project for 18 months. Gary said: “The 3D motion camera uses depth and tracking technologies to capture and map the movements of the conductor’s shoulders, elbows and wrists, which are key to the communication with the musicians in the orchestra. We then used some advanced software to create the androgynous avatar which mimics the performance of the conductor in front of the orchestra. It has been a wonderful opportunity for our graduate cohort to experience something outside your typical engineering environment, where they can use creative solutions to solve problems. This is another example of how the Hallé and Siemens, two organisations seemingly worlds apart, can work together to bring benefit to both organisations and our partners.”
After the competition Siemens plans to continue to develop the avatar as a useful tool for conductors and the education programmes that the Hallé and Siemens run as part of their partnership.
The virtual conductor is the latest innovation between the Hallé and Siemens following its Robot Orchestra, which creates music by playing instruments and recycled technology led by a robotic conductor. The partners have also inspired musical engagement in a dementia care home using an armband with electronics that detect movement and convert it into sound.
The Siemens Hallé International Conductors Competition will see four out of the ten semi-finalists proceed to the final where they will rehearse and perform with the Hallé on February 22.
The winner, chosen from the finalists that evening, will be offered a two-year engagement as the Hallé’s eighth Assistant Conductor, under the mentorship of music director Sir Mark Elder, CH CBE and a cash prize of £15,000.
