互動藝術的發展，使視覺融入音樂。對互動電腦音樂的發展來說，同時引入視覺的控制，更能激發創意與趣味性。無論對視覺藝術溶入音樂的互動發展來說，亦或音樂融入視覺互動來說，將會融合到一個共同的領域-互動藝術。過去十年來，各項電腦音樂科技的發展，被廣泛的應用在音樂教育、藝術表演、遊戲、娛樂與體育，也吸引各方面異業專長的人廣泛的投入。最近許多新媒體藝術推陳出新，以互動為主題，不斷的嚐試與提出，創造了無限的創意與與想像的空間。其中，將音樂與視覺做結合與互動，形成多元的藝術表演與遊戲的設計最為醒目。其擴大了電腦音樂的範疇，超越了僅止於耳朵的聽覺，涵蓋了視覺、聽覺與觸覺的整體靈動。一些有趣的創意，可從YouTube 的影片一窺端倪。不會等太久，這種類型的藝術表演或遊戲將會快速的廣為流行，即互動音樂發展為互動新媒體藝術，既可提升育樂或遊戲的娛樂層次，亦可創造新媒體藝術表演的領域與範疇。本次的研究計畫，在延伸我們上一次國科會計畫的成果，即增強Quad-Theremin 的功能，引進視覺互動的連動。我們將擴充前次所研發出的數位四體式Theremin，除了能演奏音樂以外，並能達到影音、圖像或動畫互動控制的目的。此影音控制的涵意必須是，音樂與影像是依表演者或參與者的控制而互動的，是即時的與即興式的，且有其相乘性的效果，以迸發出新的創意。亦有如即時作曲與舞台創作一樣，以Theremin 為主控器，其搭配的軟硬體系統具有即時記譜的功能，將動作或演算法音樂轉成傳統樂譜，以便其他Therermin 或傳統樂器演奏者，能共同參與。在圖像、影片與動畫的操作上，必須能與Theremin 產生有意義的連動。所以，我們的系統有如魔術師的手在空中舞動，即能同時產生音樂與影像的變化。此時藝術創作部分的設計成為一關鍵性的元素，其應用性的設計亦可強化此研究案的創新程度。 我們的系統的設計主軸將放在兩項目標上: 一為以專業的音樂表演，一群Theremin 演奏者，用手去改變銀幕上的圖像，進而產生即興演奏的合奏。集體Theremin 演奏的問題，是不易掌握共同的調性(key)，必須輔以視覺的回授(如及時記譜，或以Theremin 控制圖案的變動以調整音調)，以提高Theremin 合奏的準確性及藉由圖像的互動激發創意性，更可發展隨機音樂的演奏，如以手去一次更動一段樂句的內容。第二種為針對非音樂專業人為對象，以遊戲型的互動設計為目的，於互動遊戲中產生音樂與影像，例如可發展名之為抓金魚的遊戲，影像中魚於三度空間內配合一背景音樂自由游動，此空間與Theremin 所限制的空間重疊，手於三度空間內舞動，去抓金魚，魚需閃避被抓住，手的追逐進而產生隨機音樂的變化，如此影像與音樂的變化產生娛樂效果，亦或可視為一種新的表演藝術。前述二項視覺的設計，將使用Max/Msp/Jitter 與Processing 程式的發展來完成。
The advancement of new computer music technology for the last decade has drawn extensive interest by people and created great impact in music education, Performing art and Gaming. Many new media art performance that had used interactive audio-visual elements created un-limited imagination and innovation which attracts many computer music performers to follow. Those days, more and more people from diverse sides have jumped into this area and created many amazing live performance as they can be seen in YouTube. It is not long to wait before the new type of music performing or acting will be fascinating prevailed. In this research we will extend our previously designed Quad-Theremin and use it as the main apparatus to control (conduct) the audiovisual performance. To reach this goal, we have to enhance the functionality of our Quad-Theremin and the accompanied interactive video (graphic) system. Namely, the Theremin must be able to drive the video or graphic patterns for a kind of new media art presentation while it is playing music by waving one’s hands. Players can interact not just by the music stimulation but also with the video or graphic pattern. This feedback system could create many interesting impromptu pieces. To reach this goal, our designed system must be able to visualize the music score or notation in real-time (real-time notation in the classical music score) and a design of random music method together with a proper video, graphic or animation design. In the end, we will present our creations to open public by two demonstrations to show the effectiveness of our design. First one is a group of Theremin player Ad Lib with a given Rhythm while stimulated by and interacted with the visual pattern on the screen. The second one is the Golden Fish Chasing Game where random music is generated according to the hand’s movement in a confined area as it is chasing a moving fish. Both visual designs adopt the Max/Msp/jitter and the elaborative design of the animation by Processing Language. In our innovative design the visual media can also be extended to android phone as a plus.
