About Project:
"Sound, Sync, Shade", is an experimental wave synth I made to learn about the integration of real-time audio synthesis and visual feedback. Using p5.js, a JavaScript library for creative coding, and the Web Audio API, for audio processing in web applications, this interactive synthesizer combines audio modulation with dynamic color transitions.
I hope playing around with this interactive sketch is fun and delightful for anyone visiting this page. This is a very early prototype, as I only recently learned how to work with the Web Audio API. The learning experience has been enjoyable; it was great to discover that we can create a custom (virtual) synthesizer using these accessible technologies.
User Guide:
Desktop Users
- Waveform Selection — Click on the buttons to select the waveform type. Alternatively, use keyboard shortcuts:
key 1 = Sine wave
key 2 = Square wave
key 3 = Triangle wave
key 4 = Sawtooth wave.
- Sound Modulation — Click and drag the mouse vertically on the canvas to change the frequency of the waveform, and horizontally to adjust the the delay time and feedback gain.
Mobile Users
- Waveform Selection — Tap on the buttons labeled "Sine", "Square", "Triangle", or "Sawtooth" to select the waveform type.
- Sound Modulation —Swipe vertically on the canvas to change the frequency of the waveform, and horizontally to adjust the delay time and feedback gain.
*Please turn on your volume.
This work is licensed under CC BY-NC-SA 4.0
ครึ่งหลังของสัปดาห์ที่ผ่านมา ผมไม่ค่อยได้ออกไปเดินออกกำลังกายข้างนอกด้วยข้ออ้างเรื่องอากาศไม่ดี เลยใช้เวลาไปกับการหาอะไรเรียนช่วงเย็นถึงค่ำให้ไม่เบื่อ พอดีได้ไปอ่านเจอเกี่ยวกับ Web Audio API โดยบังเอิญ ซึ่งผมเองไม่ได้มีความรู้ทางเทคนิคในเรื่องนี้มาก่อน แต่อาจจะพอมีความเข้าใจเรื่องพื้นฐานการสังเคราะห์เสียงอยู่บ้าง และก็มีความตั้งใจอยากจะทำเครื่องสังเคราะห์เสียง (synthesizer) ในแบบที่ตัวเองอยากทำมานานแล้ว เลยลองหัดเรียนเองผ่านการทดลองสร้างโปรโตไทป์นี้ขึ้นมาครับ ถือเป็นการเรียนรู้แบบเรียนไปทดลองทำไปพร้อมกัน แต่พอได้เล่นเองแล้วก็คิดว่าสนุกดีเลยนำมาแชร์ในบล๊อกนี้เก็บไว้ครับ หวังว่าจะมีประโยชน์บ้างกับท่านที่เข้ามาเยี่ยมชมหรือหลงทางเข้ามาก็ตามในเวบไซต์นี้ครับ
ขอบคุณครับ
2024.06.29
*This program was made using p5.js, a JavaScript library for creative coding, and the Web Audio API, a tool for audio processing in web applications.
Appendix: Key Aspects
- Real-time audio modulation: This sketch generates and modifies audio signals using the Web Audio API. The oscillator's frequency is dynamically updated based on user interactions, such as mouse movements or touch gestures. The gain (volume) and delay effects are also adjusted in real time, providing immediate auditory feedback to the user.
function startOscillator() {
oscillator = audioContext.createOscillator();
oscillator.type = type; // Type of waveform (sine, square, triangle, sawtooth)
oscillator.frequency.setValueAtTime(frequency, audioContext.currentTime); // Frequency of the oscillator
gainNode = audioContext.createGain();
let gainValue = isNoteA(frequency) ? maxGain : minGain; // Adjust gain based on the note
gainNode.gain.setValueAtTime(gainValue, audioContext.currentTime); // Set the gain value
oscillator.connect(gainNode);
gainNode.connect(analyser);
gainNode.connect(delayNode);
gainNode.connect(audioContext.destination);
oscillator.start(); // Start the oscillator
}
Function Parameters:
-
oscillator.type
: The type of waveform (sine, square, triangle, sawtooth).oscillator.frequency
: The frequency of the oscillator.gainNode.gain
: The gain value, is adjusted based on the note.
- Dynamic color transitions: The background color of the canvas transitions based on the current audio frequency. Using
lerpColor
in p5.js, we interpolate between two colors. This transition is tied to the audio input, ensuring that visual changes are synchronized with the auditory experience. The colors are updated gradually, providing a delightful visual effect that matches the sound changes.
function setColorsForType(waveType) {
switch (waveType) {
case 'sine':
targetStartColor = color(sineStartColor);
targetEndColor = color(sineEndColor);
break;
case 'square':
targetStartColor = color(squareStartColor);
targetEndColor = color(squareEndColor);
break;
case 'triangle':
targetStartColor = color(triangleStartColor);
targetEndColor = color(triangleEndColor);
break;
case 'sawtooth':
targetStartColor = color(sawtoothStartColor);
targetEndColor = color(sawtoothEndColor);
break;
}
if (colorTransitionProgress === 1) {
currentStartColor = targetStartColor;
currentEndColor = targetEndColor;
}
}
Function Parameters:
-
waveType
: The type of waveform to set the corresponding colors.targetStartColor
: The starting color for the transition.targetEndColor
: The ending color for the transition.
- Synchronization of audio and visual elements: Synchronization is achieved by mapping audio properties (like frequency) to visual properties (like color). The corresponding color transition is triggered when the user changes the waveform or adjusts the frequency. This is done by setting target colors and gradually transitioning to these targets using linear interpolation, ensuring seamless integration between audio and visual elements. Using logarithms helps manage the wide range of frequencies more effectively, providing a more perceptually uniform color transition. This ensures that changes in frequency result in smoother and more visually appealing color gradients.
function draw() {
let logFreq = Math.log(frequency); // Logarithm of the frequency
let logMinFreq = Math.log(27.50); // Logarithm of A0, the minimum frequency
let logMaxFreq = Math.log(3520.00); // Logarithm of A7, the maximum frequency
let normalizedFreq = map(logFreq, logMinFreq, logMaxFreq, 0, 1); // Normalize the frequency
if (colorTransitionProgress < 1) {
colorTransitionProgress += 0.01; // Update the color transition progress
currentStartColor = lerpColor(currentStartColor, targetStartColor, colorTransitionProgress);
currentEndColor = lerpColor(currentEndColor, targetEndColor, colorTransitionProgress);
}
const hue = lerpColor(currentStartColor, currentEndColor, normalizedFreq); // Interpolated color based on frequency
background(hue); // Set the background color
drawGrid();
drawWaveform();
drawText();
drawCursor();
}
Function Parameters:
-
frequency
: The current frequency of the oscillator.normalizedFreq
: The normalized frequency used for color interpolation.currentStartColor
: The current starting color.currentEndColor
: The current ending color.hue
: The interpolated color is based on the frequency.