This tutorial is centered in the techniques needed to create new boxes and mix Java and Zaluum. The classes created are for educational purposes only, they are not optimized nor are feature complete. The Zaluum standard library aims to provide some of the functionality presented here in a ready-to-use form. Please share your thoughts on how the standard library should be.
We will create a simple application to play and visualize sound waveforms. The techniques described can be applied to other real time data situations.
package org.zaluum.tutorial.sound;
import java.io.File;
import javax.sound.sampled.*;
import org.zaluum.annotation.*;
// We use the @Box annotation to make this class show up in the palette.
@Box
public class WavSoundInput {
// The sound format properties
static final int channels = 2;
static final int bits = 16;
static final float sampleRate = 44100.0f;
static final AudioFormat format = new AudioFormat(sampleRate, bits,
channels, true, false);
private AudioInputStream audioInputStream;
private final String file;
// A helper buffer to read the 4 bytes of the frame. One short for each channel.
private final byte[] buffer = new byte[format.getFrameSize()];
// We will read chunks of 1024 samples.
public static int CHUNKSIZE = 1024;
private double[] chunk = new double[WavSoundInput.CHUNKSIZE];
// The constructor accepts a file name to open
public WavSoundInput(String file) throws Exception {
this.file = file;
open();
}
public void open() throws Exception {
AudioInputStream ais = AudioSystem.getAudioInputStream(new File(file));
audioInputStream = AudioSystem.getAudioInputStream(format, ais);
}
// Notice we use the @Apply annotation to guide Zaluum on which methods
// are candidates to be used as box behaviour
@Apply
public double[] chunk() {
for (int i = 0; i < chunk.length; i++)
chunk[i] = read();
return chunk;
}
// We read the data a frame at a time in an endless loop.
// The output data is the first channel scaled to 0 to 1.
private double read() {
try {
int read = audioInputStream.read(buffer);
if (read == -1) {
audioInputStream.close();
open();
return 0;
} else {
short sample = (short) ((((int) buffer[1] & 0xff) << 8)
+ ((int) buffer[0] & 0xff));
return (sample) / 32767.0;
}
} catch (Exception e) {
return 0;
}
}
}
Java provides a basic sound API that allows us to load and play sounds in almost any machine. The precise hardware and format support depends on the vendor and version of your JVM. Usual JVMs, such as Oracle's or OpenJDK can provide 16 bit PCM 44100Hz sound output, and read WAV files.
This class creates a simple box that reads a WAV file and give us the values of the waveform in chunks.
The purpose of this example is to show the flexibility that provides the Java interop. That's all it's needed to create a useful box.
As you can see, it's a plain Java class. The only clues that
indicate that it's used as a box, are the
@Box
and
@Apply
annotations. These annotations are only present to put an entry
in the palette. You could perfectly skip the annotations and
create a Box Instance with
#Class: org.zaluum.tutorial.sound.WavSoundInput
and
#Method: chunk|0
to achieve the same result.
Copy paste the following code that interfaces with the sound card:
package org.zaluum.tutorial.sound;
import javax.sound.sampled.*;
import org.zaluum.annotation.*;
@Box
public class SoundOutput {
private final SourceDataLine sndOut;
private final byte[] buffer;
private int bufferCount = 0;
public SoundOutput() throws LineUnavailableException {
sndOut = AudioSystem.getSourceDataLine(WavSoundInput.format);
int bufferSize = 4096;
sndOut.open(WavSoundInput.format,bufferSize);
buffer = new byte[sndOut.getBufferSize()];
sndOut.start();
}
@Apply public void playSamples(double[] in ) {
for (double d : in) {
playSample(d);
}
}
private void playSample(double in) {
int s = (int)(32767.0*Math.min(1.0,Math.max(-1.0, in)));
byte msb = (byte) (s >>> 8);
byte lsb = (byte) s;
buffer[bufferCount++]=lsb;
buffer[bufferCount++]=msb;
buffer[bufferCount++]=lsb;
buffer[bufferCount++]=msb;
if (bufferCount>=buffer.length) {
bufferCount=0;
int offset = 0;
while (offset<buffer.length){
offset+=sndOut.write(buffer, offset, buffer.length - offset);
}
}
}
}
Now let's actually use the boxes we have just created. Create a PlaySound.zaluum in the same package as the other classes and make it look like this:
It's a While box with a Literal true attached to the continue condition. Inside there is an instance of WavSoundInput and another of SoundOutput drag and dropped from the palette.
The WavSoundInput needs a constructor parameter. If you try to save the file now, you will see the error show up telling you that the constructor hasn't been found. You can double-click the error to go to the source of the error.
With the WavSoundInput box selected, go to the field
#Constructor
in the Properties view, select the constructor and type
long.wav
as the file name.
To demonstrate how to create a visual widget with Zaluum, we will build a very simple oscilloscope view that paints arrays of doubles.
All the Zaluum widgets are Swing classes, so any class extending
java.awt.Component
can be used visually. The creation of new widgets from plain Swing
is intended for power users and it is not a required skill for the
typical user of Zaluum.
Copy&paste the following code:
package org.zaluum.tutorial.sound;
import java.awt.Color;
import java.awt.Graphics;
import javax.swing.JComponent;
import javax.swing.border.LineBorder;
import org.zaluum.annotation.Apply;
import org.zaluum.annotation.Box;
@Box
public class SimpleOscilloscope extends JComponent {
private static final long serialVersionUID = 1L;
// Mark it as volatile since it will be accessed by different threads.
private volatile double[] data;
private double yzoom = 1;
private boolean centery;
public SimpleOscilloscope(){
super();
setBackground(Color.white);
setBorder(new LineBorder(Color.black));
}
@Apply
public void apply(double[] data) {
this.data = data;
repaint(); // ask Swing to repaint when she can.
}
// This paint method will be executed in the Swing thread.
@Override
public void paintComponent(Graphics g) {
double[] hold = data; // We "sample-and-hold" the data to avoid locking.
g.setColor(getBackground());
g.fillRect(0, 0, getWidth(), getHeight());
g.setColor(getForeground());
if (hold != null) {
double dx = ((double) getWidth()) / hold.length;
int midy = centery ? getHeight() / 2 : 0;
double before = hold[0] * yzoom;
double x = 0;
for (int i = 1; i < hold.length; i++) {
double now = hold[i] * yzoom;
double newX = x + dx;
g.drawLine((int) x, (int) (midy + before), (int) newX,
(int) (midy + now));
before = now;
x = newX;
}
}
}
// getters and setters
public void setYZoom(double yzoom) {
this.yzoom = yzoom;
}
public double getYZoom() {
return yzoom;
}
public void setCenterY(boolean centery) {
this.centery = centery;
}
public boolean isCenterY() {
return centery;
}
}
JComponent which is a java.awt.Component
it is all we need to make a new Widget
We have to take into account the usual threading issues of
Swing applications. The GUI widgets will be executed in a
Swing thread, and the main program in some other. In this
case, when data arrives, to the
apply
method we simply store the array reference to be painted to a
volatile
and ask for a repaint. The
volatile
keyword ensures that any other thread will see the last value
stored. We could have solved with
synchronized
, locks or atomic references. The point is that the data
comming from apply has to be properly handed to the Swing
thread. There is plenty of literature available covering how
to write thread-safe Swing components that you can
immediately apply to Zaluum.
The repaint call posts a request to the Swing thread to
execute the paint routines. When Swing sees fit, it will call
paint and eventually the overriden
paintComponent
.
The
paintComponent
method stores a local reference to
data
so the field can continue to be updated from the other thread
without locking for the entire duration of the paint
operation.
For painting we scale the x axis to the pixels available and paint a line between every consecutive point.
We also want to implement a couple of features for demonstration purposes: we should be able to select if the 0 of the y axis is centered vertically or it is on the top of the screen, and we want to choose the amount of zoom we want in the y axis. They could be passed as parameters in the apply method, or in another apply method with parameters data, yzoom and ycenter but we will implement them as JavaBeans, so they are shown as properties of the box.
true as the value and hit enter. Now our data will
be centered in the view
100 in the zoomy
property.
package org.zaluum.tutorial.sound;
import static java.lang.Math.PI;
import org.zaluum.annotation.Box;
import org.zaluum.annotation.StaticBox;
@StaticBox
public class SoundMath {
public static double[] hamming = hammingWindow(WavSoundInput.CHUNKSIZE);
public static double hammingsum = 0;
public static double[] hammingWindow(int order) {
double[] w = new double[order];
for (int i = 0; i < order; i++) {
w[i] = 0.54 - 0.46 * Math.cos((2.0 * PI * i) / (order - 1));
}
return w;
}
@Box
public static double[] multiply(double[] a, double[] b) {
double[] res = new double[a.length];
for (int i = 0; i < a.length; i++) {
res[i] = a[i] * b[i];
}
return res;
}
@Box
public static double[] multiply(double[] a, double b) {
double[] res = new double[a.length];
for (int i = 0; i < a.length; i++) {
res[i] = a[i] * b;
}
return res;
}
@Box
public static double[] sum(double[] a, double[] b) {
double[] res = new double[a.length];
for (int i = 0; i < a.length; i++) {
res[i] = a[i] + b[i];
}
return res;
}
@Box
public static double[] powerLog(double[] input) {
double[] result = new double[input.length/2];
for (int i = 0; i < result.length; i++) {
double abs = 2*Math.sqrt(input[i * 2] * input[i * 2]
+ input[i * 2 + 1] * input[i * 2 + 1]);
double scaled = abs / result.length; // not counting the window.
result[i] = -20 * Math.log(scaled);
}
return result;
}
}
@StaticBox
so it's code get's scanned by the palette and mark as
@Box
the methods we want to show. We have code to multiply and sum
arrays, Hamming window function and some code to extract the
power spectral density of the FFT's results. The Hamming window
is used to "smooth" the edges of the data feeded to the FFT to
minimize some artifacts with non-periodic signals. The power
spectrum is a way to paint the results of the FFT in a
logarithmic format.
Download jtransforms-2.4.jar (source page) and save it in your project's root directory. Then add it to the classpath: Right click the jar and go to Build Path→Add to Build Path. JTransforms is a good open source implementation of the FFT for Java made by Piotr Wendykier.
package org.zaluum.tutorial.sound;
import org.zaluum.annotation.Apply;
import org.zaluum.annotation.Box;
import edu.emory.mathcs.jtransforms.fft.DoubleFFT_1D;
@Box
public class FFT {
private DoubleFFT_1D fft = new DoubleFFT_1D(WavSoundInput.CHUNKSIZE);
@Apply
public double[] apply(double[] params){
double[] output = new double[params.length*2];
System.arraycopy(params, 0, output, 0, params.length);
fft.realForward(output);
double[] half = new double[params.length];
System.arraycopy(output, 0, half,0, half.length);
return half;
}
}
signal and power are ports.
Create an input port from the palette <ports>,
double click it to rename it to signal. Do the
same with power but as an output port.
org.zaluum.tutorial.sound.SoundMath
in the #Class property and hamming into
#Field. The left connection updates the field if
connected and the right one takes its value.
Back in the ViewSound.zaluum connect the new box like this:
Place the new Oscilloscope in the GUI view and set the ycenter
property to
false
, to have the 0dB at the top of the plot and the yzoom
to
10
.
Add this code:
package org.zaluum.tutorial.sound;
import org.apache.commons.math.util.FastMath;
import org.zaluum.annotation.Apply;
import org.zaluum.annotation.Box;
@Box
public class Generator {
double delta = 1/44100f;
double t=0;
@Apply
public double[] sin(double f) {
double[] result = new double[WavSoundInput.CHUNKSIZE];
for (int i=0; i<result.length; i++) {
result[i] = FastMath.sin(t*2*Math.PI*f) * 0.3;
t+=delta;
}
return result;
}
@Apply
public double[] sawtooth(double f) {
double[] result = new double[WavSoundInput.CHUNKSIZE];
for (int i=0; i<result.length; i++) {
double x = t*f;
if (f==0)
result[i]=0;
result[i] = 2*(x - Math.floor(x+(0.5))) * 0.3;
t+=delta;
}
return result;
}
}
Modify ViewSound.zaluum to look like this:
org.zaluum.op.Literalorg.zaluum.widget.ZSliderWidgetPlace the sliders in the GUI view, and add some labels if you wish:
