(C) 1998-2000 McNabb Software Arts

McNabb Software Arts
San Francisco, CA
http://www.mcnabb.com
info@mcnabb.com


Contents

I. Overview
 
II. Fantasia Objects
1. The Object Palette
1a. Controls
1b. Processors
1c. Modules and Module Groups
1d. Systems
1e. Server Objects
2. Projects
3. Editing Objects
3a. Selecting and Resizing
3b. Locking and Unlocking
3c. Edit Menus and the Property Inspector
 
III. Running Fantasia
 
IV. Extending Fantasia
1. Creating MIDIProcessor subclasses
2. Adding Controls

I. Overview

Fantasia is a real-time MIDI and audio processing application. It functions as an object oriented user interface client for the MIDI Kit (MK) running as a server in the background. All actual processing is done by the server at a high priority, while Fantasia displays interactive views of the server's objects, with sliders, knobs, text fields, etc. Any number of instances of Fantasia may connect to a single server instance, on the same or another computer, allowing interesting collaborative or performance interaction situations. For example, input from one performer could be processed and accompaniment material displayed to other performers on their laptops. Or, a musical education game could be devised that includes interaction between students using their own computers, wherever they might be.

Fantasia is based on the Multiple Document Interface Kit (MDIKit) from McNabb Software Arts, and the Java™ Foundation Classes, from Sun Microsystems. By itself, it is a pure Java application and therefore may be run on any computer which supports Java 2 Standard Edition version 1.4 or later.

The MIDI Kit is installed separately, and is available for free from McNabb Software Arts under the GNU Library General Public License. The MIDI and audio I/O streams in the MK call custom C functions in the native MIDI and audio libraries and are therefore operating system dependent. Currently the only MK implementations are for IRIX and Windows 95/98/NT. However, a Linux port will be available soon. The MK is constructed in an efficient multithreaded architecture which avoids method synchronization and garbage collection. It is quite fast enough for handling MIDI processing tasks in real time with insignificant throughput latency. Audio processing is also fast and reasonably low-latency (although this is still being improved). It takes advantage of a portable library of vector operations implemented in C. More extensive documentation on the MK is provided with that package (doc/Concepts.html).


II. Fantasia Objects

1. The Object Palette

Building processing networks in Fantasia consists of dragging various kinds of objects from a palette into a window, and binding their connection points with dragged lines, similarly to other types of network-style builders.

There are several types of objects available from the palette: Controls, MIDI Processors, Audio Processors, Modules, Systems, and Server objects.

2a. Controls

Controls are Java Bean - based user interface objects such as buttons, sliders, knobs, text fields, etc. The controls palette also includes icons for blank System windows and blank Modules (see below).

2b. Processors

MIDI Processors and Audio Processors represent all the types of MIDIProcessor and AudioProcessor subclasses available on the server. Dragging and dropping one of these into a System window (see below) instantiates an instance of that object on the server. The interface that you see is automatically generated from information returned by the server as to the number, names, and types of the object's variables. It is a "raw" processor interface which you may use directly or customize into a Module (see below).

There are several MIDI and Audio variables that are standard for most processors. These are considered to be reserved for "input" or "output" of MIDI and audio data. For other variables, there is no inherent distinction between "input" and "output". Connected variables should normally be thought of as a two-way binding. If one is changed, it changes all other variables or GUI controls to which it is connected either directly or indirectly. It is up to the design and function of each object to determine the flow of data. Some objects "push" data from their variables, and other objects "pull" data. Real-time updates of GUI controls from the server are handled efficiently, but should be used sparingly to avoid processing delays, especially for MIDI event type variables or array views.

MIDI type variables are drawn as little MIDI sockets, while numeric variables are drawn as simple round sockets. Text-type variables have a small "T" inside the socket, and audio variables have a small ~ (sine wave) in their socket.

To make a connection between variables or a variable and a control, move the mouse over the round "socket" icon next to a Processor's variable name until you see the crusher cursor. Then drag a line from there to the desired destination, either a control of some kind or another variable's socket. Since Controls (e.g. a text field or button) respond to mouse events themselves, it is necessary to hold down the SHIFT key when initiating a connection from a Control.

After you are satisfied with the connections within a Processor or Module, you can hide the connection lines and/or images by using the right-button menu for a single variable or the Processor as a whole.

Processor instances have globally unique identifiers which are saved with your Project. Therefore multiple references to the same Processor instance in different Systems, Projects or Fantasia clients will always refer to the same instance. (See Server Objects below).

2c. Modules and Module Groups

A Module consists of a network or collection of any number of Processors and controls, saved as a unit. To save a module, first group the desired objects (if there are more than one) by selecting them and using the "Group" menu option, then CTRL-Drag ("clone") the group and drop it into the Module area of the Palette. New independent instantiations of the module may then be created by dragging them out of the Palette, just like any other object. Note that for now modules are stored in the "modules" subdirectory of the Fantasia top-level directory.

If you have a combination of Controls and Processors that you use frequently and/or want to be able to replicate easily, consider saving them as a Module. In this respect Fantasia is its own interface builder.

A Processor is actually a Module as well. After dropping in a "raw" Processor, you can resize it and add other Processors inside it, creating and automatic Module Group.

2d. Systems

A System is a network or collection of any number of Processors, Controls, or Modules in a window. These groupings are normally saved with the project. However, generic versions can be saved just like Modules, by CTRL-Dragging the System into the System area of the Palette. Replicants of the saved Systems are created by dragging the System's icon out of the Palette into the background area. These replicated Systems are completely independent from the original System (that is, their Processor instances are newly created on the server). Creating a System is appropriate when the grouping is so large that it deserves its own window, as opposed to a Module which you would drag into an existing window. Note that for now Systems are stored in the "systems" subdirectory of the Fantasia top-level directory.

2e. Server Objects

The Server tab of the Palette shows a list of all Processors that are currently running on the server. Note that it is possible to have any number of Fantasia interface clients connected to one server. Therefore some of the objects may not be part of your Project. However, since all processor instances have unique-for-all-time names, you can drag one of someone else's Processors into your project, in order to receive or send data to the other person's Project. From then on, any time the two Projects are connected to the same server, they are guaranteed to share the exact same Processor instance.


2. Projects

When working in Fantasia, one works on a single fantasia Project (.fsa) file at a time. The Project includes all objects and windows, and is saved in a single file, which is actually a zip format file. Objects are archived as XML-style property lists in plain text files. Therefore, one can unzip a project file and look at or even edit the archives in a text editor.


3. Editing Objects

3a. Selecting and Resizing

There is a selection area several pixels wide around every component. When the mouse is moved into this area, after a moment a dashed rectangle appears. Clicking in this area selects the object for cutting, deletion, grouping, property inspection, etc. Shift-click to select additional objects. Dragging in this selection area will move or resize the object depending on the position of the cursor relative to the object (the cursor will change to indicate the operation. Alternatively, objects may be selected by clicking in the background of a System window and dragging out a selection rectangle.

3b. Locking and Unlocking

An Processor or Module may be "locked". You cannot move or resize objects within a locked object. To lock or unlock an object, use the right-button menu, or the top level menu if the object is selected.

3c. Edit Menus and The Property Inspector

The right mouse button menu presents a menu of actions specific to the object the mouse is over. The Property Inspector may also be used to modify all public properties of an object. To use the Property Inspector, first open it using the Tools->Property Inspector menu item in the top level menu. Then select the item you wish to inspect. Alternatively, you may use the right-button menu for an object in most cases.


III. Running Fantasia

Fantasia and the MK should be installed in parallel under the same directory tree, e.g. xx/midikit/midikit-19 and xx/fantasia/fantasia-17. There are executable files for both the client and server for both IRIX and Windows. These executables read a property file called fantasia.unix, fantasia.win, midiserver.unix, or midiserver.win, depending on the app and the OS, to get the Java class path. You may need to edit these files to set the location of your Java and MIDI Kit home directories.

To run Fantasia communicating with a midikit server automatically launched on the local machine, simply run "fantasia" or "fantasia.exe".

To run the client and server on separate machines, run "midiserver" in the MK project, then "fantasia" on the client machine. The host is specified using either the "-h hostname" command line argument or the "midikit.host=hostname" property in the fantasia configuration file. Fantasia will still need to locate the midikit.jar file (one way is to copy it into the same directory as Fantasia). Again, note that the client may be run on any computer which has at least Java 2 Standard Edition v1.4 or later installed, and more than one client can be connected to the same server simultaneously.

You will notice that the default color scheme for Fantasia is light-on-dark. This is because its anticipated function is live performance, where light details on a dark background is more visible to the performer and less obtrusive to the audience, as in the case of most electronic music hardware.


IV. Extending Fantasia

1. Creating MIDIProcessor subclasses

Fantasia is not really intended to be a visual programming language, such as Opcode's Max. Rather, it is based on the idea that the creation of functional modules is more easily and efficiently done using a highly-developed conventional programming language, in this case, the Java platform. The MIDI Kit and Fantasia provide an excellent framework for quickly programming any desired MIDI or control processing object, bringing up an auto-generated and self-documenting interface, and customizing that interface to any degree.

The MIDI Kit is event-driven. Computation in an object only takes place when an object receives an event or is requested to update its variables by another object. These events may be MIDI input, an update from the user interface, or a timed event placed with a Conductor earlier by the Processor or other object. In the case of a MIDIEvent, the processMIDIEvent() method is called. The default version of this method calls various methods like noteOn() or controlChange() depending on the type of event. In the case of variable update requests, the updateVar() method is called with the desired variable as the argument. That variable's value should then be brought up to date by calling var.setValue(). This will in turn update the values of all other variables to which that variable is bound. When those variables are updated, they then call their owner's varUpdated() method and the owner my make any calculations or generated any events as necessary.

A special class of MIDIProcessor is called Performer. This object is designed to make it easy to implement objects whose primary function is to generate events or make computations on a periodic basis. When the start method is called, the willPerform() method is called. If this method returns true, then the perform() method will be called at intervals determined by the nextPerform instance variable (which can be changed at any time). All Performers initially receive their timing from a persistent object called the Default Conductor, although that may be changed by making a connection from the Performer's conductor variable to any other Conductor.

Please see the MIDI Kit documentation for more information, and the Java source code in com/msa/midi/processors for examples.

2. Adding Controls

You can also add custom GUI controls to the user interface's Controls palette. Simply add their fully-qualified class names to the list labeled "controlsItems" in the Fantasia.properties file. Fantasia tries to be smart about which property is connected when you make a connection graphically. The general order of priority is to first use the Java Bean "default property" (set in the BeanInfo class), then float or String type properties named "value", "floatValue", "stringValue", etc, with preference given to properties with the same type as the Processor variable or control to which you are connecting.


Java and all Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries