Source code available on GitHub.
The Subconscious of Movement is an exploration into both the rules of motion and the gallery space. It is inspired by Cornelia Parker’s The Subconscious of A Monument (2005) – a piece which fills a room with chunks of clay from beneath the Tower of Pisa suspended in air. The fragile clay almost floats in the air, yet forms a barrier preventing entry into the room and makes impossible any perspective which may be gained from within the room. The inaccessibility of this space was the starting point for The Subconscious of Movement which asks what happens when we are allowed to enter the space which art occupies.
Using a virtual reality system, the viewer is be allowed to enter into the typically forbidden space within the artwork; however, one cannot enter without disturbing the perfect, motionless suspension of the piece. The viewer’s disruption becomes participation and allows the viewer to either creatively or destructively alter the piece.
Taking advantage of the surreal capacities of virtual reality the viewer is given a number of special controls which would ordinarily defy the laws of physics. On jumping onto one of these cubes, the viewer finds that they can stand upon it, allowing for further exploration of perspectives. They also have control over the time of day so that they can see the piece in the lighting of their choosing, be it the stark contrast of midday or the magical golden hour after sunrise. Controlling the very laws of physics, the viewer can change how gravity acts on the particles within the piece. They can activate and deactivate both true gravity, and a local swarm gravity. The first simply turns on the gravity we are accustomed to and causes the particles to fall to the floor. The latter, however, activates a strong local gravity field among the particles, causing them to pull in towards their common center. Finally the viewer has the capacity to alter the viscosity of the room, either leaving the frictionless particles to bounce around endlessly as though they were in a vacuum, or causing them to come to a stop as if the room were filled with glycerin.
The combination of these interactive forces transforms the piece from an impenetrable space into a creative sandbox. The viewer becomes artist as they discover ways in which they can manipulate the cubes to produce their own works. By activating and then quickly deactivating gravity, an immense amount of energy can be added to the room, as the cubes bounce around unhindered. By activating swarm gravity, the particles slowly begin to coalesce, those with low energy clump together into a central mass, some with higher energy form orbits around this mass, while those with very high energy continue to bounce through the room as though unaffected, occasionally colliding with other particles, perhaps even crashing through the central mass and briefly annihilating it. The viewer can at any point turn up the viscosity and turn off the other forces in order to bring the particles to a stop, forming an unpredictable abstract sculpture. With the amount of interaction is this still the same piece that the viewer originally encountered? Is it improved or has it been destroyed?
The viewer may also discover a number of mini-games that form from their interaction. They may choose to create structures and try to jump on top of them, or jump on top of a cube and try to stay on it while it is jostled around the room by the swarm gravity. Alternately they may find it annoying that a number of cubes have bounced out of the room and could try various ways of cleaning up the mess, or perhaps they will try to move the large clump out into the hallway. Considering the apparent simplicity of the piece, the possibilities are staggeringly endless.
This piece goes beyond using virtual reality to break down boundaries between the viewer and the art. As its name suggests, the piece is an exploration of the subconscious of the movement created through simulation. All of the actions that occur are driven by complex processes within the device, constantly calculating each particle’s mass, location, velocity, appearance etc. The viewer can access some basic elements of these processes by activating and deactivating them, or interacting directly with the cubes; however, much is left obscured, as though physical relations form an inaccessible subconscious of the virtual world. This very truly mimics our interaction with the real world, interacting in minute ways with an unfathomable, unattainable subconscious.
The piece was created for Gear VR using Unity with scripts written in C#. Oculus Utilities for Unity was employed as a starting point in order to get the basic camera positions and viewer movement up and running quickly. All of the models were created directly in Unity using primitives in order to streamline the design process. All of the code is either part of the Oculus or Unity utilities package, or was written from scratch with reference to the Unity Manual and Scripting API. Textures as a whole were kept to a minimum, the cubes have a simple canvas bump map applied while the floor texture was obtained from Bricks ‘n’ Tiles.
There were a number of challenges encountered in the development of this piece, many of them a result of the constraints of using the Gear VR, while others were centered around the emulation of physics. Mainly the task was to minimize the processor demand to maintain a reasonable frame rate despite the calculation intensive nature of the work. This considerably limited the number of particles that could be used in the simulation, originally over two thousand on the development computer, only 240 could be reliably simulated on the Gear VR. This is also the reason for using cubes rather than a more complex mesh, as a cube is one of the simplest things to render and simulate collisions with.
The first challenge encountered was lighting. Originally the intent was to have the sun travel through the sky in order to continually alter the lighting in the room; however, recalculating the light paths every frame was far too intensive, and as such, the viewer was given control over the position of the sun. This was extremely beneficial as it only required computation once every few frames while the button was pressed, and gave the viewer more control over the piece. Many more lights were to be used in the hallway and in the main room at night, but this caused a significant reduction in frame rate on the Gear VR and as such these lights were removed and replaced with an increase in ambient light. An alternative solution would have been baking the lights into the textures; however, this was found to produce an undesirable pixelated artifact on the surfaces, and there was not enough time to troubleshoot a number of problems encountered when baking lighting in Unity for use on an Android device.
The next challenge encountered was that of control. If the viewer stood on top of a cube or walked over a number of them on the ground the camera would oscillate rapidly, likely to cause simulator sickness. To avoid this the cubes were made not to interact with the viewer from below – this produced the beneficial ability to stand on the cubes and thus explore the room in three dimensions. Mapping the controls to an Xbox controller linked by Bluetooth was another challenge; however, it was greatly simplified by updating the controller to make it natively compatible with the GearVR.
Perhaps the most interesting challenge was modelling the physics of the swarm gravity. Much of the physics used in this system are built in to Unity, but this aspect had to be designed from the ground up. In effect the problem of gravitation of hundreds of particles is an n-body problem, which becomes very computationally expensive. Instead of directly calculating the force of each particle on each other particle, the simulation was greatly simplified by treating it as a two-body problem:
The system is simplified to two bodies in which one body is the individual cube, and the other the mass of all of the other cubes. This allowed the following simple two body vector field calculation to be used as a basis:
In this simulation, the mass of all particles is equal, so the mass of the larger body can be simplified as the mass of the individual multiplied by the number of particles minus one.
The calculation of the vector r can also be simplified. This is the distance and direction between the two objects, typically calculated with the large object at the origin; however, in order to simplify the calculation we will center the origin at the individual cube to remove the negative from the equation. Once every frame the average center of mass is calculated for the entire array. This average center of mass includes the individual cube we are concerned with, so it must first remove itself from the average, then calculate the vector produced by subtracting one from the other, the resulting calculation is below.
Finally, combining these equations we get the reduced equation specific to this simulation:
Of course, gravity is a relatively week force, and if we simply apply this formula in the space of a room, nothing much will happen – a fact we are familiar with in the apparent lack of gravity between everyday objects. In order to make this observable we must scale up the force to a point where it will be noticeable to the user. Simply by increasing the gravitational constant by a factor of 13, from 6.674×10-11 to 667.4 we can see real-time interactions that look a lot like gravity. This scales gravity to the size of our room and the mass of the particles; however, it is still not quite behaving as desired. The particles near the center of the room interact violently and sometimes are fired off into oblivion like bullets, meanwhile those at the extents of the room are hardly affected. In order to combat this issue, and make the interaction more lively the power of the magnitude of the vector is reduced from 3 to 2. This will simultaneously reduce the violence occuring at the center of the room while increasing the force exerted on particles farther from the center, yet retaining a fairly realistic albeit gravitational model. A comparison of the force of actual gravity, gravity with a modified constant, and the applied “swarm” gravity can be seen below.
The resulting code version of this equation finally implemented in C# looks like this:
GetComponent<Rigidbody>().AddForce(((ArrayCreate.gravityConstant * myMass * myMass) / (ArrayCreate.arraySizeN * ArrayCreate.arraySizeN * Mathf.Pow(Vector3.Magnitude(ArrayCreate.arrayCOM - myCOM),2))) * (ArrayCreate.arrayCOM - myCOM));
Further optimization is possible by pre-calculating common aspects of this equation including mass and total mass, and the program is in fact set up to easily be converted to this mode; however, it was left in this form for experimentation to facilitate quick changes of the formula. Further optimization could be obtained by grouping particles in contact and performing calculations for the group as a whole rather than for the individual constituent particle. These interactions are the highest load on the processor, pulling the frame rate as low as 15 fps when the cubes have gathered tightly into a ball and continue to collide with one another. In addition a performance limit should be implemented to prevent overheating as it currently must shut down if too much is demanded of the processor.
I am very happy with the results of this piece. It is a successful application of both technical skills and artistic theory. It pushes the limits of the technology used while simultaneously exploring the conceptual realm behind that technology. With some more refining it could become more efficient and perhaps it could be made more aesthetically vibrant through added textures and shapes, but as is it is a major leap in my ability to create art in the medium of virtual reality.
 James Stewart, Calculus, 5th ed., (Belmont, CA: Brooks/Cole, 2003), 1095.
Frith Street Gallery, “Cornelia Parker: Subconscious of a Monument,” Frith Street Gallery, accessed February 8, 2017, http://www.frithstreetgallery.com/shows/view/subconscious_of_a_monument.
Oculus VR LLC, “Documentation,” Oculus Developer Center, Accessed February 8, 2017, https://developer3.oculus.com/documentation/.
Unity Technologies, “Unity Scripting API,” Unity Documentation, accessed February 8, 2017,https://docs.unity3d.com/ScriptReference/.
Unity Technologies, “Unity User Manual,” Unity Documentation, accessed February 8, 2017, https://docs.unity3d.com/Manual/.