Michael Garate

I’m a Musician and Software Engineer. I previously worked at LinkedIn building Production Infrastructure and at Etsy leading the Machine Learning Infrastructure team.

Music Video Fractal Art with Mandelbulb

March 2022

I wanted “Jade” to have a video that was both meditative and electrifying. Fractal animation brought that dream to life. Please enjoy the music video!

The full EP is available on Spotify and other streaming apps. For more about the music, see this post.

This video is possible thanks to the wonderful Mandelbulb3d project. Of all the mandelbulb animation software I tested, mb3d gave the best quality results and a huge amount of controls. This post describes the video creation process.

1. Creating a shape

The shape comes from 5 formulas working together in an alternating fashion. The earlier the function is in this list, the greater its effect on the object. mb3d provides dozens of functions to explore.

1. First we apply sin to the y axis. This creates a smooth infinite repetition of the overall shape. Without this, the object would roughly resemble a sphere or box.
2. Next, we apply MengerKoch to give large scale features of the shape.
3. AmazingBox2 gives a lot of the “feel” of the surface textures and the medium size details. This is what gives it the boxy “space station” vibe.
4. ReciprocalZ3b transforms the Z axis by computing its reciprocal. It adds a lot of variety to the object. Without it, there is more symmetry and closely nested boxes.
5. Sierpinski4 gives a moderate curvature and additional surface detail.

Each of the above functions has 3-8 tunable parameters, and a small adjustment to any one of them can have a dramatic effect. It is a delicate balance to fine-tune a shape without diving into total chaos.

With the shape in place, we can choose an interesting starting position for the camera. This animation shows zooming from the full object view, down into the object, and aiming directly at the y axis to reach the starting keyframe.

Performing a 360 degree rotation from the starting position. If the camera is too close to an object, it can produce stretch marks.

We also apply some extra lighting to give depth. This gif animates turning on and off a custom light. There is also a custom mapping for the background and surface colors.

2. Animation

To make it move, we modulate two fractal parameters over time.

Moving the camera through the shape

Since the sin formula repeats the object on the Y axis and the camera directly faces the Y axis, modulating the sin offset parameter between 0 and π creates the effect of moving the camera through the shape until seamlessly reaching the starting point again. Note that if the camera hits an object, we get blown out artifacts. In the end result, we should avoid this.

Making the object breathe

MengerKoch offers a CScaleY parameter that gradually moves large parts of the object relative to each other. After experimenting with many different params, I chose this one to add some motion to the object itself while the camera flies through. Since this value is not naturally looping like the sin function, the value “breathes” up from value a to b and back down to a. I would have preferred a cyclical, naturally looping value similar to sin, however all of the ones I tried changed the shape too dramatically. Maybe I will have better luck in a future video.

Mandelbulb supports modulation using keyframes. This loop used the following 3 frames.

We can summarize the change between them as follows.

- frame_1:
    sin_y:
      offset_1: 0
    menger_koch:
      cscale_y: 1.05
- frame_2:
    sin_y:
      offset_1: 1.57080
    menger_koch:
      cscale_y: 0.875
- frame_3:
    sin_y:
      offset_1: 3.14159
    menger_koch:
      cscale_y: 1.05

It took about 6 days for my home PC to render 1700 frames at 1440p (with 2x upscaling for antialiasing). Due to the long render times, I tested by exporting low framerate and low resolution versions. As I got closer to the end product I spun up a 64 core windows VM in Azure for a few hours to allow faster iteration. I did not use this for the final render due to cost. Instead I kicked off a render on my Ryzen 5 3600 before going on a road trip to Mendocino for a few days.

Next, I manually touched up a few images that had blown out highlights and sent all of the images through Flowframes to quadruple the framerate using RIFE (NCNN) for interpolation. The result was a nearly 2 minute video at 60fps that could seamlessly loop from start to end.

4. Final Touches

Several effects bring the fractal into a more abstract space.

The Adobe After Effects effect “CC Kaleida” wraps the image geometrically in a Kaleidoscope style. We modulate the size, rotation angle, and opacity of the effect along with the music.

We use “Colorama” to rotate the hue based on the formula (time * 5) % 360, and a custom opacity param allows ramping it up and down following the music.

You may also notice a few layers of snow/dust sprinkled in.

Credits

Thank you to Julius Horsthuis for the video tutorial and inspirational portfolio. Really beautiful artwork.

---

Contact

• Email
• LinkedIn
• GitHub
• Mastodon
• Resume