Sunday, March 21, 2010

Back To Basics - Krakatoa Best Practices For Fast Iterations.

 This morning I received a PM from a CGTalk member with approximately the following content:

...These days I am studying Krakatoa by myself, it is interesting. Now I can generate particles from FumeFX data. To render Particle Flow, I use 800000 particles, but it is very slow to update PF, takes almost 10 minutes or even more, is it normal? Or did I miss some steps..  My PC is a Dell T5500 workstation, 64bit OS. When PF and Krakatoa work together, how can it be made faster? I can't image using millions of pc would update PF all day long, haha!
For the more advanced users of Krakatoa out there, the following answers might look like deja-vu from the Online Help's Introductory Tutorials and the Studio Daily Introductory Videos posted back in 2007. But I feel it is something that cannot be repeated often enough, and given that nobody ever reads any Help files (I know I don't), a blog post on a site hosted by Google might be more discoverable...

Here is my answer, slightly expanded.

Well, we cannot speed up PFlow, so we generally try to work around these issues. Here are some things one could do to speed up work.

1. Save the particles to PRT sequence once, then render as often as you want.
PFlow is slow at updating to a specific frame because it is history-dependent and has to calculate all preceding frames. So if you have 200 frames of animation and want to render frame 190, you have to wait for all 190 frames to update. But if you have already updated frame 190, getting frame 191 should be relatively fast, and this is true for the saving process too. So you set Krakatoa to save PRTs to disk and let it run (overnight? ). Once it is done, you have traded off time for disk space. Now you have Gigabytes of particle data on disk, but you can render each frame in about a second without any preroll! The Krakatoa PRT Loader also lets you tweak the timing/speed of the playback, cull particles, deform the particles if they don't match your vision, bend the particles to follow a spline path, deform by a skinned mesh to create particles moved by a character animation, create and modify particle data channels using MagmaFlow, duplicate the particles by cloning the PRT Loader multiple times and so on. It is a whole new particle data workflow you have there...

2. Partitioning.
You can expand the above approach with some partitioning (saving out several smaller PRT sequences with varying random seeds to produce a denser final result). So you could for example reduce the particle count to something you can live with time-wise, for example 50K, 100K or 500K particles, and partition only one PRT sequence to check out if you like the motion. You could also use the ability of the Iterative Mode of Krakatoa to render a frame at a fraction of the resolution while preserving the density to get a representative frame from few particles that gives an idea what the same frame with 10 times more particles could look like.

If it looks good after it has finished saving, you can let the computer run multiple additional partitions with the same settings to produce more and more particles using the same FumeFX sim (and/or PFlow setup). This way you don't have to wait for the whole quantity to be processed at once and make more iterations until you like the general motion, then let a copy of Max running in the background save 10 or 20 or 50 sequences of the same setup with different random seeds to combine into one dense sequence with millions of particles within the PRT Loader.

3. Use all your CPUs/Cores.
PFlow is single-threaded, but your machine might have more CPUs sitting idle. You can open several copies of 3ds Max + Krakatoa on your workstation (thankfully, both the 3ds Max and the Krakatoa licenses support that), load the same scene in each one of them and let each one partition a sub-range of the total range of PRT sequences. For example, if you have 4 cores, you can open 4 copies of 3ds Max and let the first one partition from 1 to 3, the next one from 4 to 6, the third one from 7 to 9 and the last one from 10 to 12. After about the same time it would take a single copy of Max to create 3 partitions you will have 12 and all cores will be used. I have done this with 8 cores and it works great, assuming memory is not an issue. Saving particles through Krakatoa does not load particle data into memory, only 3ds Max, FumeFX and PFlow will use memory, so chances are this won't be an issue.

4. Render the FumeFX directly.
Krakatoa lets you render every voxel of a FumeFX simulation as a particle. This does not give you the same look as driving PFlow particles through the simulation grid, but for some effects it can be a lot faster.
The drawbacks: The particles don't have velocity so you cannot apply motion blur and you need a very fine-resolution grid to create enough particles, which of course increases the FumeFX simulation time.

Once you have simulated the FumeFX and you like the look, you can just check the ">FumeFX" button in the Krakatoa Main Controls rollout to enable it as source and render away. But you can also use some techniques to increase the particle density. Save the simulation to a  single PRT sequence, load in a PRT Loader, add a High-Frequency Noise Modifier to the stack as outlined in this tutorial on our site, check the ">PRT Loader Modifiers" option in the Partitioning Rollout and partition the resulting particle system multiple times to increase the density by jittering the particles around their original positions. The result might look a bit noisy though, but it is a possibility.

Finally, you could just render the FumeFX as voxels, which generally requires less particles to produce the same density. Usually you would need to set the Krakatoa Voxel size to the same value as the FumeFX grid size.

These are some of the typical approaches to speed up the iterative process of creating good looking particles and rendering them in Krakatoa. It does not apply solely to FumeFX and Particle Flow - you can combine this knowledge in other situations regardless of the original source of the particles.

Please add your comments below with your own Best Practices. I will migrate this to the FAQ someday.

May your cores be always busy!