Anim8or Community

Please login or register.

Login with username, password and session length
Advanced search  

News:

An update to Anim8or, v1.00b, is available with a few bug fixes. Get your copy HERE. See the "ReadMe" file for details.

Pages: [1] 2 3

Author Topic: Multi-Threaded Rendering  (Read 3316 times)

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1662
    • View Profile
Multi-Threaded Rendering
« on: July 08, 2018, 07:08:38 pm »

I just posted a new development release  v1.01.1329 dated July 7, 2018 that uses all of your CPU's cores to render ART images. It also has an improved FastAA algorithm that improves shadow edges, especially for soft shadows, and properly ignores shadows from objects marked to not cast shadows.

Note: There are some minor issues, particularly with noise in movies when the image isn't changing. These are cause by the separate threads calling the random number generator in different a different order for each render. I'm working on fixing this.

Please give this build a try. Let me know of any problems that you have, and how much it speeds up your renders.
« Last Edit: July 09, 2018, 04:04:11 pm by Steve »
Logged

nemyax

  • Full Member
  • ***
  • Posts: 224
    • View Profile
Re: Multi-Threaded Rendering
« Reply #1 on: July 09, 2018, 12:00:38 pm »

Edit | Rotate is disabled for component selections in this build. Is that intentional?
Logged

selden

  • Full Member
  • ***
  • Posts: 185
    • View Profile
    • Modelling for Celestia
Re: Multi-Threaded Rendering
« Reply #2 on: July 09, 2018, 12:06:16 pm »

Steve, thanks!

However, there's a typo in the link above: it has a spurious "[b" at the end. Here's a corrected url:

http://www.anim8or.com/download/preview/files/animcl1329.zip

Logged
Selden

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1662
    • View Profile
Re: Multi-Threaded Rendering
« Reply #3 on: July 09, 2018, 04:28:50 pm »

nemyax: Oops! It looks like Edit | Rotate was disabled in build 13.25 as well. I'll look into it.

selden: Thanks, I fixed it :)
Logged

AlecJames

  • Full Member
  • ***
  • Posts: 161
    • View Profile
Re: Multi-Threaded Rendering
« Reply #4 on: July 11, 2018, 12:24:13 pm »

Hi Steve

I've got Raxx's phur script loading at startup.  when I start 1329 I get

Compiling "C:\Program Files (x86)\anim8or\scripts\PHUR v1.4.a8s":
2531 lines 0 errors
Compiling "C:\Program Files (x86)\anim8or\scripts\PHUR_Comb.a8s":
error on line 47: undefined member reference 'LockAmbiantDiffuse'
error on line 47: incompatible types in assignment
47 lines 2 errors

I'm on Win7 x64.

(This is not something I'm using at the moment its just left over from experiments.  Hope this is useful feedback.)

Thanks for the hard work :)

Logged

AlecJames

  • Full Member
  • ***
  • Posts: 161
    • View Profile
Re: Multi-Threaded Rendering
« Reply #5 on: July 11, 2018, 01:31:35 pm »

Hi Steve

I have a 200 frame 1920 x 1080 simple animation on a green background rendering using ART, anti aliased, xvid codec.
I'm running on an 8 core i7 4800MQ 2.7GHz.

1329 renders in 31 minutes  :(  [processor is running at 96% during render]
1321 renders in 15 minutes!     [ processor is running at 15% during render]

Both tests are running with the PC using about 5GB of 8GB RAM, anim8or is using about 250MB during both tests.  Multithreaded is checked.

Let me know if I've might of set it up incorrectly.

Logged

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1662
    • View Profile
Re: Multi-Threaded Rendering
« Reply #6 on: July 11, 2018, 05:25:17 pm »

AlecJames: I haven't tested on an 8 core CPU yet. It may be using too much cache. Try adding the int attribute ChunkSize with a value of 32 and see if that helps. (Use the menu command Scene->Attributes to add an attribute)

Limiting the number of threads might help, too, but unfortunately the attribute I added to do that is broken.

AlecJames: Oops! I broke it in v1.01.1325. I'll fix it!!!
Logged

AlecJames

  • Full Member
  • ***
  • Posts: 161
    • View Profile
Re: Multi-Threaded Rendering
« Reply #7 on: July 11, 2018, 09:21:30 pm »

I ran some rough tests, ChunkSize vs frames rendered in 1 min:

ChunkSize : Render speed
Not set    5.5 frames per min
16   5 frames per min
32   1.9 frames per min
64   3.5 frames per min
128   7 frames per min
256   12.6 frames per min
384   18.5
448   18
512   21.5 frames per min
640   20
768   19.5
1024   15 frames per min
4096   9.5

Hope it helps :)
Logged

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1662
    • View Profile
Re: Multi-Threaded Rendering
« Reply #8 on: July 11, 2018, 10:00:08 pm »

It looks like the default setting of ChunkSize = 100 is far from optimal for your computer and your scene. This may take a bit of experimenting to figure out how to automatically choose the right parameters.
Logged

Raxx

  • Administrator
  • Hero Member
  • *****
  • Posts: 1461
    • View Profile
Re: Multi-Threaded Rendering
« Reply #9 on: July 13, 2018, 07:37:03 am »

Gave it a shot. The new build is definitely rendering it slower, but the image seems to come out cleaner in 1329 than in 1321.

1321 apparently has the FastAA setting in it as well, so if it was coded in that version then I'm not sure why it's looking cleaner. Is there a setting you programmed in that's increasing render quality and slowing down render times in 1329? The number of rays seems to double in the new version.

Windows 10 Home
Intel i7-6820HK 2.70 GHz (Quad core, 8 threads)

Below are the results.







Also discovered some clear spots in the shadowing for the 1329 renders.



« Last Edit: July 13, 2018, 07:38:55 am by Raxx »
Logged

AlecJames

  • Full Member
  • ***
  • Posts: 161
    • View Profile
Re: Multi-Threaded Rendering
« Reply #10 on: July 13, 2018, 12:30:17 pm »

I don't know my threads from my cores  :-[  :)
Logged

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1662
    • View Profile
Re: Multi-Threaded Rendering
« Reply #11 on: July 13, 2018, 04:00:24 pm »

AlecJames: A core is a complete set of hardware registers, etc. and can run any program. They execute in parallel and independent from each other. So a 4 core CPU can continuously execute 4 programs at the same time. If it is executing more, then all but 4 are paused at any given time, and the OS swaps them in and out as necessary to make it appear that they are all always running.

A thread is a stream of instructions being executed by the CPU, i.e. software. You can run many, many threads at once. Windows normally has dozens and dozens of them active. But at any one moment in time, there can only be as many executing as there are cores.

When Anim8or is rendering with the ART ray tracer, it uses as many threads as there are cores.
Logged

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1662
    • View Profile
Re: Multi-Threaded Rendering
« Reply #12 on: July 13, 2018, 04:48:57 pm »

Raxx: Are you using ambient occlusion? I haven't tried any examples with the new fast AA.

I changed the fast AA heuristics. It should make soft shadows better and faster, among other things. Here's how it works for N samples/pixel:

1. A small number of samples, sqrt(N), is made. If all of the following criteria are satisfied then those are averaged for the final value:

a) all samples are of the same material, including the closest sample in the 4 adjacent pixels,
b) the min Z and max Z differ by less that 5%, including the closest sample in the 4 adjacent pixels,
c) each light is either visible but all samples, or not visible, including the closest sample in the 4 adjacent pixels,
d) the average normal from this pixel and the 4 adjacent pixels are within 30 degrees,

The previous fast AA heuristic used color contrast between the samples and did not use materials or visibility of lights.

You can see what criteria are used with the environment variable int ShowFastAA.

1 - Green is fast AA, black is full evaluation,
2 - Red is large delta Z, gray is fast AA, black is other full evaluation,
3 - Yellow is different lights visible,
4 - Cyan is divergent normals,
5 - Magenta is multiple materials,
6 - Orange is reflective material (currently disabled),
7 - Blue is large color contrast (currently disabled),
8 - Violet is multiple materials, not counting adjacent pixels' nearest samples.

I attached renders of a room with 4 soft shadow local lights that show how this works.
« Last Edit: July 13, 2018, 05:10:58 pm by Steve »
Logged

AlecJames

  • Full Member
  • ***
  • Posts: 161
    • View Profile
Re: Multi-Threaded Rendering
« Reply #13 on: July 13, 2018, 06:32:37 pm »

Steve: When I saw Raxx had said "Quad core, 8 threads", I wondered why because I always thought that I could have 1000s threads running on as many CPU cores that I saw in the task manager performance tab.   Why 8 threads?

I found that CPU threads relates to hyper threading where a single CPU core uses 2 (hyper) threads to create 2 virtual cores. They context switch when one thread is idle waiting on some resource.  I guess you don't get 2x performance but its better that 1x.  (I found a really good description this morning but can't find it now :( )

Since there are virtual cores they appear in task manager as separate cores (so I though I had 8 cores but I've got 4 x 2 hyper-threaded cores).  I must have seen the "hyperthreading" term loads of times over the years and just assumed it was some marketing term :)


Logged

Steve

  • Administrator
  • Hero Member
  • *****
  • Posts: 1662
    • View Profile
Re: Multi-Threaded Rendering
« Reply #14 on: July 13, 2018, 08:28:34 pm »

It is possible to achieve a 8x throughput improvement with a quad code, 8 threaded Intel CPU. Somewhat simplifying, each thread only is allocated every other cycle. So if 1 thread takes 1000 cycles of CPU use and finishes in 2000 cycles clock-time, then 2 threads can run on the same core and finish in 2001 cycles clock-time. So Raxx's CPU could run 8x faster that a single thread render.

There are other resources that limit the performance: memory bandwidth, cache size, etc., so in practice the actual result is less than the theoretical maximum. Changing the "chunk" size in Anim8or changes how memory and the cache are accessed so, depending on the details of the scene, the optimal size can vary greatly.

I'm also working on other ways to limit the memory requirements.
Logged
Pages: [1] 2 3