Anim8or Community

Please login or register.

Login with username, password and session length
Advanced search  

News:

An update to Anim8or, v1.00b, is available with a few bug fixes. Get your copy HERE. See the "ReadMe" file for details.

Author Topic: Anti-Aliasing with Jagged external edge  (Read 3210 times)

wild_qwerty

  • Newbie
  • *
  • Posts: 10
    • View Profile
Anti-Aliasing with Jagged external edge
« on: March 23, 2008, 07:58:08 am »

Is it possible to render a model with anti-aliasing turned on and still maintain a jagged outside edge?

The reason I am asking is that I am using anim8or to model some new sprites up for a game mod I work on. Since we are modding Fallout 2 which is a sprite based game the image need to have a 'hard' or 'jagged' outside edge some that they blend into the game correctly. Normally with anti-aliasing on you get that blurred edge.

Sure the sprites work fine with anti-aliasing turned off, but they dont look quite as good. So it would be nice if I could just anti-alias the inside pixels but keep the edges hard.

Any suggestions?

Does the AA Filter effect this at all?
Logged

Raxx

  • Administrator
  • Hero Member
  • *****
  • Posts: 1450
    • View Profile
Re: Anti-Aliasing with Jagged external edge
« Reply #1 on: March 23, 2008, 08:18:10 am »

The AA filter does not affect this kind of request. It's recommended you use bicubic as your AA filter to get the sharpest results, but it performs AA on the entire render regardless of the filter you choose.

Something you can try:
1. Render your image(s) antialiased (preferrably against a black background), save them to file.
2. Render the same images again except only the alpha channel with AA disabled.
3. Use a 2D app to apply the alpha channels to your antialiased renders. Use any color background you need as well. Now you have your jaggies on the outside but AA on the inside.

I'm guessing the engine converts a single color to the transparency, similar to how gifs work (no semi-transparent pixels--bright pink is usually used). If the engine does use semi-transparent pixels (usually the png format), I'd recommend using the same steps, except rendering both channels as antialiased.
« Last Edit: March 23, 2008, 08:22:42 am by Raxx »
Logged

wild_qwerty

  • Newbie
  • *
  • Posts: 10
    • View Profile
Re: Anti-Aliasing with Jagged external edge
« Reply #2 on: March 23, 2008, 11:18:42 pm »

I've already tried that but it doesnt really seem to work:

Logged

Raxx

  • Administrator
  • Hero Member
  • *****
  • Posts: 1450
    • View Profile
Re: Anti-Aliasing with Jagged external edge
« Reply #3 on: March 23, 2008, 11:44:27 pm »

That's why I said to use a black background, so that there won't be any ugly blue artifacts when cutting the render out using the alpha channel (black blends better in most cases). Not to mention that Fallout itself is a pretty dark game, so it blends even better. Ideally, use any background that blends with the outline of the character/object the best, when rendering. After you applied the alpha channel, you add the blue background to it and you won't have those artifacts anymore.

Don't be afraid to touch up your alpha channel also before you apply it. Higher res images aren't much of a problem, but you're asking a machine to handle some pretty delicate stuff here. You can easily add one black pixel to the character's arm, and take out one or two from the neck and leg. It's still easier than hand-erasing everything around the silhouette from scratch...
Logged

Sirren

  • Newbie
  • *
  • Posts: 1
    • View Profile
Re: Anti-Aliasing with Jagged external edge
« Reply #4 on: March 24, 2008, 11:13:40 am »

Hello there folks, and hallo there Wild_Qwerty.
Well, let's go with some order...
One thing you didn't point out: Fo2 creatures need to cast a shadow, so you need a floor to cast it upon... That floor is rendered with the creature and this means that the alpha cahnnel trick does not work (no, shadows are not rendered when cast on a completely clear foor).
We discussed this topic for a while on the BGE forum. I think a similar trick could work:
Render your animations to BMP with a higer resolution than you need
Batch convert them to a format which supports transparency
Select your background by colour and cut it off
Invert your selection and resize your image.
I only did it a bunch of times to put new trees for my mod and it worked.
Personally I firmly refused to do it for a 600+ animation frames creature (which is roughly the minimum). A sprite usable by the player is some 4,5 K frames, so please evaluate you odds.
By the way, you showed an interesting silohuette here... What kind of character is it?
See you
Logged

Raxx

  • Administrator
  • Hero Member
  • *****
  • Posts: 1450
    • View Profile
Re: Anti-Aliasing with Jagged external edge
« Reply #5 on: March 25, 2008, 06:44:06 am »

Sirren, your solution will make pretty much the same level of quality as the solution I offered, except that you can't perform batch operations with it. Not only that, but depending on what you're using to select by color, it'll still leave artifacts or take out chunks of the character if you try to compensate for those artifacts.

Fallout2 doesn't support semitransparent pixels, correct? Therefore rendering a larger size, cutting the image out and then resizing back down won't work since you'd have to resize back down without any interpolation (no cubic/lanczos/whatever) in order to not have antialiased edges. This would in effect downgrade the quality of the inner details and make it less distinguishable. It probably worked for you because trees don't require specific detailing to look alright when it's low-res, not to mention they're less dynamic.

Simple way to render an alpha channel with shadows is to make your own alpha channel. After you've rendered all the rgb frames, make a copy of the anim8or file(s), and give your objects a stark black material with no specular. Make a new object and add a plane to that, assigning a material with black ambience and white diffuse (value of 2 or 3 for the diffuse). In the scene, set up your single light (to 100% dark raytraced shadows) if it's not already set, add the plane object to it with only receiving shadows, whereas the character object casts and receives, then set the scene background to pure white as well. Render without antialiasing. I've prepared an example for you:

Example file (right click and save link/target as...)

RGB


Alpha:


Combined


There are batch image processing programs out there such as ImageMagick that will allow you to process all the frames at once via some simple scripting, making it a breeze to handle even thousands of frames automatically.
Logged