Attention: you're visiting an archived version of this post on my old website. Click here to view this same post on my new website.

Seeing shades of the future

Other than fixes to broken features and other small things, there are two huge features I see in the future of Flash: hardware-accelerated 3D support, and pixel shader support. This dream became closer to reality yesterday, as Adobe announced some of the new features of Flash Player 10 (codenamed “Astro”) at Adobe Max 2007 (see a video here).

What caught my attention – other than the “real” 3d for planes – is the fact that now they’ll have their own language for pixel shaders.

Codenamed Hydra, and apparently created by these guys, the language is the basis of Adobe Image Foundation, Adobe’s new image processing platform, to be supported by their whole line of products. With its C-like syntax, it’s also analogous to other shader languages, such as OpenGL’s GLSL and DirectX’s HLSL, so shader programmers will feel right at home.

However, what’s cool about it is that Hydra will allow developers to write their own shaders and execute them inside the Flash Player, as they were executing normal filters such as Blur and Glow. By running Hydra code at the lowest level possible, developers will be able to use the available computer power at its best, so this opens a huge new era for graphic effects inside Flash movies, and one that has already been known for many people specially on the game development industry.

The big question, however, is whether there will be hardware acceleration on Hydra shaders. The presentation mentioned running those shaders with “the performance of native filters that are already in the Flash Player”, with no mention of GPU support for either; the LLVM documentation mentions Hydra can run both at CPU and GPU level, and while the AIF toolkit can only be executed on computers with a recent GPU (Pixel Shader 3 is required), it doesn’t mention whether specific products such as Astro will make use of the GPU acceleration path or not.

Regardless, even if it only means software support, a whole new field will open for Flash developers. Several people have already been experimenting with the toolkit, so I saved some time to test it out and port some of the shaders and C++ code I had been creating for some college work. Here’s a sample, before and after the shader is applied:

This employs an adjustable threshold – unlike a straight threshold filter, you can set the black and white threshold, and everything in-between those two values is featured as a grayscale point – that ends up allowing for a kind of anti-aliased threshold, or a very strong levels adjustement.

Here’s the source in Hydra:

kernel AdjustableThreshold {
    parameter float blackThreshold < 
        minValue:float(0);
        maxValue:float(1);
        defaultValue:float(0.4);
    >;

    parameter float whiteThreshold < 
        minValue:float(0);
        maxValue:float(1);
        defaultValue:float(0.5);
    >;

    void evaluatePixel(in image4 src, out pixel4 dst) {
        float4 inputColor = sampleNearest(src, outCoord());
        float brightness = (inputColor.r + inputColor.g + inputColor.b) / 3.0;
        
        dst = inputColor;

        if (brightness < blackThreshold) {
            // Below threshold
            dst.r = dst.g = dst.b = 0.0;
        } else if (brightness > whiteThreshold) {
            // Above threshold
            dst.r = dst.g = dst.b = 1.0;
        } else {
            // Between the threshold
            dst.r = dst.g = dst.b = (brightness - blackThreshold) / (whiteThreshold - blackThreshold);
        }

    }
}

The black threshold must always be lower than the white threshold.

Astro’s support for Hydra has some limitations, however, so it can’t have IF statements or loops. Some of it can be worked around with inline selections, so the code below do the same thing, but it seems to be Astro-compatible:

kernel AdjustableThreshold {
    parameter float blackThreshold < 
        minValue:float(0);
        maxValue:float(1);
        defaultValue:float(0.4);
    >;

    parameter float whiteThreshold < 
        minValue:float(0);
        maxValue:float(1);
        defaultValue:float(0.5);
    >;

    void evaluatePixel(in image4 src, out pixel4 dst) {
        float4 inputColor = sampleNearest(src, outCoord());
        float brightness = (inputColor.r + inputColor.g + inputColor.b) / 3.0;
        
        dst = inputColor;
        
        float grayscale = (brightness - blackThreshold) / (whiteThreshold - blackThreshold);
        float ifAboveBlack = brightness > whiteThreshold ? 1.0 : grayscale;
        dst.r = dst.g = dst.b = brightness < blackThreshold ? 0.0 : ifAboveBlack;
    }
}

Maybe this is not the best example of AIF's new capabilities, as it's not just about color manipulation, and there's a bit of unneeded overhead on how the colors are calculated (should use a lookup table instead of calculating the color for every pixel). However, what's cool about this is that I simply ported my code directly from a quick HLSL test I had, and it works pretty much the same, with just a few adjustments needed for syntax (and working around the IF/loop issue, which I admit was a let down).

Anyhow, max will have an AIF-specific presentation tomorrow (wednesday). Hopefully by then we'll be able to answer, with 100% certainty, whether Astro will use GPU-accelerated hydra processing or not.

Posted on 2/Oct/2007 at 10:41 am | Comments (0) | Trackbacks (0) | Categories: AIF/Hydra, Flash, Misc, Sources