Special Shadertoy features

Mouse

(example here , utils here, many values there, ref shader here )

• iMouse.xy: last mouse click or current mouse drag position.
  Attention: these are integers, while pixels are between integers.
• iMouse.zw:
  • >0: ongoing drag; starting drag position.
  • otherwise abs() contains the last drag-start position anyway.
• mouse button events ( added in Nov, 2020 ):
  • down even (i.e. just clicked) if  iMouse.z > 0. && iMouse.w > 0.
    
  • currently down: if  iMouse.z > 0. ( && iMouse.w < 0.)
  • released: if  Mouse.z < 0 && iMouse.w < 0.
  • up event: you have to track for 1st release
  • demo: ref here and  here, use here. 
  • Double-click , up event, and more: here.

Keyboard

( example here, utils here and there , updated ref shader here )

• special texture keyboard: size 256 x 3 ;   values in .x field.
  • (ascii+.5)/256, 0.5/3. : 1 if key #ascii is currently pressed (keydown) otherwise 0
  • (ascii+.5)/256, 1.5/3. : 1 at the moment key is pressed (keypressed) otherwise 0
  • (ascii+.5)/256, 2.5/3. : 1/0 toggle for key #ascii

Note that only 2 states were first implemented; keypressed was added at an undetermined time. But compatibility should be safe if only non-zeroness or > .5 was tested rather than value 1.

A reminder that world-while people don’t have Qwerty keyboards: for games and exploration, please use arrows (possibly in addition to your usual controls).
FYI:  LEFT:37  UP:38  RIGHT:39  DOWN:40
PAGEUP:33  PAGEDOWN:34  END : 35  HOME: 36
Modifiers:  SHIFT: 16  CTRL: 17  ALT: 18

Time

( many values there , ref shader: iTime, iTimeDelta  )

• float iTime (or iGlobalTime ) : seconds(+fracs) since the shader (re)started.
• vec4 iDate: year-1, month-1, day, seconds(+fracs) since midnight.
• int iFrame: frames since the shader (re)started.
• float iTimeDelta: duration since the previous frame.
• float iFrameRate: average FPS.
• float iChannelTime[4] : current time in video or sound.

Resolution

Test current values here.

• vec3 iResolution: for window and buffers.
• vec3 iChannelResolution[4] : for texture input  of any kind.

•  .x,y is width,height.
  • Don’t expect the ratio to be the same on all computers (or even to have width>height, e.g. on smartphones and tables). Indeed it can even change between icon, working view and fullscreen.
  • Don’t expect videos (and textures) to have the same ratio than your window: texture(fragCoord/iResolution.xy) wrap the full image, meaning possible distortions.
  • A reminder that a distortion-free scaling means scaling by a scalar, not a vector. e.g. fragCoord/iResolution.y , not .xy. We often like centered coordinates, with y in range [-1,1]:
    vec2 uv = (2.*fragCoord-iResolution.xy ) / iResolution.y ;
• .z is the ratio of the pixel shapes. For now it seems to always be 1.

For buffers relying on persistence, they won’t keep valid if resolution changes while running: consider testing change, or a magic key to reinitialize. Similarly if you want to allow fullscreen, let 3″ at start for the user can switch ( if (iFrame<200)… )

Buffers A..D

Instead of being displayed (as for buffer “image”), the result is stored in the special texture of same name.

• At each frame they are evaluated in A..D order, then image.
• For persistent or incremental effects, make your buffer read the same texture it produces (i.e. previous state vs next state). It seems to be initialized to 0.
• To use persistence, you might compute something only at frame 0 : if (iFrame==0)…
  Still, if it relies on image textures, they need some time to be asynchronously loaded so consider something like if (iFrame<30) { init } else { fragColor = texture(sameBuffer); }
  But this is naïve not always sufficient: see better solution here.
• Note that these textures are floats, not bytes, so not bounded to [0,1]. Itt seems to be full ordinary 32 bits floats on most systems.
• If you want to use the texture as an array, a reminder that fragCoords are mid-integers → consider fragCoord-.5 to get the integer index, or (vec(i,j)+.5)/iResolution.xy to get the texture coordinate.
  Or use texelFetch( ch, ivec2(fragCoord) , 0 ).
• Attention: MIPmap of buffers is off by default. See flags at texture binding as channelN.

CubeMapA

Similar to BuffA, with the following specificities:

• Result is a cube map, so you indeed render the 6 faces of a cube.
  So instead of a main(out vec4 fragColor, in vec2 fragCoord), you have a more complicated mainCubemap( out vec4 fragColor, in vec2 fragCoord, in vec3 fragRayOri, in vec3 fragRayDir ) (Note that it is similar to the alternate mainVR() ).
  fragCoord and iResolution are relative to each face,
  fragRayOri is vec3(0) ( = cube center )
  fragRayDir is the ray direction hitting the cubemap texel, the same way you use it when fetching a cubemap value.
• This buffer is always 1024 × 1024 × 6, whatever the window resolution (which can makes costly life icons).
• This buffer is stored as half-floats
• A reminder that texelFetch() don’t work on CubeMaps. ( but textureLod() and MIPmap do ).
• As for Buffers, MIPmap is off by default. See flags at texture binding as channelN.

Note that it is up to you to recompute the environment at every frame, or only once using if (iFrame==0).
Geeky detail: this is the only power-of-two buffer, meaning here dynamic MIPmap is not bugged. Enjoy 😉

So there are many potential use modalities: E.g.n as a spherical environment (possibly even built by ray-tracing at every frame), as 6 extra buffers (with the lovely properties mentioned above), or even as big bulk storage to encode anything, comprising volumes (e.g. 128³ RGBA interpolated volume, raw ~900³ binary voxels, or volumetric light or  fluid simulation) or video images (another).
Attention: if you just want to use it as “one 1024×1024 buffer”, do determine that you are in face #0 or the computation will been uselessly done in the 5 other faces too. E.g.: if ( dir.x > max(abs(dir.y),abs(dir.z)) )
The orientation of each face coordinates is strange, so better see the examples below.

• Using faces Id and face coords: Simple example , another . storing 96 images (e.g. video frames).
• Using only the 3D direction: another for mapping; complex example for precomputed environment

Common

This is a special convenience tab, which content is included in all the other tabs.
Useful to set up only once all your global consts, macros, and utility functions.

Attention: to avoid inter-buffers incoherence, most uniforms ( iChannel*, iResolution,  iTime, etc) are not directly available here (or pass them as function parameters).
But it can still work in macros, as code is inlined only once in Buffers. E.g. #define R iResolution.xy  is accepted ( if no regular function uses it in Common ) when vec2 iResolution.xy  is not.
Now, relying on macros functions instead of regular functions just because you need to use an uniform can be cumbersome. A tricks is then to use a macro just as translator, as here (at end of Common): 
type myfunc(type1 param1,...N, type1' uniform1,...M) { ... }
#define myfunc(param1,...N) myfunc(param1,...N, uniform1,...M)
A reminder that the type of iChannelN is sampler2D  .

Provided textures

Note that most of them (comprising volumes, Buffers and video) can be toggled between nearest / linear / MIPmap interpolation, and clamp / repeat warp flag.

Attention: black&white textures are now encoded as red texture. Use vec4(texture()) or texture().rrrr to get the corresponding RGBA. Note that you can use textureSize() to detected 1-channel textures.
Attention: textures often need a few frame to load, which might spoil custom initialization if done at iFrame==0. -> iChannelResolution[] stick to 0 if the texture is not yet ready (see test). Same for higher LODs in textureSize().
-> Safe texture-load-dependent initialization.

special textures :

• Keyboard (see above), webcam (video), mic(sound), soundcloud (sound)
• Buffers A,B,C,D (see above)
• Noise random texture (see below), Bayer texture (see below)
• Font texture (see below)

regular textures :

• Image textures: various colors or grey images of various resolution are provided.
  • some wrap continuously and some not.
  • .a is always 1.
  • Color space is sRGB: no transform required for display, but if you want to process computation on them (physical rendering, image treatment) you should convert them to flat before (pow(img, 2.2) ) and back to sRGB after (pow (img, 1./2.2)). Note that this can be approximated by img^2 and sqrt(img).
• Videos : time-evolving textures (sound is immediately played; the shader can’t access it).
  • They are rectangular: check iChannelResolution.
  • No wrapping: use fract if required.
  • No MIP-mapping.
• Nyan cat: 256 x 32. Stores a 8 frame cartoon animation. .a is defined.
• Cubemaps: encode environment as 6 maps. to be used with textureCube()
• Audio textures: (see below)

Noise textures

6 textures of random uniform values are provided: two 2D low-res, two 2D high-res, one 2D bluenoise, two 3D textures; 3 grey and 3 rgba.

• 2D Color texture: G and A channels are R and B translated by (37.,17.) , if no vflip. This allows to fake interpolated 3D noise texture. Ref example here.
• 2D blue-noise texture, i.e., with values “well distributed” rather than pure random (i.e., white noise). Examples of use here.
• 3D textures. Ref example here.
• A reminder that the uniform random value is at pixel centers, that are between-integer coordinates. Other uv values can be useful for nice interpolation, but are no longer uniform and lesser std-dev).
• These textures are in flat colorspace: ready to use for computation, but gamma/sRGB conversion required for faithful display.

Procedural noise (ex, with base noise=texture):
– ref shader here.
– see header comments for all the variants: classical vs simplex noise, value vs gradient noise, etc.

Bayer texture

 is an ordered Bayer matrix :

• It allows easy dithering and half-toning (just threshold it with the grey level), example here.
• It also provides a permutation table in [0,63], example here.
• This texture is in flat colorspace: ready to use for computation, but gamma/sRGB conversion required for faithful display.

Font texture

This special texture encodes 256  characters in tiles.

• It contains 16×16 tiles of 64×64 pixels.
  • int c=#ascii   is found at vec2( c % 16, 15-c / 16 )/16.
  • a reminder that #ascii = 64+#letter and lowercase = 64+32+#letter
• .x provides an anti-aliased mask of the characters.
• .w gives the signed distance to the character border.
• .yz gives the distance gradient.

More details here. Example & utils here and here and here and there.

Sound in (audio textures)

A provided music or a user-chosen music from SoundCloud can be used as texture.

• Texture size is bufferSize x 2. (e.g.,  512 x 2).
• index, .75 : music samples
  • It is a buffer refreshed along time, but the precise working seems very system dependent plus the synchronization with image frames is not guaranteed. So drawing the full soundwave or using a naive sound encoding of image won’t be faithful (or possibly only for you).
• index, .25: FFT of the buffer
  • x / bufferSize = f / (iSampleRate /4.)   , example here.
    iSampleRate give the sampling rate, but it seems incorrectly initialized on many systems: if precision is important, try manually 44100 or 48000.

Ref shaders: music, soundCloud, microphone.

Sound out (sound buffer)

Instead of being displayed (as for buffer “image”), the result is stored in a audio buffer.

• in: you have the choice between indexing your sound formulas with time or sample id. Time is easiest, but some precision is lost after a few dozen seconds (sometime noticeable in some sounds). Method to use sample id: here.
• out: x,y channels correspond to left and right audio stereo. FragCoord corresponds to the time sample. ( iSampleRate give the sampling rate, but it seems incorrectly initialized on many systems) .
• This buffer is evaluated once before the image shader run, then the music is played statically. So there is currently no way to interact between image and sound shaders. One can’t even read textures or access image related Uniforms parameters.

More here.

VR

If your system can display stereo, you can compute stereo shaders.
Just implement the additional adapted mainImage variant:
mainVR( fragColor, fragCoord, fragRayOrigin, fragRayDir )

More here.

Assert

This debug helper has been added in October 2020.
st_assert( cond [, alertId ] ) displays a color corresponding to alertId 0,1,2,3 if cond fails (default Id = 0).
NB: Alert Id 1 wins other alert 0, etc.
Two typical uses:
– make pixel red where expression(uv) reaches an invalid range
– make some red region when a uv-independent value reaches an invalid range.

Attention: it is only allowed in Image Buffer.

Reference example here. Utils to test several expressions here.
Since it allows to display colors even when visiting a deep function, it can also be twisted to display variables as viewmeters or numbers.