Why Skinned Normals Are Approximate

We move vertices with bones, then do the same thing to normals and tangents. Seems reasonable. I would expect the lighting to follow the mesh. Let's test that assumption.

A normal tells lighting which way the surface faces. A tangent runs along it and helps orient a normal map.

The tip normal averages the three face normals equally. Its tangent comes from the UVs, then is made perpendicular to the normal. Move a neighbor: the tip can stay in place while its directions change.

In the comparisons below, the top-left view tests our current approach. The top-right is our ground truth: normals come from the final triangle edges; tangents come from those edges and UVs.

This reference averages neighboring contributions by area. Both sides use the same vertices and camera, so differences in color reveal differences in direction.

The bottom views show the selected method's normal error and tangent error against that reference. Green means zero error; red means 45° or more. All four views move together.

1. LBS positions only

LBS (linear blend skinning) is the usual default for skeletal animation in games. It mixes bone transforms using weights: how much each bone influences a vertex.

The maximum influences per vertex varies with the engine, platform, and asset settings. Four and eight are common limits; some paths support more. Unreal's rendering docs show these options.

Our sheet uses five bones and four influences per vertex, with weights summing to one. Let's move only its vertices and leave normals and tangents unchanged.

In the equations, A_i and t_i include the adjustment from each bone's bind pose.

1// Rigid, bind-corrected bone transforms; column vectors.
2float3 p = 0.0;
3[unroll]
4for (uint i = 0; i < 4; ++i) {
5    float4x4 M = boneMatrices[joints[i]];
6    p += weights[i] * mul(M, float4(restP, 1.0)).xyz;
7}
8float3 n = restN;
9float4 tangent = restT;

Turn Bone twist or Bone bend. The left side stays green while the reference changes color. That's expected: our unchanged normals still point upward. Tangents shows the same problem with the other direction.

2. Skin normals and tangents too

The obvious fix is to skin those directions too. Our bones are rigid: rotate normals and tangents, blend using the same weights, then normalize. Translation doesn't directly change a direction.

1float3 SafeUnit(float3 v) {
2    float len2 = dot(v, v);
3    if (len2 < 1e-24) return 0.0;
4    return v * rsqrt(len2);
5}
6 
7// Rigid, bind-corrected bone transforms; column vectors.
8float3 p = 0.0;
9float3 n = 0.0;
10float3 t = 0.0;
11[unroll]
12for (uint i = 0; i < 4; ++i) {
13    float4x4 M = boneMatrices[joints[i]];
14    p += weights[i] * mul(M, float4(restP, 1.0)).xyz;
15    n += weights[i] * mul((float3x3)M, restN);
16    t += weights[i] * mul((float3x3)M, restT.xyz);
17}
18n = SafeUnit(n);
19t = SafeUnit(t);
20float4 tangent = float4(t, restT.w);

That's better. The colors now follow the bones, with the same geometry as before.

But the heatmaps still show a mismatch. Neighboring vertices can follow different bones, changing the surface around this vertex. Our normal calculation doesn't account for that.

Even constant weights can blend rotations into scale or shear. One shared rigid transform is an exact special case; we'll look at the scale/shear correction later.

Kavan's SIGGRAPH skinning notes, section 6 explain this weight-gradient problem. Tangents also depend on how neighboring positions move.

3. Blend shapes without normal or tangent deltas

Now let's add a blend shape before skinning, storing only its position deltas. The bones still skin the original normals and tangents. I would expect trouble where the shape changes the surface.

1float3 SafeUnit(float3 v) {
2    float len2 = dot(v, v);
3    if (len2 < 1e-24) return 0.0;
4    return v * rsqrt(len2);
5}
6 
7// Rigid, bind-corrected bone transforms; column vectors.
8float3 morphP = restP;
9float3 morphN = restN;
10float3 morphT = restT.xyz;
11for (uint k = 0; k < targetCount; ++k) {
12    morphP += morphWeights[k] * deltaP[k];
13    // No normal or tangent deltas in this step.
14}
15 
16float3 p = 0.0;
17float3 n = 0.0;
18float3 t = 0.0;
19[unroll]
20for (uint i = 0; i < 4; ++i) {
21    float4x4 M = boneMatrices[joints[i]];
22    p += weights[i] * mul(M, float4(morphP, 1.0)).xyz;
23    n += weights[i] * mul((float3x3)M, morphN);
24    t += weights[i] * mul((float3x3)M, morphT);
25}
26n = SafeUnit(n);
27t = SafeUnit(t);
28float4 tangent = float4(t, restT.w);

Change Towel twist. The surface twists through the middle, but the left-side directions only follow the bones. That's wrong for the new shape.

Try Swirl, Middle fold, or Ripples too. The two heatmaps show that normals and tangents can disagree by different amounts.

4. Add normal and tangent deltas

Let's store normal and tangent deltas for each target too, add them before skinning, and see whether the two sides get closer.

1float3 SafeUnit(float3 v) {
2    float len2 = dot(v, v);
3    if (len2 < 1e-24) return 0.0;
4    return v * rsqrt(len2);
5}
6 
7// Rigid, bind-corrected bone transforms; column vectors.
8float3 morphP = restP;
9float3 morphN = restN;
10float3 morphT = restT.xyz;
11for (uint k = 0; k < targetCount; ++k) {
12    morphP += morphWeights[k] * deltaP[k];
13    morphN += morphWeights[k] * deltaN[k];
14    morphT += morphWeights[k] * deltaT[k];
15}
16 
17float3 p = 0.0;
18float3 n = 0.0;
19float3 t = 0.0;
20[unroll]
21for (uint i = 0; i < 4; ++i) {
22    float4x4 M = boneMatrices[joints[i]];
23    p += weights[i] * mul(M, float4(morphP, 1.0)).xyz;
24    n += weights[i] * mul((float3x3)M, morphN);
25    t += weights[i] * mul((float3x3)M, morphT);
26}
27n = SafeUnit(n);
28t = SafeUnit(t);
29float4 tangent = float4(t, restT.w);

With the default settings, they do. The geometry is unchanged; only the lighting directions improved.

Combine a few targets, though, and the errors can still be large. A triangle's normal comes from crossing its final edges:

n_face = normalize((p_1' - p_0') × (p_2' - p_0'))

Crossing blended edges and then normalizing generally differs from blending target normals and normalizing their sum.

Here, target directions come from numerical derivatives of procedural surfaces. They can differ slightly from our finite triangle mesh even at one full target, before blending or bone motion adds further error.

Making the tangent perpendicular to the normal gives us a usable frame, but doesn't guarantee either direction matches the surface.

5. Recalculate from the final mesh

Let's use the result itself. Once all vertices have moved, calculate each triangle's normal and UV tangent. Average adjacent contributions by area, normalize, and make the tangent perpendicular to the normal.

The UV bitangent gives us tangent handedness: the sign used to orient the normal-map frame.

1float3 SafeUnit(float3 v) {
2    float len2 = dot(v, v);
3    if (len2 < 1e-24) return 0.0;
4    return v * rsqrt(len2);
5}
6 
7// Gather after the final-position pass.
8float3 p = finalPositions[vertexID];
9float3 N = 0.0;
10float3 T = 0.0;
11float3 B = 0.0;
12for (uint j = 0; j < adjacentFaceCount; ++j) {
13    uint3 f = adjacentFaces[j];
14    float3 e1 = finalPositions[f.y] - finalPositions[f.x];
15    float3 e2 = finalPositions[f.z] - finalPositions[f.x];
16    float3 faceN = cross(e1, e2);
17    N += faceN;
18    float2 uv1 = uvs[f.y] - uvs[f.x];
19    float2 uv2 = uvs[f.z] - uvs[f.x];
20    float det = uv1.x * uv2.y - uv2.x * uv1.y;
21    if (abs(det) > 1e-12) {
22        float3 faceT = (e1 * uv2.y - e2 * uv1.y) / det;
23        T += faceT * length(faceN);
24        float3 faceB = (e2 * uv1.x - e1 * uv2.x) / det;
25        B += faceB * length(faceN);
26    }
27}
28float3 n = SafeUnit(N);
29float3 t = SafeUnit(T - n * dot(n, T));
30float hand = dot(cross(n, t), B) < 0.0 ? -1.0 : 1.0;
31float4 tangent = float4(t, hand);

Both top views now use the rebuilt frame, so their directions match and the bottom heatmaps turn green. That's expected. Use Deformation stage here to check each earlier approximation against the same reference.

A nondegenerate triangle has an exact face normal; smooth vertex normals depend on the averaging rule. Preserve hard edges, UV seams, and authored shading boundaries. A collapsed triangle has no unique face normal.

Our simple tangent calculation isn't MikkTSpace. For a normal-mapped asset, use a basis compatible with its bake. Degenerate UVs cannot give us a unique UV tangent.

On the GPU, final positions must be available before gathering neighboring triangles. Precomputed adjacency avoids floating-point atomics, but the extra passes and memory traffic explain why cached directions are appealing. NVIDIA's Deformers chapter discusses both approaches.

Can a better normal matrix fix it?

This is an alternative to the ordinary normal skinning above. Keep LBS for positions, but use the inverse transpose of its blended matrix for normals. It handles scale and shear without gathering neighboring triangles.

Our earlier shader uses the cheaper direct blend. Inverse-transposing the blended matrix adds work, requires a nonsingular matrix, and still misses changing weights. Recalculation instead derives directions from the final triangles.

Think of a ramp prop. Stretch it along the ground without changing its height. It gets less steep, so its normal must tilt to stay perpendicular.

The gold normals on the left were stretched like the geometry and lean the wrong way. The inverse transpose keeps the right-side normals perpendicular.

For axis scaling, it uses reciprocal scales: double the mesh's X scale, halve the normal's X component, then normalize. For a pure rotation, it gives the same rotation.

Let's apply it to the blended bone matrix. Positions and tangents keep their usual transforms; only the normal calculation changes. Click Normal to highlight it.

1float3 SafeUnit(float3 v) {
2    float len2 = dot(v, v);
3    if (len2 < 1e-24) return 0.0;
4    return v * rsqrt(len2);
5}
6 
7float3 InverseTransposeNormal(float3x3 A, float3 n) {
8    // HLSL indexing gives matrix rows.
9    float3 r0 = A[0], r1 = A[1], r2 = A[2];
10    float3x3 C = float3x3(
11        cross(r1, r2), cross(r2, r0), cross(r0, r1));
12    float det = dot(r0, C[0]);
13    if (abs(det) < 1e-12) return 0.0;
14    return mul(C, n) / det; // C / det = inverse transpose
15}
16 
17// Input attributes include any preceding morph deltas.
18// Bone matrices include bind-pose correction; column vectors.
19float4x4 M = (float4x4)0;
20[unroll]
21for (uint i = 0; i < 4; ++i) {
22    M += weights[i] * boneMatrices[joints[i]];
23}
24float3 p = mul(M, float4(inputP, 1.0)).xyz;
25float3x3 A = (float3x3)M;
26float3 n = SafeUnit(InverseTransposeNormal(A, inputN));
27float3 t = SafeUnit(mul(A, inputT.xyz));
28float4 tangent = float4(t, inputT.w);
29// Tangent handedness is retained, assuming no orientation reversal.

The input attributes include any morph deltas. The zero fallback avoids inverting a collapsed matrix; it doesn't produce a valid normal.

At a character's elbow, one vertex mostly follows the upper arm; its neighbor follows the forearm. Their different blended matrices change the triangle between them.

The left already uses inverse transpose, yet its normals disagree with the rebuilt mesh. Each calculation sees its own vertex's matrix and misses how the influences change across the triangle.

An even simpler test: leave one bone still and translate another upward. With weights changing along the strip, the surface slopes upward even though neither bone rotates.

Both linear bone matrices are identity, so their blend and its inverse transpose are identity too. The left normal still points upward. That's wrong for the sloped surface.

An analytic solution needs the Jacobian: a local description of how nearby points move, including their changing weights. The inverse transpose applies to that fuller description.

With blend shapes, it must also track the morph before skinning. That gives us another way to account for the changing surface, provided we have a continuous model of the shapes and weights.