NIST conformance test suite results

Results of testing our engine on NIST Conformance Test Suite. NIST test suite is maintained and available currently on web3d.org. For historical purposes, you can also see at the old and original version of NIST VRML Test Suite (but the links to actual models are broken there).

Each test was classified as "pass" only if it passed fully. Which is a good objective measure, but also means that many tests failed because unrelated features are not implemented. For example, don't be discouraged by many failures in PROTO category. Prototypes were 100% working in all tests, and I consider their implementation as practically finished. But unrelated things like missing Script support for ECMAScript prevented the tests in PROTO category from passing completely.

The style of table below was modeled after similar page at OpenVRML Conformance Test Results.

Node Group Node Test Number Result Notes
Appearance Appearance 1 +
2 +
3 +
4 +
5 +
6 +
7 +
8 +
9 +
10 + You have to set RGB Textures Color Mode -> GL_REPLACE to get 100% correct result.
11 +
12 +
FontStyle 1 +
2 +
3 + Note that the test looks strange because the X axis line starts at X = -200. This is an error in the test file.
4 +
5 +
6 - For horizontal text test passes, but vertical text is not implemented yet.
7 - Handling ECMAScript not implemented yet.
ImageTexture 1 +
2 +
3 +
4 +
5 +
6 +
7 +
8 +
9 +
10 +
11 +
12 +
13 - The texture top is not aligned precisely with text top.
14 +
15 +
16 +
17 +
18 +
19 +
20 - Like case 13: The texture top is not aligned precisely with text top.
21 +
22 +
23 +
24 +
25 +
26 +
27 +
28 +
29 +
30 +
31 +
32 +
33 +
34 +
Material 1 +
2 +
3 +
4 +
5 +
6 +
7 + You have to set RGB Textures Color Mode -> GL_REPLACE to get 100% correct result.
8 +
9 +
10 +
11 +
12 +
13 +
14 +
15 +
16 +
17 +
18 +
19 +
20 +
21 +
22 +
23 +
24 +
25 +
26 +
27 +
28 +
29 +
MovieTexture 1 +
2 -Audio from MovieTexture is not played yet
3 -
4 +
5 +
6 +
7 +
8 +
9 +
10 +
11 +
12 +
13 +
14 +
15 +
16 +The movie text.mpg is still (5 identical frames, according to ffmpeg, gstreamer and xine)
17 +
18 +
19 +
PixelTexture 1 +
2 +
3 +
4 +
5 +
6 +
7 +
8 +
9 +
10 +
11 +
12 -Texture top is not aligned precisely with Text top
13 +
14 +
15 +
16 +
17 +
TextureTransform 1 +
2 +
3 +
4 +
5 +
6 +
7 +
8 +
9 +
10 +
11 +
12 +
13 +Results look a little different, but matching precisely Xj3D and OpenVRML results.
14 +
15 +
16 +
17 +
18 +
19 +
20 +
21 +
22 +
23 +
24 +
25 +
26 +
27 +
28 +
29 +
30 +
31 +
32 +
33 +
34 +
35 +
36 +
37 +
38 +
39 +
40 +
41 +
42 +
43 +
44 +
45 +
46 +
47 +Results look slightly incorrect, but matching precisely Xj3D and OpenVRML results. I think this is a shortcoming of my GPU (ATI Mobility Radeon X1600), precisely transforming small textures may make small errors?
48 +
...here I skipped some tests, to be checked later...
Grouping Transform all 25 + Note that "scaleorient Y" and "scaleorient Z" text descriptions are incorrect. Our results are correct (and match at least InstantPlayer).
Billboard all 6 + Note that "axis - rot Z" doesn't make much sense, when axisOfRotation is Z then rotating around it doesn't change the local Z vector, so the results are undefined... We do just nothing in this case. Results in other browsers (at least InstantPlayer) are weird, and it's also OK — it's undefined situation.
...here I skipped some tests, to be checked later...
Geometry Box 1 + This links to Text test, that passes (but has nothing to do with Box)
2 +
3 +
4 +
5 +
6 +
Cone 1 + This links to Text test, that passes (but has nothing to do with Cone)
2 +Again, tests linking to unrelated testcases for Box node (that pass)
3 +
4 +
5 +
6 +
7 +
8 +Unrelated Box test... (that passes)
Cylinder 1 + Unrelated Text test again...
2 +Unrelated tests for Box again...
3 +
4 +
5 +
6 +
7 +
8 +Unrelated Cone test...
9 +Unrelated Box test...
ElevationGrid 1 (default - grid) + Note that by default ElevationGrid is not smoothed (creaseAngle = 0), this is following the spec.
2 +
3 +
4 +
5 +
6 +
7 +
8 +
9 (test - normalfaces) + The reference image of the test is bad. The result should be more obvious (whole rows of quads have the same normal), and it is — with our engine.
10 +
11 + Although we use two-sided lighting.
12 +
13 +
14 +
Extrusion 1 +
2 + Reference images show the incorrect non-uniform scaling of the caps. We handle it right.
3 +
4 +
5 +
6 +
7 +
8 +
9 +
10 +
11 +
12 +
13 +
14 - While generally looks Ok, it seems that our triangulating algorithm can't handle this particular shape perfectly.
15 +
16 +
17 +
IndexedFaceSet 1 +
2 +
3 +
4 +
5 +
6 +
7 +
8 +
9 +
10 +
11 +
12 +You have to set RGB Textures Color Mode -> GL_REPLACE to get 100% correct result.
13 +
14 +
15 +
16 +
17 +
18 +
19 +
20 +
21 +
IndexedLineSet 1 +
2 +
3 +
4 + (These tests have nothing to do with IndexedLineSet, they are for IndexedFaceSet.)
5 +
6 +
7 + (These tests have nothing to do with IndexedLineSet, they are for IndexedFaceSet.)
8 +
9 +
10 +
PointSet 1 +
2 +
3 +
4 +
5 +
Shape 1 +
2 +
3 +
4 +
5 +
Sphere 1 - 3 (all) +
Text 1 +
2 +
3 +
4 +
5 -Text.length is not supported yet
6 -
7 -
8 -Text.maxExtent is not supported yet
9 -
10 +
11 +
12 -Texture mapping is a little incorrect, text is too small
...here I skipped some tests, to be checked later...
Misc EXTERNPROTO 1 +
2 +
3 +
4 +
5 +
6 +
7 +
8 - Currently, base URL for EXTERNPROTO is from the file where EXTERNPROTO is written, not from the file where it's instantiated.
9 - ECMAScript is not supported yet. Also, the DEF declaration inside a script causes known problem with cycles in VRML graph.
10 -
11 - ECMAScript is not supported yet.
12 -
13 +
14 +
15 +
16 +
17 +
18 +
PROTO 1 +
2 +
3 +
4 +
5 +
6 + (It links to unrelated Text test that works?)
7A + Result is Ok, but we do not handle Sound, AudioClip nodes (yet).
7B - Static result seems Ok, but we do not handle VisibilitySensor (yet).
7C - Static result seems Ok, but we do not handle VisibilitySensor and Collision.collideTime is not generated (yet).
7D +
7E +
7F +
7G +
7H + Result is Ok, although actually we do not handle ECMAScript (yet).
7I +
7J +
7K +
8 +
9 - Tested features work perfectly. But VisibilitySensor is not handled (yet), so animation doesn't start (you can replace it by e.g. ProximitySensor with large sizes, and animation will work).
10 +
11 +
12 +
13 +
14 +
15 +
16 +
17 +
18 +
19 +
20 +
WorldInfo 1 +
2 +
Sensors

(Actually tested on X3DV versions here.)

CylinderSensor all except below +
minmax disk ? We do honor min/maxAngle. However, this test uses min/max values of -2Pi/+2Pi, effectively allowing any rotation value. The test requirement ("one full rotation") assumes that browser calculates following rotations as angles outside of [-2Pi, 2Pi] range, which is not required by the specification (and not really clearly doable, when you receive discrete mouse events you cannot 100% reliably detect when "full rotation" occurs.)
multisensor - We do not "see" the higher TouchSensor so it doesn't even get isOver events.
PlaneSensor 1 .. 11 +
12 (multisensor) - We do not "see" the higher TouchSensor so it doesn't even get isOver events.
13 .. 14 +
ProximitySensor 1 .. 10 (all) +
SphereSensor all except multisensor +
multisensor - We do not "see" the higher TouchSensor so it doesn't even get isOver events.
TimeSensor 1 .. 19 (all) +
TouchSensor 1 .. 9 (all) +
VisibilitySensor 1 .. 9 (all) - VisibilitySensor is not implemented yet.
... here I again skipped some tests ...
Special_Groups LOD 1 +
2 +Note that switching between Viewpoints in these tests has very strange VRML code. Namely there are interpolators with two equal keys (so they don't actually make any change, and this is correctly optimized in the engine). Moreover, they are connected to time sensors with 2 seconds cycle. This causes strange effects when clicking fast on various touch sensors, as many interpolators conquer to change the same Transform.position values. I'll emphasize: we handle it correctly, and optimize correctly, we have to evaluate simultaneous changes to the same field from various routes... The test is just strange, without any purpose.
3 +
4 +
5 +
6 +
Switch 1 + This is actually an Anchor bound-500 test, that passes. (Possibly, wget messed up my local copy of NIST tests... Online server with NIST tests is broken, so I can't check).
2 +
3 +
4 +
5 +
6 +
Do you want to check the remaining tests? Please speak up on forum, we'll gladly finish this testing.