The comparison with the Windows virtual machine is complicated by the fact that there is an intermediate OpenGL abstraction layer in the virtual machine presented by VmWare. In any case, I ran GLView3.0 (http://www.realtech-vr.com/glview/) to see what OpenGL features might differ between the native Mac HD 5770 environment and the Windows virtual machine on the same hardware. We are trying to answer the question "Why does the the v3d 3d viewer freeze on the native Mac HD 5770 environment, but not on the virtual Windows 7 environment on the same hardware?." Here are the results of that comparison:

GLView reports that both environments support ALL features of OpenGL versions 1.1, 1.2, 1.3, 1.4, 1.5, 2.0, and 2.1. GLView also reports that both environments support NONE of the features of OpenGL versions 3.3, 4.0, and 4.1. That leaves only OpenGL versions 3.0, 3.1, and 3.2. These versions of OpenGL differ in the partial support of features reported by GLView between the native Mac HD5770 environment and the virtual Windows 7 environment on the same hardware.

GLView reports that the native Mac HD5770 environment supports the following features that are reported as unsupported on the Windows virtual machine:

OpenGL 3.0:
GL_ARB_color_buffer_float
GL_ARB_depth_buffer_float
GL_ARB_framebuffer_sRGB
GL_ARB_half_float_pixel
GL_ARB_texture_compression_rgtc
GL_ARB_texture_float
GL_ARB_texture_rg
GL_EXT_draw_buffers2
GL_EXT_framebuffer_multisample
GL_EXT_gpu_shader4
GL_EXT_packed_float
GL_EXT_texture_array
GL_EXT_texture_integer
GL_EXT_texture_shared_exponent
GL_EXT_transform_feedback
GL_NV_conditional_render
GL_NV_half_float
OpenGL 3.2:
GL_ARB_geometry_shader4

On the other hand, the Windows virtual machine claims to support GL_ARM_copy_buffer (OpenGL 3.1), while the native Mac environment does not. I am surprised to see the virtual machine claiming to support an OpenGL feature apparently not supported by the underlying hardware platform.

I'm not sure what to make of these results. Hopefully this can help shed some light on the HD5770 freezing issue on the Mac Pro.