Glyph Imaging


Materials and methods for 3d mouse ear visualization


AVI stereo movie of mouse ear - 441kb
QUICKTIME stereo movie of mouse ear - 689kb

    In order to better visualize the structures in the inner ear of mice, the 3d animation software Alias Wavefront | Maya was used to construct 3d models from light micrographs of sequential sections through ear tissue. Maya is not generally used as scientific visualization software but its powerful scripting environment (Maya Embedded Language, or MEL), variable tolerances for model precision, and high quality renderer make it very well suited to this purpose.

microscopy / image preparation

    First, digital micrographs were captured of the sections. In this case, many images were taken of a single section and then montaged together to create one single hi-res section image, approximately 3000 pixels square. This was repeated for each of 106 sequential sections through the tissue. For this project we wanted the option of viewing the macroscopic features and then zooming in on cellular detail in the 3d movie, so high resolution section images were used. This may or may not be necessary for other applications. A rule of thumb for image resolution is that one wants to end up with more or less square 'voxels' (3d pixel-cubes) in the final 3d output, so the pixel size in the section images should be approximately equal to the distance between sections plus the section thickness. For example if the sections are 1 micron thick and every section is to be used, then the distance between sections (1) plus the section thickness (0) is 1 micron. If the tissue sample is 200 microns in diameter then the section images should be approximately 200 pixels wide. Section pixel size can be made smaller than but shouldn't exceed the section thickness. Final output movie resolution should also be considered when choosing source image resolution. If the final movie is to be on video (640x480) then using 200x200 pixel section images may cause the final render to look pixellated. Another general rule for section image resolution is that it should be higher than the final output movie resolution, as much as 2-3x higher to avoid pixellation. This may cause the section image pixel size to be larger than the section thickness, but that will be ok. Resolution can always be reduced if necessary but not effectively increased once the micrographs are taken. An image of a micron calibration slide was taken at the same magnification as the rest of the section images to calibrate the 3d scene units according to real-world units.

    Once the section images were montaged, they were copied and these new images were hand colored individually to highlight specific features that were to be visualized separately in the 3d output. Hand coloring the sections allowed them to be batch processed using Adobe Photoshop to separate out the different colored features into batches of images with only those specific features present. Contrast and saturation of the section images was also tweaked to provide optimum translucence for the 3d model, so that the tissue did not appear as a solid mass because the value was too opaque, and alpha channels were created to mask the textures in Maya.

from 2d to 3d

    The section images were colored as follows: Arterial lumen was colored red, sensory patches in the vestibular apparatus were colored purple, coclear duct was yellowish, and the rest was left blue, the color of the stain. Some of the colors changed in the 3d texture processing, however. In the final output the arterial lumen reconstruction became a deep red with some cyan on the surface, sensory patch reconstruction became ivory, and the coclear duct reconstruction became a yellow-green. Uncolored features reconstruction remained similar in color. All these separate movies were composited in Adobe AfterFX to a single reconstruction movie with all the above features. The above movies are of the first control ear. Below are the same links and links to KO images.

Control and ko ear avi movies:

control ear
ko ear

...The 3d process used was to stack the sections as textures with alpha transparency on planes. The main hurdle to overcome when converting planes into 3d images was the 2-dimensionality of the planes causing them to be disappear when viewed directly from the side. The first idea was to use a shader glow in Maya instead of standard color mapping for the section image texture. However, since shader glow is still dependent on how much of the plane is visible the glow still disappears when the plane is parallel to the camera or incident angles. The glow provided a nice luminous transparent effect and helped bring out some subtle color differences in tissue. Two solutions were found to overcome the 2d plane issue. unfortunately both were very cpu intensive at render time. The method used in this example is to apply a geometry displacement map with a random 3d noise texture to all the frames, providing a 3d grain effect similar to film grain, so that no matter which way the plane stack is viewed the planes one can never see through to the other side because the surfaces have small random bumps on them, equal to the distance between planes. The number and smoothness of these simulated grains will of course have a direct effect on render time. A compromise was reached for this project which fell short of completely eliminating the dimming at incident angles but did improve it significantly. Process optimization, budget, and available hardware will dictate the quality of the final output.
    Another, perhaps better solution which was explored was to take this 3d grain idea one step further and create a 3d particle system for each section with randomly generated particles aligned to those section planes. These particles were then mapped with the section images and alpha channels. This would allow for very accurate ray-traced shading of the features rather than the glow emission used in our example. To match the resolution of the section planes, unfortunately, will require approximately 10,000 texture mapped particles per plane, times 106 sections in this example equals 1,060,000 particles, times 40 frames for the rendered movie, times 4 different movie layers for the different ear features. With all these multipliers the render time on a standard 3d workstation quickly becomes inordinate so this method was shelved pending a major hardware upgrade.

    Review of Meatmorph software from Meta Imaging

...Meta Imaging's Metamorph is a scientific imaging application that is designed to do what we did here with Maya. I tested a copy (briefly) and made this 3d reconstruction with our source images. Render time was a fraction of Maya's, but the output quality was slightly lower. Perhaps with some color tweaking image quality would improve.
...Meta Imaging dealt with the 2dness of the planes in a different way than we did. There is a blur or some similar effect applied to the color in a direction perpendicular to the planes. When the planes turn just a little bit to the side they will start to blur, (to observe this stop the 3d recon movie at approx. 90) so you only really see the finest detail when looking at the stack head-on. At a glance I think its a decent easy to use program, at least the 3d reconstruction part, which is all I looked at. Considering Maya complete is a fraction of the price of Metamorph and much more flexible, especially if you're working with any developers, we will continue working with Maya and developing our particle system templates for 3d visualization from sections.

    For more information on this process, to purchase templates, or to arrange a consultation please email