See Javadoc

The MaterialState of a Spatial defines how light interacts with the object. That is, how light will reflect, how light is emitted, etc. The main attributes of material are defined as the ambient color (this is the color of the object when no light is shining on it), diffuse color (the color of the object when light is evenly distributed, i.e. sunlight), specular color (the light reflected off the object when direct light is applied, i.e. spotlight), and finally, emissive color (the color the object gives off, i.e. glowing). Material also defines a shininess value, where the higher this value the shinier the object. A low shininess might be a material such as felt or wood. Where high shiniess might be metal.

Playing with the MaterialState allows for objects to take on a whole new level of realism. However, it must be mentioned that in order for a MaterialState to be used properly, it must be used in conjunction with a LightState.

Proper use of the MaterialState is first deciding what the material properties of the object are.

I want to model the properties of a Green Glass Orb with Redish reflectors. First, the ambient color I will set to a light grey. This is the color the orb will appear if no light is affecting it. Next, it's diffuse color will be the green color, as in my example, it's made from a green glass. The specular color will be the red, since I said there is some material contained in the glass that will reflect red when under direct light. Because my glass orb is magical, it will emit a slight green glow. Lastly, as it's glass I will give it a very high shininess value.

NOTE: The use of other states: AlphaState for the transparency of the glass, TextureState for the enviromental reflections, etc can compliment the MaterialState very well, and make a nice looking glass orb. As such the example below sets all alpha values of the colors to 0.5 to allow for future transparency.

The code to set-up the above example would be the following:

MaterialState ms = display.getRenderer().createMaterialState();
ms.setAmbient(new ColorRGBA(0.5f,0.5f,0.5f,0.5f));
ms.setEmissive(new ColorRGBA(0,0.75f,0,0.5f));
ms.setDiffuse(new ColorRGBA(0,1,0,0.5f));
ms.setSpecular(new ColorRGBA(1,0,0,0.5f));
ms.setShininess(100);
orb.setRenderState(ms);

I added a MaterialState but now the lighting doesn't seem to have any effect
If the Emmissive value of the MaterialState is too high it might actually be overpowering the lighting in the scene. If you don't need emissive don't use it.

I've set the alpha value of the MaterialState, but it's not showing as transparent
You still must use the AlphaState to enable transparency, the MaterialState values just describe how it is affected by the alpha, but doesn't actually turn it on.

I set the MaterialState, but my object is just a big white “blob”
Sounds like you didn't turn on lighting? LightState is required to activate MaterialState. That is, without lighting, the material properties have nothing to run against.