Try visiting the OpenH323 Project. They provide an OpenSource H.323 protocol stack which is great if you programme in C++. The documentation is pretty bad, but there is a good mailing list for support.
Also, a good reference book on H.323 and SIP is Carrier Grade Voice over IP by Daniel Collins.