Pre Vatican II the Priests faced the tabernacle (the box in which the consecrated host is kept) which was kept on the high Altar. In traditional symbolic thought this meant that the Priest and the Congregation celebrated the liturgy while facing God (in the form of the Eucharist) and while facing east (towards the rising sun). No one ever said the Church was subtle about their symbols. :D

Catholic women covered their head because St. Paul said to (1 Corinthians 11 - I think). Of course he also said that women should be silent in Church and if they had questions they should ask them of their husbands at home. Many, many of the Church's problems stem from taking St. Paul seriously.