In "The Minister's Black Veil," what does the minister say about why he wears the black veil?

In the story, Mr. Hooper never explicitly explains why he began to wear the black veil. The first sermon he delivers after he puts it on has to do with

secret sin, and those sad mysteries which we hide from our nearest and dearest, and would fain conceal from our own consciousness, even forgetting that the Omniscient can detect them.

Thus, the veil seems to be connected to this idea that each of us hides our sinful natures from others, though Mr. Hooper never verbally connects the two for his congregation.

Later, when Mr. Hooper speaks to his fiancee, Elizabeth, she ask him about he veil's meaning directly. Although he does not answer her directly, he does say,

If I hide my face for sorrow, there is cause enough. . . and if I cover it for secret sin, what mortal might not do the same?

Such a response to her queries connects the veil to this idea that we all hide something crucial of ourselves that prevents us from being truly known or understood by anyone else.

Finally, on Mr. Hooper's deathbed, when another minister suggests that he remove the veil, he says,

When the friend shows his inmost heart to his friend; the lover to his best beloved; when man does not vainly shrink from the eye of his Creator, loathsomely treasuring up the secret of his sin; then deem me a monster, for the symbol beneath which I have lived, and die!  I look around me, and, lo! on every visage a Black Veil!

Again, then, although he never comes right out and explains the meaning of the veil, Mr. Hooper connects it to this idea of secret sin one final time, insisting that the only thing that differentiates him from everyone else is that he has donned the physical veil as a means of representing the figurative veil we all wear, and for this reason he has been shunned.