How could it be argued that guilt is the most important theme?

In the end, it is the narrator's own heart that he hears, not the old man's.  The old man has been dismembered and buried beneath the floor boards, and so the sound cannot be coming from him.  Instead, when the narrator hears the terribly loud heart beat as the police sit calmly in the room, you could argue that it is the result of his own anxiety and guilt.  Further, the narrator tries really hard to attempt to explain the necessity of committing this murder throughout the entire story, and it seems like a situation where he is somewhat defensive because he knows what he did was wrong.  Sometimes, when a person feels guilty about something, they go out of their way -- consciously or not -- to prove that whatever action they took was somehow justified.  It is entirely possible that this is what the narrator is doing.