I found the tower using the map (since quest guide doesn't seem to want to work for this), but it doesn't give me an option to enter the tower to fight him. How do I get in there?
Mr. Grimcaster has fooled the quest arrow into telling you that a side door is the main entrance.
You have to go around to the right of the building to where the sigils are. On your map, the map arrow will look like it's on the right of the circle representing Mr. Grimcaster.
If the sigils aren't lit up, then be sure you have the quest "You call this easy?". If you do and the sigils are still not lit up, send in a ticket to Support.