Normal/Specular Map issues (on model)

Alright, so I made a normal map. Then I made a specular map and added it as an alpha channel to the normal map. The problem is, an area of the specular map that I had as black (non reflective, yes?) but that also has some normal mapping, ends up reflective in game. If you need the images in question, I'll post them upon request.