Difficult question. Mostly I view manga and anime differently, especially when there are adaptations from one another, since it is almost impossible to do a better adaptation than the original work, be it manga or anime, or in some cases, light novels.
I love the animes, but usually the mangas are my favourite, since they are (mostly) the original work and translate best the authors messages. Yes the animes are a good introduction to the story, a sort of the taste for the real thing, when they are not completely AU from the manga storyline. Then I kinda view them as official fiction work, since they use the 'image' of the characters, but not the story/personality.
As stated before somewhere and in diferent ways, it really depends on the manga/anime, and mostly for me, the storyline.
If it hooks me from the beginning, even if the manga plotline is different, I will defenitly watch the anime, because of my love of the manga, and I may happen to love the anime as it is made. (even if it is by being different from the original plot).
If this isn't a direct answer... sorry about that, but it is the most honest one I have.