For atmosphere, I like to read books set in the country I'm in. It's lead me to read some classics I wouldn't have read, and discover some new authors.
I love it when you hit a street that's mentioned in the book, or you get a feeling of the climate described.
I wouldn't class these as travel books, though I guess some have been.
There are obvious ones, like Phantom of the Opera (Paris) and Singapore Grip (or King Rat) but there are sites that you can google (Books set in...). Or books written in a city like Maugham writing in Raffles, or Hemingway in Paris....