I've also struggled to get fannish about books, even for stuff that I think should lend itself really well to fannishness. I just can't get into their fics. But if a book series has a movie adaptation that I've seen, even if the fic isn't set in movie!verse, I don't have a problem (e.g., Harry Potter). It's so strange!