Visit him and talk with him to start borrowing books.
The basic principle is that the first time you speak with him he will let you borrow 1 book. Once you've walked around enough different rooms and areas in the castle, and talk to him again, he will let you borrow another, for total of 2 books. And same again for walking around more different rooms and areas, for a total of 3.
Entering new areas/biomes enough times and go back to OD and talk with him again. It seems the maximum needed area/biome change is 2, and you can do this with just using the teleport.
This means you can easy get all 3 books before fighting Andy.
Not a bug, NG+ will always set your books to 2, from the first time you talk with OD, and this can't be changed/increased.
Defeating the optional boss OD, will let you take all the books. (21)
Bugs:
Straight forward, as you uncover more map percentage, OD will have new books that you can borrow. This is completely independent of the number of books you can borrow at the same time (above). You could have 100% map completion, and still only be allowed to borrow one book at a time (Through either avoiding OD or a bug).