If you wish to use material from the Library, you must first register as a reader, which is free of charge, and can be done on your first visit as long as you have personal identification and proof of address, and agree to abide by the terms of library. If you're not up for all this, you might be able to get a glimpse of the room by just wandering in, but remember this is a research space, rather than a tourist attraction.
Once this step has been taken, you can use the library, but only if you sit in your regulated seat, and use only material you have specifically requested from the desk on the appropriate forms. This is, after all, a serious library which is here for a serious purpose - to allow people to access the National Art Library Collections.
The library is open Tuesday - Saturday, 10am - 5.30pm (except Fridays until 6.30pm). For more, see http://www.vam.ac.uk/nal/about/index.html