Robert Cotton liked collecting books, including manuscripts from the generations prior to his.
How rare were books? Who had them? What would constitute "a lot" of books?
We know of about 76,000 wills that survive from the 14th and 15th centuries. You expect wills to be made by people who had items of value that were worth disbursing to specific people. An examination of one-tenth of these wills reveals that only 388 mention books. Presumably, books would be mentioned specifically, given the care and expense they represented. But there were books around, so who had them? Here's one case of a private library.
The Chaucer scholar Derek Brewer tells us about William Ravenstone, a schoolmaster at the Almonry* Cathedral School of St. Paul's in London. Ravenstone had 84 books, which was an extraordinary number for a private library. He had Latin books on grammar, poetry, mathematics, music, and various Roman authors. In 1358, his will left them all to the school (where some argue the students would have access to them).
Thirty years before the Ravenstone collection was left to the school, it received the collection of a previous almoner, William Tolleshunt. His library included books on logic, grammar, natural history, medicine, civil and canon law, and theology.
Winchester College in 1446 and Eton College's charter written in 1440 (Eton opened in 1443) established libraries. At Eton (as at other libraries of the time), the books were chained to desks so that they could be read but not taken away.
Each college required only a single room to house all their books.
*An almonry was a place where alms were given out to the poor; those who worked there distributing alms were called "almoners."