I think he might have suspected Linus, yeah, not knowing that Linus had stolen part of Sinistra's sculpture as his challenge-mandated theft. And he'd assume that, firstly, Linus wouldn't take anything either dangerous or irreplaceable, and secondly, that the "return my books you BOOK THIEF" compulsion would work especially well on Linus.