The more expensive the line the less likely you are to find children aboard. Silversea, and Crystal are almost always childfree, as are Windstar, Seabourn and Oceania and Azamara. Going during term time always helps in keeping the blighters away too....
Saga is exclusively for over-55s if you are of that age bracket.
Thomson Calypso is adults only ....and they have some weeks when other ships are adults only as well...I think I read somewhere that Spirit or Celebration was adults only in June and September so worth checking their website.
RCI & Celebrity do the odd over 55s but any cruise out side the school holidays would have only a small amount of kids on board.