Maybe the prudent approach is to create a parallel system which includes all the newer technologies. Instead of an all or nothing transition, transfer one (or a few) functions at a time. If something goes wrong, you have a failback. If things work well, you can speed up the transition until everything is moved over.

There's no reason that Wordpress couldn't handle this, but it's true that you may have outgrown your current setup. I recently migrated a site with 30K users and 19K articles with legacy 301s for all of them. It was cumbersome, but doable.

You may want to use dedicated hardware, but it all depends on the load and it's probably not necessary. This is definitely the kind of work you want to pay to have done. If you DM me I'll connect you with a firm I worked with to do mine.

Good Answers here. I would agree that WordPress should not have trouble scaling with your needs if setup properly. You may also try cloning your site for testing purposes to revisit how your content architecture is structured vis-a-vis your various content types and add-on technologies (data, d3.js, ads). Doing this both from a front-end perspective and backend editorial workflow would probably yield productivity gains on both human and server/machine sides. Even if you decide to leave WordPress the the exercise would probably still yield useful lessons for your new choice of CMS.

Throwing my hat in the ring and hopefully give you something of substance in the process.

Q: what provider are you running your VPS on and how?
Ex: WHM with cPanel and Wordpress 32GB RAM 4 CPU @ 2.8Ghz
A: we use GCE google Compute Engine, pay by the drink and zero slow down.

Q: Have you considered splitting things up? User Management server (AD ish), SSO to Wordpress & other DB servers, with hand offs and API calls to each other?
A: having each VM or VPS run side by side in a VPC will make for quick light work.

Q: you mentioned legacy Wordpress... Older version or??
A: We don't have any clients with WP issues on speed, it's usually the horse power of the VPS / VM and potentially plugins that are not quite up to par for a use case. Running WP on a pure LAMP makes for huge differences in performance.

Over all, I would recommend isolating servers and use them for specific tasks only and let the VMs do that one job very well. API to communicate between the VMs as needed.

Lastly, as for the ad engine - you may wan to look into an OpenX server.