What to actually evaluate
Once you're comparing serious providers, the marketing pages all sound identical. These are the things that separate them in practice.
Server response time under load, not at rest. Any host is fast at 2 a.m. Ask how the platform behaves with a warm cache, a realistic catalog, and a few hundred concurrent sessions, and run a load test before committing. For Magento specifically, ask about PHP worker counts and database sizing; those two numbers decide how many simultaneous shoppers you can serve before the queue starts.
Caching architecture. A properly configured full-page cache and a CDN in front of it absorb the majority of anonymous traffic before it ever reaches your application. Providers that treat caching as your problem rather than part of the product tend to be cheap for a reason.
Scaling for peaks. What happens on Black Friday: does capacity scale automatically, is it a support ticket, or is it a contract renegotiation? The honest answer tells you a lot.
Security and recovery. PCI-compliant handling of payment traffic, a web application firewall, automated offsite backups, and a tested restore time. Ask how long a full restore takes; "we have backups" and "we can restore in 40 minutes" are very different answers.
Support that knows your platform. Generic hosting support reads server graphs. Platform-specific support knows that a Magento indexer lock or a runaway cron job looks like a hosting problem but isn't. That difference is worth real money at 3 a.m. It's also the layer we cover for clients through Magento Care+ when the host's own support stops short.