HTML5 Video Tag
Others have described well the issues with the video tag available in browsers supporting HTML5. For browsers not supporting the video tag, there are tricks to fall back to Flash to play video, so that the vast majority of users across browsers and platforms will see video, if the video is coded in such a way that the browser/system can decode it. Unfortunately, due to a history of licensing issues around H.264 codecs and the like, the different browsers support different codecs. The good news is that the video tag allows you to specify multiple sources, so that a browser will progress through the chain until it finds one it can play. The bad news is that to cover most browsers/systems, you will need to encode your video in at least two formats. Here is some advice that helped get a Summer 2013 ETC project going with respect to video playback on the web using the video tag.
The problem of getting video to work well everywhere has been around for awhile, e.g., see this 2010 blog post. Over time, a set of tools developed. Rich Shupe's 2012 two-part article at Part 1: Deliver Video Seamlessly to Desktop and Mobile Devices and Part 2: The Top 5 Cross-Platform Video Players provides an updated context. From there, you can read what formats are supported by browsers and why, and be led to some tools. Brightcove has some documentation on Encoding for Mobile Delivery.
How to Use Certain Encoders
Producing WebM and H.264 video with AAC audio are likely steps to cover most of the browser/platform systems. There's mixed advice on how to best do this, as the nature of the video (e.g., head shots or action footage), and planned viewing (typically window within browser, or full-screen) changes expectations. Here's a mixed bag of advice:
The future of html video may also allow for greater accessibility with features like timed text or perhaps synchronized multimedia imagery. The W3C long ago had a recommendation for the smil to do similar work, but now it appears that there is enough vendor interest and definitely enough video on the web to really advance the work. Keep watching for updates with support for features like webVTT for timed text, e.g., here's June 2012 musings on future of HTML5 video.
Issues are solved all the time within the frameworks like VideoJS, but if you run into specific problems on a particular device, there may be older posts that still help, e.g., iPad video tag issues.