I like both answers by Dan Wolfgang and Ross Millikan so far, but they're not complete. I'd like to try:
Length
As you already stated and Dan elaborated on, length provides speed and better tracking (keeping direction). Better tracking also improves stability. If waves hit you from a 90 degree angle (sideways) you can very easily capsize.
Hull shape
A flat, round shape of the hull often used in agile kayaks for small rivers makes the kayak more manouverable, provides less tracking, and thus makes life harder. There're so called first and second points of stability. The second point determines at which angle the boat will capsize. V-shapes are optimal for both tracking and a good second point of stability. However, they are bad on the first point of stability. What this means is that the kayak will immediately feel "tippy" and beginners feel uncomfortable in them because of fearing to tip over. But, the actual point of tipping over is often much further out in V-shaped kayaks than with other shapes on the market. Look at topkayaker.net for a good visual description, and wikipedia explanation of this.
Buoyancy
This is Ross' main point: The ability to stay afloat. This serves two purposes: It helps to stay out of the water, and once you've capsized the kayak must not sink, so you have an island of safety hopefully with the option of getting in again. As Ross pointed out, for this to work properly sea-kayaks must have at least two sealed compartments. When buying a boat (new or used) always test the compartments: Put some paper towels into all compartments, close all hatches, wade into shallow water, and push the bow under water and hold for a minute or so. It helps to put some weight into the compartments. Ensure the hatch is ~30cm under water level to build up some water pressure. Then repeat with stern. Then tip over the kayak so that the place where you'd normally sit in fills up with water. Again push bow and then stern under water. Pull the boat to shore, carefully open the hatches without spilling water from their seals or lids into the compartments, and check the paper towels and look inside the compartments. A few drops here and there are okay. Soaken paper or half a liter of water inside are not. If you capsize in the ocean, you do this for a reason: conditions are rough. So you boat must be able to support you for an hour or longer. If it's full with water, all compartments broken, it might sink, and you're in even deeper trouble.
Volume
Important for multi-day tours. Always pack lightly. Still you do need water, food, spare clothes, and some gear for sleeping & cooking. Whatever you take, put it into water proof bags ("roll bags") Sea water is agressive and even stainless steel will rust eventually. Depending on your need for "luxury" (chairs, some take foldable bikes, ...) you need a bigger boat. Packing stuff on top of the kayak is not an option because of
Wind
If your boat or your luggage provides too much of a silhouette for the wind to attack, this makes keeping your direction harder. You certainly will want a
Rudder
In whatever way this is implemented a sea-kayak should have some way to trim your direction. This can be the typical and simple external rudder mounted to the back with two pedals used as feet rests for steering and an additional rope to pull up or down into the water, or the older skeg that hides beneath the bow.
On rivers, some actually frown upon on people using a rudder instead of just adjusting paddle strokes and/or tilting/tipping the kayak to ease it around curves. However on the sea it's been used differently: Wind almost always moves you sideways to some extent, and so you already have to offset you direction of paddling to compensate and reach your wanted point of destination. To make this easier you use rudder or skeg to trim your boat. Depending on the length of the tour this is not just a matter of convenience but a matter of physical exhaustion and thus safety. Always take a boat with a steering/trimming facility. If you don't need it, it's ok. But on that particular day, you'll be happy you've got it.
Equipment (Compass, bilge pump, life jacket, paddle float, clothing)
Sea-kayaks are not complete without a compass and a bilge pump. If you know the area well you might get by without the compass. And some insist they never fall in and if they do their sponge is enough to get water out. If you've ever capsized and actually have managed to get in again, you know how long it takes to get all the water out with a sponge. So imagine handling the sponge with one hand, while trying to not tip over again with one-handed paddling. In rough water. 50cm wave. Gusts of wind. No fun. A manual bilge pump at least shortens the time. Several people build electrical pumps into their kayaks with the added problem of having to tightly seal off the battery and all electrical parts against sea water while at the same time allowing charging.
If you need to get back into your kayak, you need a paddle float (or be very very talented). It's a inflatable wrapper you put around one end of the paddle, pump it up by breathing into the valve. Then fasten the other end of the paddle beneath some ropes on your boat's deck and eh voila - you've got an outrigger. Practice getting out, and back in with your own kayak and paddle float. Once it works, do a practice run with friends on a small lake in adverse conditions. Ensure you're doing it where the wind pushes you against the shore, not away. Safety first!
A life jacket is essential to help keep you afloat should you not manage to get into the water. It's also practical because it helps keep you warm, they have pockets for quick release knifes or a camera. They have hooks and loops again for a camera, for emergency whistles or for attaching e.g. a cow tail with a line to be able to tow someone else.
User BitOff commented that gloves can be helpful to keep temperature and also reduce fatigue in the fingers and lower arms. Gloves also come in a variant that wraps around the paddle, so that your fingers have direct contact to the paddle, and everything is wrapped together (do an image search for "paddle gloves").
I'm not going in depth into all sorts of clothing issues. But I do want to point out that generally, for safety reasons, when paddling on open water (large rivers, any lake, or ocean) one should dress for keeping warm in the water. Often enough that means either Neoprene or a Dry-Suit is advocated. Hypothermia is a real risk to your life (read->understand->act). While paddling on our small river (almost no current, 5m wide) in autumn, winter, and early spring, the water temperature is close to 4 deg Celsius and thus I wear a drysuit, and a neoprene cap.
Anything else I forgot? Pls comment.
