I've worked for 3 startups.
The first was right when the Internet was going big (mid 90's). It was a decent idea but the company was a bunch of idiots. Bad engineering, bad management - then they hired some guys who were smart but didn't deliver. All talk and no product. I was in the middle of this, junior programmer with 2 years experience, but after 15 months it was clear where it was going, and I bailed.
2nd was with a friend's company. He'd just got VC money and hired some guys. His initial product was neat but I got on board when he did v2 and it looked really cool. It was cool at the time. Company had an "us against the world" feel. The engineering wasn't awesome, but I liked the team. The problems were that we were doing social networking really early on and weren't pushing the right buttons to engage users, and the founder didn't think big enough. We were handling a LOT of data - this is right when "big data" was starting up - we'd read the G-table and MapReduce papers and go try to talk to the boss about doing something like that, but he wanted to throw it all in an RDBMS and build a traditional 3 tier architecture. The funny thing is after a couple years, he started talking to MySpace and it was clear where this would all end up, both technologically and business-wise.
3rd company I came in late stage. 3rd round of VC funding I think. Still felt like a startup, huge growth, great team. Not the most talented group, but extremely dedicated. Eventually the company went public, we've done well. My options paid off well. Not Fuck You money, but it's a life-changing amount.
My general advice is
1. Know who you're getting in bed with. You will need to trust the team, and you need to be able to rely on their dedication. It's really hard being the only guy on a team who really gives a fuck.
2. Really, really believe in the product. You should be working on something that you would be using anyway. If it's some abstract thing that you don't care about, it won't work. You need to be involved at the level that you're online at 11PM using the product you made because you want to. So ask yourself - is this one of these "that sounds like a good idea" companies? Or is it a product that you want to go use right now if you don't already?
2a. That said, there are some great areas that you won't really care about unless you're a doctor or truck driver or whatever. You'll still need to build empathy with those people because the best software comes from people who are solving problems they're directly interested in. I think this kind of thing is really untapped, but there's a deeper level of networking involved. Side point - the 3rd company I worked for, when I was hired, if you worked there you either were related to the founder, went to church with someone who already worked there, or had extensive prior work experience with someone in the company. That company was very much organically built. They had minimal HR and did not use contractors or talent scouts.
3. Direction from the execs should be consistent. Startups will change direction for good reasons (I've heard a few stories of companies changing direction radically and doing well), but you have to watch out for the flavor of the week. People throw around ideas like "agile" thinking it's an excuse to not build a plan an execute it. Just be careful. The best founders have a great vision and don't usually change direction. I mean, if you start off building a Magic the Gathering trading card site and end up trying to do currency trading, it's probably going to end badly. A really early stage startup, you should be able to interview with the founder(s). Talk to these people, they should leave you feeling like you absolutely have to do this or you'll regret it. Their enthusiasm should be contagious.